Information processing device, information processing method, and program

Abstract

To provide an information processing device with which it is possible to determine the object to be or not to be detected, in a camera video installed in a store, etc., without using bodily features.SOLUTION: An information processing device 100 comprises: a human body detection unit 104 for detecting a human body 311 included in an image 500; a human body tracking unit 105 for tracking the human body 311 detected by the human body detection unit 104 and thereby measuring the staying time of the human body in a prescribed region; a setting unit 106 for setting a condition related to the object of motion detection; and a subject determination unit 107 for determining whether or not the human body 311 should be the object of motion detection, on the basis of the staying time measured by the human body tracking unit 105 and the condition set by the setting unit 106.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to an information processing device, an information processing method, and a program. [Background technology]

[0002] There is a system that detects a subject based on its characteristics and determines whether or not to notify the results. Patent Document 1 discloses a system that acquires an amount of activity of a subject in an image, and determines whether or not to notify judgment result information about the subject based on the amount of activity of the subject. Furthermore, Patent Document 2 discloses a system that acquires color information of a subject in an image, and judges whether or not to notify judgment result information about the subject based on a comparison with a pre-specified value. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Patent Publication No. 2021-103360 [Patent Document 2] JP 2020-123246 A Summary of the Invention [Problem to be solved by the invention]

[0004] However, the systems described in Patent Documents 1 and 2 may not be able to make appropriate distinctions in stores such as convenience stores and restaurants because there is little difference in the bodily characteristics (activity level, clothing, etc.) of people to be detected (such as customers) and people who are not to be detected (such as employees and store clerks).

[0005] The present invention has an object to appropriately set a detection target for a person in a camera image. [Means for solving the problem]

[0006] In order to solve the above problems, an information processing device in an embodiment of the present invention is characterized in that it includes a human body detection unit that detects a human body included in an image, a human body tracking unit that measures the time the human body stays in a specified area by tracking the human body detected by the human body detection unit, a setting unit that sets conditions related to the target of behavior detection, and a subject determination unit that determines whether or not to make the human body a target of behavior detection based on the time spent by the human body tracking unit and the conditions set by the setting unit. Effect of the Invention

[0007] According to the present invention, a person in a camera image can be appropriately set as a detection target. [Brief description of the drawings]

[0008] [Figure 1] 1 is a diagram illustrating a functional configuration of an information processing device 100 according to an embodiment. [Diagram 2] 1 is a diagram illustrating a hardware configuration of an information processing device 100. [Diagram 3] FIG. 2 is a flowchart illustrating an information processing method according to the embodiment. [Figure 4] FIG. 11 is a diagram showing details of the setting step S301. [Diagram 5] FIG. 11 is a diagram for explaining details of a human body detection step S303 to a determination step S308. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0009] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. However, the present invention is not limited to the following embodiment. In each drawing, the same members or elements are given the same reference numerals, and duplicated descriptions are omitted or simplified.

[0010] In this embodiment, a human body is detected and tracked from video (images) from a camera installed in a store, etc., and based on the aggregate value of the corrected (weighted) dwell time, it is determined whether or not to include the detected human body as a behavior detection target (notification target).

[0011] [Information processing device] FIG. 1 is a diagram showing the functional configuration of an information processing device 100. As shown in FIG. The information processing device 100 includes a communication unit 101, an image acquisition unit 102, a preprocessing unit 103, a human body detection unit 104, a human body tracking unit 105, a setting unit 106, a subject determination unit 107, a behavior detection unit 108, a behavior notification unit 109, and the like. A network camera 111 is connected to the information processing device 100 via a network 110. At least one network camera 111 is connected to the information processing device 100.

[0012] The communication unit 101 receives a signal from a network camera 111 via a network 110. The signal from the network camera 111 includes an image captured by the network camera 111.

[0013] The image acquisition unit 102 decodes the video data included in the signal from the network camera 111, and performs white balance adjustment, gamma processing, noise reduction processing, etc. on the decoded video data. The image acquisition unit 102 stores the processed video data in an external storage device 214, which will be described later. The image acquisition unit 102 also provides the processed video data to the pre-processing unit 103. The preprocessing unit 103 performs preprocessing on the video data received from the image acquisition unit 102 so as to facilitate detection and tracking of a human body from images of the video data. The preprocessing includes, for example, grayscaling, threshold processing, and filtering.

[0014] The human body detection unit 104 detects a human body from each frame image (hereinafter also simply referred to as an image) constituting the video data processed by the preprocessing unit 103. Furthermore, the human body detection unit 104 sets a human body region in the image based on the discrimination result. The human body tracking unit 105 tracks the human body detected by the human body detection unit 104 by associating the human body with the human body between multiple frames. The human body tracking unit 105 also measures the time the tracked human body stays in a predetermined area. The predetermined area in this embodiment is assumed to be an area within a store where the network camera 111 is installed.

[0015] The setting unit 106 sets conditions related to the target of behavior detection in accordance with an input operation from the input device 120 . The setting unit 106 sets, as conditions, a specific area requiring correction of the stay time, a correction value for the specific area, and a threshold value for the aggregate value after correction of the stay time. The specific area is, for example, a customer area (customer area 501) or an employee-only area (employee area 502). The correction is, for example, weighting. Furthermore, the setting unit 106 sets, as thresholds, one or both of a first threshold for making a behavior detection target and a second threshold for making a behavior detection non-target.

[0016] The subject determining unit 107 calculates a corrected total value of the stay time (weighted stay time) based on the stay time of the same person associated by the human body tracking unit 105 and the conditions set by the setting unit . If the threshold is exceeded, the body is determined to be a non-target human body, and the body is treated as a target human body until the threshold is exceeded. Conversely, if the threshold is exceeded, the body is determined to be a target human body, and the body is treated as a non-target human body until the threshold is exceeded.

[0017] The behavior detection unit 108 performs behavior detection on the human body determined as the detection target by the target person determination unit 107. The behavior detection includes, for example, shoplifting detection, crowding detection, fall detection, and passage detection. The behavior detection unit 108 may perform the detection for all human bodies detected by the human body detection unit 104, and select the detected behavior of the human body determined by the subject determination unit 107 as a subject.

[0018] The behavior notification unit 109 issues a behavior notification based on the detection result of the behavior detection unit 108. For example, when a behavior such as shoplifting is detected by the behavior detection unit 108, the behavior notification is issued. The behavior notification is, for example, a voice notification. A display device 121 may be provided to perform email notifications, push notifications, and the like to the display device 121.

[0019] FIG. 2 is a diagram showing a hardware configuration of the information processing device 100. As shown in FIG. The information processing device 100 includes a system bus 210, a CPU (Central Processing Unit) 311, a RAM (Random Access Memory) 212, a ROM (Read-Only Memory) 213, an external storage device I / F 214, an input / output I / F 215, a network I / F 216, and the like.

[0020] The CPU 211 is connected to a RAM 212 , a ROM 213 , an external storage device 214 , an input / output I / F 215 , and a network I / F 216 via a system bus 210 . The CPU 211 is a device that performs overall control of the information processing device 100 and performs calculations and processing of data. The CPU 211 controls each part of the information processing device 100 according to a computer program stored in, for example, a ROM 213, using a RAM 212 as a work memory. The CPU 211 can be configured with one or more processors.

[0021] The RAM 212 is a volatile memory, and is used as a temporary storage area such as a main memory or a work area for the CPU 211. The ROM 213 is a non-volatile memory, and image data and other data, various computer programs for operating the CPU 211, and the like are stored in respective predetermined areas. Note that the computer programs are not limited to being stored in the ROM 213, and may be stored in the external storage I / F 114.

[0022] The external storage I / F 114 is configured by, for example, a magnetic disk such as a hard disk drive or a flash memory, etc. The external storage I / F 114 stores application programs, control programs, an OS, related programs, and the like. The external storage I / F 114 can read and write data under the control of the CPU 211. The external storage I / F 114 may be used in place of the RAM 212 or the ROM 213.

[0023] The input / output I / F 215 is an interface for receiving operation input from a user via an input device and outputting data to an output device. The input device can be configured as, for example, a keyboard, a mouse, a touch panel, a button, or a voice. The output device is a display that displays a received image, an image stored in the external storage I / F 114, or a predetermined input screen, and is configured as, for example, a liquid crystal panel.

[0024] The network I / F 216 is a communication interface for communicating with the network camera 111 via the network 110 under the control of the CPU 211 . When the network camera 111 is connected to a wired network, the network I / F 216 includes a wired communication module for a wired connection, which enables communication with other devices via one or more external ports. When the network 110 is a wireless network, the network I / F 216 includes a wireless communication module having well-known radio circuitry, including an antenna system, an RF transceiver, one or more amplifiers, and so on. Network I / F 216 may also include various software components that process data.

[0025] A network camera 111 is connected to the information processing device 100 via a network 110. At least one network camera 111 is connected. The information processing device 100 processes an image signal supplied from a network camera 111 via a network 110 . Furthermore, the information processing device 100 supplies signals for setting and controlling the network camera 111 to the network camera 110 via the network 110. The network 110 may be either a wired network or a wireless network.

[0026] [Information processing method] FIG. 3 is a flow chart illustrating the information processing method. The information processing method is realized, for example, by executing a computer program by the CPU 211. When a signal including moving image data is transmitted from the network camera 111 to the information processing device 100 via the network 110, the information processing method starts.

[0027] The information processing method includes a setting step S301, an image acquiring step S302, a human body detecting step S303, a human body tracking step S304, an area specifying step S305, a staying time calculating step S306, a threshold comparing step S307, and a determining step S308. The area specifying step S305 to the determining step S308 correspond to the target person determining step of the present invention.

[0028] First, in a setting step S301, in accordance with an input operation on the input device 120, the setting unit 106 sets conditions related to the target of behavior detection.

[0029] FIG. 4 is a diagram showing details of the setting step S301. The setting unit 106 sets, as conditions, a specific area requiring correction of the stay time, a correction value (weighting) for the specific area, and a threshold value for the aggregate value after correction of the stay time. Here, an example of setting the same value across the board will be given, but the value may be set according to the type of behavior detection (shoplifting detection, crowding detection, etc.).

[0030] 4(a) shows an example of a screen for setting information related to subject determination. This screen is displayed on the output device by the setting unit 106 via the input / output I / F 215. The setting dialog 400 displays radio buttons 401 for specifying a target threshold (first threshold) and a non-target threshold (second threshold), and a threshold input field 402. One or both of the first threshold and the second threshold can be set. It also includes a check box 403 for setting whether or not to use a specific area that requires correction for the dwell time, a specific area name input field 404, an input field 405 for a correction value for the specific area, an area setting button 406, a delete area button 407, and an add area button 408. The employee area and customer area are input in the specific area name input field 404. In the correction value input field 405, 10 is input for the employee area and 0.4 is input for the customer area.

[0031] FIG. 4B shows an example of an input screen for setting a specific region. This screen is displayed by the setting unit 106. The image is displayed on the output device via the input / output I / F 215. On the specific area setting screen 420, an image captured by the network camera 111 or an image read from the external storage device 214 is displayed. The specific region setting screen 420 includes a human body region 421, a cursor 430, and a setting area 431. Furthermore, the specific region setting screen 420 includes an OK button 430 and a cancel button 441 at the bottom.

[0032] The human body region 421 is a region set by the human body detection unit 104 . The cursor 430 is displayed on the specific region setting screen 420, and is operated by the user using a mouse or the like of the input device 120. Using this mouse, one or more specific regions are set at any position (corresponding to a pixel of the displayed image) within the specific region setting screen 420. At this time, the user can set the specific region by referring to the human body region 421. When the user presses an OK button 440, various information and conditions related to the specific area are set, and when the user operates a cancel button 441, the various settings and conditions are cancelled.

[0033] 3, in an image acquisition step S302, the image acquisition unit 102 receives moving image data from the network camera 111 via the network I / F 216. The image acquisition unit 102 performs predetermined image processing on the moving image data, and provides the processed moving image data to the pre-processing unit 103. Then, the pre-processing unit 103 performs pre-processing on the video image data received from the image acquisition unit 102 .

[0034] In the human body detection step S303, the human body detection unit 104 detects human bodies in the acquired image and determines information such as the body part, orientation, etc. of each detected human body. The human body detection unit 104 sets a plurality of human body regions in the image based on the determined body parts. The subsequent processes (steps) are performed for each human body region set by the human body detection unit 104.

[0035] Next, in a human body tracking step S304, the human body tracking unit 105 tracks the human body region detected by the human body detection unit 104. The human body tracking unit 105 links the human body region detected in the human body detection step S303 with past time-series human body information and holds the history.

[0036] In the area identification step S305, the subject determination unit 107 acquires information about the specific area set by the setting unit 106 from the ROM 213 or the external storage device 214. Then, the subject determination unit 107 determines whether the human body detected by the human body detection unit 104 is present in the specific area or outside the specific area. At this time, it is determined whether an arbitrary point in the human body area set by the human body detection unit 104 is within the specific area or based on the proportion of the human body area included in the specific area.

[0037] Next, in a staying time calculation step S306, the subject determination unit 107 multiplies the staying time by a weighting value for each specific area, and calculates the sum of the weighted staying times. In areas that are not specific areas, the weighting value is calculated as 1.

[0038] Then, in the threshold comparison step S307, the subject determination unit 107 determines whether or not the threshold (second threshold) for determining that the subject is not a detection target is exceeded. If it is determined that the threshold is not exceeded, the process proceeds to a determination step S308. On the other hand, if it is determined that the threshold is exceeded, the information processing method is terminated. Alternatively, it may be determined whether or not a threshold value (first threshold value) for determining that the object is to be detected is exceeded. If it is determined that the threshold value is exceeded, the process proceeds to a determination step S308.

[0039] FIG. 5 is a diagram for explaining the details of the human body detection step S303 to the determination step S308. In FIG. 5, a customer area (specific area, visitor's stay area) 501 and an employee area (specific area, employee-only stay area) 502 are set for an image 500 transmitted from the image acquisition unit 102.

[0040] The human body tracking unit 105 determines which area the human body is in from an arbitrary point in the human body area, but in this case, the determination is made from the center coordinates of the human body area. Therefore, human body A511 is in the customer area 501, human body B512 is in the employee area 502, and human body C513 is in a non-specific area that is not a specific area.

[0041] The subject determining unit 107 determines whether the human body A511, the human body B512, and the human body C513 are detection subjects or not based on the above-mentioned threshold value. Specifically, it is assumed that the threshold (second threshold) for excluding a person from the detection target is set to 1800 seconds, the weighting value for the customer area is set to 0.5, and the weighting value for the employee area is set to 2 by the setting unit 106. In other words, it is determined whether human body A 511, human body B 512, and human body C 513 are employees. If they are employees, they are excluded from the detection target. The stay time of human body A511 is 1800 seconds in customer area 501, 180 seconds in employee area 502, and 300 seconds in the non-specific area, for a total stay time of 2280 seconds. In this case, the weighted stay time (aggregated value after correction of stay time) is 1800×0.5+180×2+300×1=1200 seconds. Because it does not exceed the threshold value of 1800 seconds, it is not excluded from detection and is determined to be a detection target. Furthermore, human body B512 has a total stay time of 1150 seconds (aggregated value after stay time correction), consisting of 240 seconds in customer area 501, 780 seconds in employee area 502, and 130 seconds in the non-specific area. In this case, the weighted stay time is 240×0.5+780×2+130×1=1810 seconds. Since it exceeds the threshold value of 1800 seconds, it is determined to be outside the detection range.

[0042] As another example, suppose that the setting unit 106 has set the target threshold (first threshold) to 600 seconds, the weighting value of the customer area 501 to 5, and the weighting value of the employee area 502 to 0.1. In other words, it is determined whether the human body A 511, the human body B 512, and the human body C 513 are customers (visitors). If they are customers, they become detection targets. The total stay time (aggregated value after correction of stay time) of human body C513 is 240 seconds, consisting of 120 seconds in customer area 501, 60 seconds in employee area 502, and 60 seconds in non-specific areas. In this case, the weighted stay time is 120×5+60×0.1+60×1=666 seconds. Since it exceeds the threshold of 600 seconds, it is determined to be a detection target.

[0043] Returning to FIG. 3, when the second threshold value is used as the threshold value in the determination step S308, the subject determining unit 107 sets the human body determined as the detection target as the detection target of the behavior detection unit . When the first threshold is used as the threshold, a human body determined to be outside the detection target is excluded from the detection targets of the behavior detection unit 108.

[0044] According to the information processing device and the information processing method described above, it is possible to appropriately set a person in a camera image as a detection target. For example, it is possible to quickly and efficiently determine whether or not a human body in a store should be set as a detection target without using the characteristics of the human body. This makes it possible to quickly execute the behavior detection process by the behavior detection unit 108 and the behavior notification process by the behavior notification unit 109.

[0045] <Other embodiments> The present invention can also be realized by a process in which a program for implementing one or more of the functions of the above-described embodiments is supplied to a system or device via a network or a storage medium, and one or more processors in a computer of the system or device read and execute the program. The present invention can also be realized by a circuit (e.g., ASIC) for implementing one or more of the functions.

[0046] The disclosure of the present embodiment includes the following configuration of an information processing device. (Configuration 1) a human body detection unit that detects a human body included in an image; a human body tracking unit that measures a stay time of the human body in a predetermined area by tracking the human body detected by the human body detection unit; A setting unit for setting conditions related to a target of behavior detection; a target person determination unit that determines whether or not to treat the human body as a target for behavior detection based on the staying time measured by the human body tracking unit and the condition set by the setting unit; The present invention is characterized by comprising: (Configuration 2) The setting unit is A specific area in the predetermined area that requires correction of the staying time; A correction value for the specific region; and A threshold value for the corrected aggregate value of the stay time; 2. The information processing device according to claim 1, wherein the above-mentioned condition is set as follows: (Configuration 3) The setting unit is The threshold value is A first threshold value to be used for the behavior detection; A second threshold value that excludes the behavior from the target of the behavior detection; 3. The information processing device of configuration 2, wherein one or both of the above are set. (Configuration 4) 4. The information processing device of configuration 2 or 3, wherein the specific area is an employee-only area and / or a customer-only area. (Configuration 5) 5. The information processing device according to any one of configurations 1 to 4, further comprising a behavior detection unit that performs behavior detection in the image on the human body that the subject determination unit has determined to be a subject of behavior detection. (Configuration 6) 6. The information processing device of configuration 5, further comprising a behavior notification unit that issues a notification based on a detection result of the behavior detection unit. (method) A setting step of setting conditions related to a target of behavior detection; a human body detection step of detecting a human body included in the image; a human body tracking process of tracking the human body detected in the human body detection process and measuring a stay time; a target person determination step of determining whether or not the human body is to be a target of behavior detection based on the staying time measured in the human body tracking step and the conditions set in the setting step; An information processing method comprising: (program) A program for causing a computer to function as any one of the information processing devices according to configurations 1 to 6.

[0047] Although the preferred embodiments of the present invention have been described above, the present invention is not limited to these embodiments, and various modifications and changes are possible within the scope of the gist of the present invention. [Explanation of symbols]

[0048] 100 Information processing system (information processing device) 102 Image acquisition unit 104 Human body detection unit 105 Human body tracking section 106 Setting section 107 Subject Judgment Department 108 Behavior Detection Unit 109 Action Notification Department 110 Network 111 Network Camera 211 CPU (Computer) 500 images 501 Customer area (specific area, area where customers stay) 502 Employee Area (specific area, employee-only area) 511 Human body A 512 Human body B 513 Human body C

Claims

1. a human body detection unit that detects a human body included in an image; a human body tracking unit that measures a stay time of the human body in a predetermined area by tracking the human body detected by the human body detection unit; A setting unit for setting conditions related to a target of behavior detection; a target person determination unit that determines whether or not to treat the human body as a target for behavior detection based on the staying time measured by the human body tracking unit and the condition set by the setting unit; An information processing device comprising:

2. The setting unit is A specific area in the predetermined area that requires correction of the staying time; A correction value for the specific region; and A threshold value for the corrected aggregate value of the stay time; 2. The information processing apparatus according to claim 1, wherein the condition is set as follows:

3. The setting unit is The threshold value is A first threshold value to be used for the behavior detection; A second threshold value that excludes the behavior from the target of the behavior detection; 3. The information processing apparatus according to claim 2, further comprising: a setting unit for setting one or both of the above.

4. 3. The information processing apparatus according to claim 2, wherein the specific area is an employee-only area and / or a customer-only area.

5. The information processing apparatus according to claim 1 , further comprising a behavior detection unit configured to perform behavior detection in the image for the human body determined by the subject determination unit to be a subject of behavior detection.

6. The information processing apparatus according to claim 5 , further comprising a behavior notification unit that performs notification based on a detection result of the behavior detection unit.

7. A setting step of setting conditions related to a target of behavior detection; a human body detection step of detecting a human body included in the image; a human body tracking process of tracking the human body detected in the human body detection process and measuring a stay time; a target person determination step of determining whether or not the human body is to be a target of behavior detection based on the staying time measured in the human body tracking step and the conditions set in the setting step; An information processing method comprising:

8. A program for causing a computer to function as the information processing device according to any one of claims 1 to 6.

Images