Detection device, detection method, and detection program

Abstract

It accurately detects human bodies from video footage. [Solution] The detection device comprises a skeleton detection unit that detects at least coordinates corresponding to the neck and hips of the target person, a setting unit that sets the region connecting the neck coordinates and the hip coordinates detected by the skeleton detection unit as an evaluation region, and an evaluation unit that evaluates whether or not the evaluation region is a part of the human body based on the image information of the evaluation region set by the setting unit.

Description

Technical Field

[0001] The present invention relates to a detection device, a detection method, and a detection program.

Background Art

[0002] Using a technique for acquiring information from video such as a surveillance camera, the number of people shown in the video is detected to determine the degree of congestion, or the movement trajectory of the people shown in the video is detected to evaluate the flow line in the facility. In order to perform such detection, it is essential to detect people from the video. Examples of human detection include skeleton detection (see, for example, Patent Document 1). Skeleton detection acquires the coordinates of each part of the human body such as eyes, ears, head, neck, shoulders, elbows, hands, the base of the legs, knees, and ankles from the video, and determines whether or not it is a human based on the acquired coordinates. By performing skeleton detection, not only can a person be simply detected, but also the posture and movement of the detected person can be accurately analyzed, so it is widely adopted in surveillance systems.

Prior Art Documents

Patent Documents

[0003]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0004] When using the above-described skeleton detection technology to detect people, for example, inside a building, a shadow projected on a wall by lighting may be recognized as a person, resulting in a possibility of false detection.

[0005] The present invention has been made in view of the above, and an object thereof is to provide a detection device, a detection method, and a detection program capable of accurately detecting a human body from video.

Means for Solving the Problems

[0006] The detection device according to the present invention comprises a skeleton detection unit that detects coordinates corresponding to the neck and the base of both legs of a target person, a setting unit that sets the region connecting the coordinates of the neck and the coordinates of the base of both legs detected by the skeleton detection unit as an evaluation region, and an evaluation unit that evaluates whether or not the evaluation region is a part of the human body based on the image information of the evaluation region set by the setting unit.

[0007] The detection method according to the present invention includes detecting coordinates corresponding to the neck and hips of a target person, setting the region connecting the detected neck coordinates and the hip coordinates as an evaluation region, and evaluating whether the evaluation region is a human body part based on the image information of the set evaluation region.

[0008] The detection program according to the present invention causes a computer to perform the following processes: detecting coordinates corresponding to the neck and hips of a target person; setting the region connecting the detected neck coordinates and the hip coordinates as an evaluation region; and evaluating whether or not the evaluation region is a part of the human body based on the image information of the set evaluation region. [Effects of the Invention]

[0009] According to the present invention, it is possible to accurately detect human bodies from video footage. [Brief explanation of the drawing]

[0010] [Figure 1] Figure 1 is a functional block diagram showing an example of a detection device according to an embodiment. [Figure 2] Figure 2 is a schematic diagram illustrating the operation process of the detection device. [Figure 3] Figure 3 is a schematic diagram illustrating the operation process of the detection device. [Figure 4] Figure 4 is a schematic diagram illustrating the operation process of the detection device. [Figure 5] Figure 5 shows an example of setting an evaluation area in the image captured by the imaging device, specifically around the subject and their shadow. [Figure 6] Figure 6 shows the results of determining the brightness distribution for the image information of the evaluation region shown in Figure 5. [Figure 7] Figure 7 is a flowchart showing an example of a detection method according to the embodiment. [Modes for carrying out the invention]

[0011] Hereinafter, embodiments of the detection device, detection method, and detection program according to the present invention will be described with reference to the drawings. However, the present invention is not limited by these embodiments. Furthermore, the components in the following embodiments include those that are easily substituted or substantially identical to those that are easily substituted by those skilled in the art.

[0012] Figure 1 is a functional block diagram showing an example of a detection device 100 according to this embodiment. As shown in Figure 1, the detection device 100 includes a communication unit 10, an input unit 20, an output unit 30, a processing unit 40, and a storage unit 50.

[0013] The detection device 100 detects a human body from images captured by a camera or other imaging device 60. The detection device 100 uses skeletal detection technology to detect the skeleton of a human body based on the captured images.

[0014] The communication unit 10 performs wired or wireless communication with external devices. The communication unit 10 includes an interface such as a network interface card.

[0015] The input unit 20 receives various operations from the user and outputs them as operation signals to the processing unit 40. The input unit 20 can be, for example, a mouse, keyboard, touch panel, button, lever, dial, switch, or other input device.

[0016] The output unit 30 outputs the processing results from the processing unit 40. Examples of output units 30 include display devices such as liquid crystal displays and organic EL (Electro-Luminescence) displays, and audio output devices such as buzzers and speakers.

[0017] The processing unit 40 performs various information processes. The processing unit 40 includes a processor such as a CPU (Central Processing Unit) and memories such as a ROM (Read Only Memory) and a RAM (Random Access Memory). The processing unit 40 may be realized by an integrated circuit such as an ASIC (Application Specific Integrated Circuit) or an FPGA (Field Programmable Gate Array).

[0018] The processing unit 40 has a skeleton detection unit 41, a setting unit 42, and an evaluation unit 43.

[0019] The skeleton detection unit 41 detects at least coordinates corresponding to the neck and the roots of both legs of the target person. Further, the skeleton detection unit 41 can further detect coordinates corresponding to both shoulders, both elbows, and both knees of the target person. The skeleton detection unit 41 can detect each coordinate from, for example, a captured image captured by the imaging device 60. Note that the skeleton detection unit 41 may detect each coordinate using the detection result of a sensor different from the imaging device 60.

[0020] The setting unit 42 sets a triangular region connecting the coordinates of the neck detected by the detection unit and the coordinates of the roots of both legs as an evaluation region. Further, the setting unit 42 can further set, as evaluation regions, a region connecting the neck and both shoulders, a region connecting both shoulders and both elbows, and a region connecting the roots of both legs and both knees, based on the coordinates detected by the skeleton detection unit 41. The setting unit 42 can determine the widths of the regions connecting the neck and both shoulders, the regions connecting both shoulders and both elbows, and the regions connecting the roots of both legs and both knees, which are the evaluation regions, based on the distance from the coordinates of the root of the target person's neck to the coordinates of the midpoint between the roots of both legs.

[0021] The evaluation unit 43 evaluates whether the evaluation region is a human body part based on the image information of the evaluation region set by the setting unit. The evaluation unit 43 can evaluate whether the evaluation region is a human body part based on the luminance distribution in the image information of the evaluation region.

[0022] The memory unit 50 stores various programs, data, and other information. The memory unit 50 includes storage devices such as an HDD (Hard Disk Drive) and an SSD (Solid State Drive).

[0023] The memory unit 50 stores a detection program that causes the computer to perform, for example, the following steps: a process to detect coordinates corresponding to the neck and hips of a target person; a process to set a triangular area formed by connecting the detected neck coordinates and the hip coordinates as an evaluation area; and a process to evaluate whether or not the evaluation area is a part of the human body based on the image information of the set evaluation area.

[0024] In the detection device 100, the processor in the processing unit 40 reads various programs and loads them into memory, thereby executing information processing corresponding to the functions of each of the above-mentioned parts. Examples of various programs include programs received by the communication unit 10, programs stored in the storage unit 50, and programs recorded on an external recording medium. The detection device 100 functions as an information processing device (computer) that performs various information processing.

[0025] Next, the operation of the detection device 100 configured as described above will be explained. Figures 2 to 4 are schematic diagrams showing the operation process of the detection device 100.

[0026] For example, when a predetermined command is input by the user from the input unit 20, the skeleton detection unit 41 acquires the captured image taken by the imaging device 60. As shown in Figure 2, the skeleton detection unit 41 detects at least the coordinates P1 of the neck and the coordinates P2 and P3 corresponding to the base of both legs of the subject person M from the acquired captured image. At this time, the skeleton detection unit 41 may further detect the coordinates P4 and P5 corresponding to both shoulders, the coordinates P6 and P7 corresponding to both elbows, and the coordinates P8 and P9 corresponding to both knees of the subject person M.

[0027] Next, as shown in Figure 3, the setting unit 42 sets the first region AR1, a triangle formed by connecting the coordinates P1 of the neck detected by the skeleton detection unit 41 and the coordinates P2 and P3 of the bases of both legs, as the evaluation region AR. Furthermore, based on the coordinates detected by the skeleton detection unit 41, the setting unit 42 sets the second region AR2 connecting the neck and both shoulders, the third region AR3 connecting both shoulders and both elbows, and the fourth region AR4 connecting the bases of both legs and both knees as further evaluation regions AR.

[0028] Next, the evaluation unit 43 evaluates whether the evaluation area AR is a human body part based on the image information of the evaluation area AR set by the setting unit 42. In this case, the evaluation unit 43 evaluates whether the evaluation area AR is a human body part based, for example, on the brightness distribution in the image information of the evaluation area AR.

[0029] The evaluation area AR is set as an area corresponding to the human skeleton. In the captured images taken by the imaging device 60, the images corresponding to the human skeleton include, for example, an image of a person and an image of a person's shadow. In the captured images, for the image of a person, differences in brightness, i.e., a distribution of luminance, are likely to occur due to differences in the angle of light shining on the person's clothing, the pattern and color of the clothing, etc. On the other hand, for the image of a person's shadow, if, for example, the ground or wall surface on which the shadow is formed is a single color, differences in brightness are less likely to occur, and a distribution of luminance is less likely to occur.

[0030] Therefore, in this embodiment, in order to distinguish between a person and a shadow in the captured image taken by the imaging device 60, the evaluation unit 43 determines whether or not the evaluation area AR is a part of the human body based on the brightness distribution in the image information of the evaluation area AR.

[0031] Here, the wider the range designated as the evaluation area (AR), the higher the accuracy of the evaluation. On the other hand, as shown in Figure 4, when setting a wide range for the evaluation area (AR), for example, if the rectangular area formed by connecting the coordinates P4 and P5 of both shoulders and the coordinates P2 and P3 of both hip joints is set as the evaluation area (AR), when the person's posture changes, the evaluation area (AR) will extend beyond the person's area and include images of areas outside the person. This can lead to a decrease in detection accuracy in the brightness distribution and a decline in evaluation accuracy. Therefore, in this embodiment, the evaluation area (AR) is set as a triangular area formed by connecting the coordinates of the target person M's neck and the coordinates of both hip joints. This prevents the evaluation area (AR) from extending beyond the person's body when the person's posture changes.

[0032] Furthermore, in the images captured by the imaging device 60, people closer to the imaging device 60 will appear larger, while people further away from the imaging device 60 will appear smaller. For this reason, the widths m of the second region AR2, the third region AR3, and the fourth region AR4 (the width of the area connecting the joints) need to be adjusted appropriately. Also, the person may not only be facing towards the imaging device 60, but may also be facing sideways to the imaging direction of the imaging device 60. In this case, the dimensions of the person's skeletal structure in the left-right direction will change depending on the person's orientation. Therefore, the widths m of the second region AR2, the third region AR3, and the fourth region AR4 can be set based on the distance L connecting the base of the neck and the midpoint of the base of both legs, which is less affected by the person's orientation.

[0033] Here, the relationship between width m and distance L can be defined, for example, as shown in Equation 1 below. m=α×L (Formula 1) However, α is a constant. α can be set by assuming that the subject has a standard body type.

[0034] Figure 5 shows an example of setting evaluation regions in the image captured by the imaging device 60, specifically in the area of ​​the subject person M and its shadow. In Figure 5, evaluation region ARa is set for the subject person M, and evaluation region ARb is set for the shadow S of the subject person M. Figure 5 also shows extracted image information IM1 for evaluation region ARa and image information IM2 for evaluation region ARb.

[0035] Figure 6 schematically shows the results of determining the brightness distribution for the image information of the evaluation region shown in Figure 5. Figure 6 is shown in histogram format. In Figure 6, the horizontal axis represents brightness, and the vertical axis represents the number of detected pixels. As shown in Figure 6, the brightness distribution H1 of the image information IM1 of the evaluation region ARa of the target person M shows a wide range of detections from high-brightness pixels to low-brightness pixels, and the number of pixels per section within the detected brightness range is small. In contrast, the brightness distribution H2 of the image information IM2 of the evaluation region ARb of the shadow S shows that the brightness range is confined to a certain range, and the number of pixels per section within the detected brightness range is large.

[0036] Therefore, the evaluation unit 43 can evaluate the image information of the evaluation area AR as a human body part, for example, when the range of brightness detected in the brightness distribution is greater than or equal to a threshold, or when the average number of pixels per section within the detected brightness range is less than a threshold. In the example shown in Figure 6, the evaluation unit 43 evaluates the image information IM1 as a human body part because the range of brightness detected in the brightness distribution H1 is greater than or equal to a threshold, or the average number of pixels per section within the detected brightness range is less than a threshold. The evaluation unit 43 also evaluates the image information IM2 as a part that is not a human body part, i.e., a shadowed part, because the range of brightness detected in the brightness distribution H2 is less than a threshold, and the average number of pixels per section within the detected brightness range is greater than or equal to a threshold. The evaluation unit 43 may also perform evaluations based on criteria different from those described above. The evaluation unit 43 outputs the evaluation results in this manner to the output unit 30.

[0037] Figure 7 is a flowchart showing an example of a detection method according to this embodiment. As shown in Figure 7, the detection method according to this embodiment includes an image acquisition step S10, a skeleton detection step S20, an evaluation area setting step S30, and an evaluation step S40.

[0038] The image acquisition step S10 involves acquiring an image from the imaging device 60. The skeletal detection step S20 detects at least the coordinates corresponding to the neck and the base of both legs of the target person.

[0039] In the evaluation area setting step S20, the triangular area connecting the detected neck coordinates and the coordinates of the bases of both legs is set as the evaluation area AR.

[0040] The evaluation step S40 evaluates whether the evaluation area AR is a part of the human body based on the image information of the set evaluation area AR.

[0041] As described above, the detection device 100 according to this embodiment includes a skeleton detection unit 41 that detects coordinates corresponding to the neck and the base of both legs of a target person, a setting unit 42 that sets a first triangular region AR1 formed by connecting the coordinates of the neck and the coordinates of the base of both legs detected by the skeleton detection unit 41 as an evaluation region AR, and an evaluation unit 43 that evaluates whether or not the evaluation region AR is a part of the human body based on the image information of the evaluation region AR set by the setting unit 42.

[0042] The detection method according to this embodiment includes detecting coordinates corresponding to the neck and hips of a target person, setting a triangular region formed by connecting the detected neck coordinates and the hip coordinates as the evaluation region AR, and evaluating whether the evaluation region AR is a human body part based on the image information of the set evaluation region AR.

[0043] The detection program according to this embodiment causes the computer to perform the following processes: detecting coordinates corresponding to the neck and hips of the target person; setting a triangular region formed by connecting the detected neck coordinates and the hip coordinates as the evaluation region AR; and evaluating whether the evaluation region AR is a human body part based on the image information of the set evaluation region AR.

[0044] With these configurations, the evaluation area AR is set as a triangular region connecting the coordinates of the target person's neck and the coordinates of the base of both legs. This prevents the evaluation area AR from extending beyond the body's boundaries when the person's posture changes. As a result, when evaluating whether the evaluation area AR is a part of the human body based on its image information, the influence of shadows and other factors formed outside the body in the image information is suppressed, allowing for accurate detection of people from video.

[0045] In the detection device 100 according to this embodiment, the evaluation unit 43 evaluates whether the evaluation area AR is a human body part based on the brightness distribution in the image information of the evaluation area AR, the skeleton detection unit 41 further detects the coordinates corresponding to both shoulders, both elbows and both knees of the target person, and the setting unit 42 further sets the evaluation area AR as a second area AR2 connecting the neck and both shoulders, a third area AR3 connecting both shoulders and both elbows, and a fourth area AR4 connecting both hips and both knees, based on the coordinates detected by the skeleton detection unit 41.

[0046] With this configuration, in addition to the first triangular region AR1, the second region AR2, third region AR3, and fourth region AR4 are further designated as evaluation regions AR, enabling accurate detection of people from video.

[0047] In the detection device 100 according to this embodiment, the setting unit 42 determines the widths of the evaluation regions AR, namely the second region AR2, the third region AR3, and the fourth region AR4, based on the distance from the coordinates of the base of the target person's neck to the coordinates of the midpoint between the bases of both legs.

[0048] With this configuration, the widths of the second region AR2, the third region AR3, and the fourth region AR4 can be appropriately set based on the distance from the coordinates of the base of the neck to the coordinates of the midpoint of the base of both legs, which is less affected by changes in posture.

[0049] The technical scope of the present invention is not limited to the embodiments described above, and modifications can be made as appropriate without departing from the spirit of the invention. [Explanation of symbols]

[0050] AR, ARa, ARb…Evaluation area, AR1…First area, AR2…Second area, AR3…Third area, AR4…Fourth area, IM1, IM2…Image information, M…Target person, S…Shadow, 10…Communication unit, 20…Input unit, 30…Output unit, 40…Processing unit, 41…Skeleton detection unit, 42…Setting unit, 43…Evaluation unit, 50…Storage unit, 60…Shooting device, 100…Detection device

Claims

1. A skeletal detection unit that detects at least the coordinates corresponding to the neck and the base of both legs of the target person, A setting unit sets the region connecting the coordinates of the neck detected by the skeletal detection unit and the coordinates of the bases of both legs as the evaluation region. An evaluation unit that evaluates whether or not the evaluation area is a part of the human body based on the image information of the evaluation area set by the setting unit. A detection device equipped with the following features.

2. The evaluation unit evaluates whether the evaluation area is a part of the human body based on the brightness distribution in the image information of the evaluation area. The skeletal detection unit further detects the coordinates corresponding to both shoulders, both elbows, and both knees of the subject person, The setting unit further sets the following as evaluation regions based on the coordinates detected by the skeleton detection unit: a region connecting the neck and both shoulders, a region connecting both shoulders and both elbows, and a region connecting the base of both legs and both knees. The detection device according to claim 1.

3. The setting unit determines the width of the evaluation area, which is the region connecting the neck and both shoulders, the region connecting the shoulders and both elbows, and the region connecting the leg joints and both knees, based on the distance from the coordinates of the base of the neck of the subject person to the coordinates of the midpoint of the base of both legs. The detection device according to claim 2.

4. At least the coordinates corresponding to the neck and the base of both legs of the target person are detected, The region connecting the detected neck coordinates and the coordinates of both leg joints is set as the evaluation region, Based on the image information of the defined evaluation area, it is evaluated whether or not the evaluation area is a part of the human body. A detection method that includes [details omitted].

5. A process to detect at least the coordinates corresponding to the neck and the base of both legs of the target person, The process involves setting the region connecting the detected neck coordinates and the coordinates of both leg joints as the evaluation region, A process to evaluate whether the evaluation area is a part of the human body based on the image information of the set evaluation area. A detection program that causes a computer to execute a command.

Images