Human body detection device and human body detection method

Abstract

The human body detection device of the present application has: a human body detection unit that detects a human body from a captured image; a moving body determination unit that determines whether the detected human body is a moving body based on the image; a condition determination unit that determines whether the detected human body satisfies a prescribed condition under which a human body entering a mirror can be seen in the image; a mode setting unit that maintains a first mode until it is determined that the prescribed condition is satisfied, and performs switching from the first mode to a second mode according to a case where the prescribed condition is satisfied; and an information output unit that outputs information of the detected human body in a case where it is determined that the detected human body is a moving body in the first mode, and outputs information of the detected human body regardless of whether it is determined that the detected human body is a moving body in the second mode.

Description

Technical Field

[0001] This invention relates to a technique for detecting the human body from captured images. Background Technology

[0002] As a technique for detecting human bodies from captured images, moving body detection or human body detection is known. For example, a technique related to moving body detection is disclosed in Patent Document 1.

[0003] Existing technical documents

[0004] Patent documents

[0005] Patent Document 1: Japanese Patent Application Publication No. 2000-105835 Summary of the Invention

[0006] The problem that the invention aims to solve

[0007] However, moving object detection detects all moving objects, such as robotic arms. Furthermore, it cannot detect stationary human bodies. While human body detection can detect stationary human bodies, it sometimes misdetects human-like objects, such as mannequins, as human bodies. Therefore, neither moving object detection nor human body detection can detect human bodies with high precision.

[0008] The present invention was made in view of the above-mentioned actual situation, and its purpose is to provide a technology that can detect the human body with high precision.

[0009] Methods for solving problems

[0010] To achieve the above objectives, the present invention employs the following structure.

[0011] A first aspect of the present invention provides a human body detection device, comprising: a human body detection unit for detecting human bodies from captured images; a motion determination unit for determining, based on the image, whether the human body detected by the human body detection unit is a moving body; a condition determination unit for determining whether the human body detected by the human body detection unit satisfies a predetermined condition that allows a human body to be seen in the image; a mode setting unit for maintaining a first mode until the condition determination unit determines that the predetermined condition is met, and switching from the first mode to a second mode based on whether the condition determination unit determines that the predetermined condition is met; and an information output unit for outputting information about the human body in the first mode when the motion determination unit determines that the human body detected by the human body detection unit is a moving body, and for outputting information about the human body in the second mode regardless of whether the motion determination unit determines that the human body detected by the human body detection unit is a moving body. The human body information may include, for example, an identifier assigned to the human body, or information indicating the location (or area) where the human body was detected.

[0012] In the above structure, when a detected human body is determined to be a moving body, the information of that human body is output until the detected human body meets the predetermined conditions that allow it to be seen in the captured image (Mode 1). That is, the human body is detected by combining motion detection and human body detection (the logical product (AND) of the motion detection result and the human body detection result is set as the human body detection result). Therefore, human bodies as moving bodies can be detected with high precision. However, when the human body is stationary, it becomes impossible to detect it. Therefore, in the above structure, after the detected human body meets the predetermined conditions, the information of that human body is output regardless of whether the detected human body is determined to be a moving body (Mode 2). That is, the human body is detected by human body detection. Therefore, even when the human body is stationary, it can be detected with high precision (if the human body has entered the captured image, it can be detected with high precision through human body detection).

[0013] The mode setting unit can also set a mode for each human body detected by the human body detection unit. This enables the detection of multiple human bodies with high precision.

[0014] In the second mode, the motion detection unit may not determine whether the human body detected by the human body detection unit is a moving body, whereas in the first mode, it does determine whether the human body detected by the human body detection unit is a moving body. This reduces the processing load of motion detection (determining whether the human body detected by the human body detection unit is a moving body).

[0015] When a human body frames out of the captured image, other objects resembling a human body may be mistakenly detected as such during human body detection. Therefore, if the human body detection unit does not detect a human body, the mode setting unit can reset to the first mode. Thus, since human bodies are detected through a combination of motion detection and human body detection, it is possible to suppress the possibility of other objects resembling an undetected human body being mistakenly detected as such. Situations where the human body detection unit does not detect a human body include, for example, situations where the human body detection unit does not detect a human body even once, or situations where the state of not detecting a human body persists for a longer than a predetermined time.

[0016] When a human body is captured in an image, it usually indicates movement with a relatively large amount of motion and a relatively long duration. Therefore, the specified condition can also be set as the condition that the human body moves with a greater amount of motion than specified and for a longer period than specified. Alternatively, the specified condition can be set as the condition that the total time the human body has moved with a greater amount of motion than specified during the specified period is longer than specified time. By using the latter condition as the specified condition, even if the human body repeatedly moves and remains still, it is possible to appropriately switch from mode 1 to mode 2.

[0017] The locations where the human body primarily engages in the scene (e.g., the areas where the person mainly performs their work) are mostly predetermined. Therefore, the predetermined condition can also be set as the condition that the human body moves from the outside to the inside of the predetermined range in the image. Thus, for objects (not human bodies) that are present within the predetermined range from the beginning, applying a combination of motion detection and human body detection can suppress the possibility of the object being falsely detected as a human body. The predetermined range is, for example, the central part of an image taken from directly above the floor or ground; in the case of a fisheye image, it is a circular range at a distance of less than a predetermined distance from the center of the image.

[0018] The locations through which a human enters the frame (e.g., room entrances and exits) are mostly predetermined. Therefore, the predetermined condition can also be set as the condition that the human body moves from the inside to the outside of a predetermined range in the image. Thus, for objects (not human bodies) entering the frame from locations outside the predetermined range, a combination of motion detection and human body detection is applied, which can suppress the possibility of the object being falsely detected as a human body.

[0019] Alternatively, when detecting a human body from the image, the human body detection unit calculates the reliability of the probability that the detected human body is indeed a human body. The specified condition is that when the motion determination unit determines that the human body detected by the human body detection unit is a moving body, the cumulative value of the reliability is higher than a predetermined value. Therefore, for objects with low cumulative values, the combination of motion detection and human body detection is applied, thus suppressing the possibility of other objects resembling a human body being mistakenly detected as such. Furthermore, even if the human body repeatedly moves and remains stationary, the switching from mode 1 to mode 2 can be appropriately performed.

[0020] A second aspect of the present invention provides a human body detection method, characterized by comprising: a human body detection step for detecting a human body from a captured image; a moving body determination step for determining, based on the image, whether the human body detected in the human body detection step is a moving body; a condition determination step for determining whether the human body detected in the human body detection step satisfies a predetermined condition that allows a human body to be seen in the image; a mode setting step for maintaining a first mode until the predetermined condition is satisfied in the condition determination step, and switching from the first mode to a second mode based on whether the predetermined condition is satisfied in the condition determination step; and an information output step for outputting information about the human body in the first mode if the moving body determination step determines that the human body detected in the human body detection step is a moving body, and for outputting information about the human body in the second mode regardless of whether the moving body determination step determines that the human body detected in the human body detection step is a moving body.

[0021] This invention can be understood as a human body detection system having at least a portion of the above-described structure or function. Furthermore, this invention can be understood as a human body detection method or a control method for a human body detection system including at least a portion of the above-described processes, or a program for causing a computer to execute these methods, or a computer-readable recording medium that non-temporarily records such a program. The above-described structures and processes can be combined with each other to constitute this invention, provided they do not create technical contradictions.

[0022] Invention Effects

[0023] According to the present invention, human body can be detected with high precision. Attached Figure Description

[0024] Figure 1 This is a block diagram illustrating a structural example of a human body detection device to which the present invention is applied.

[0025] Figure 2 (A) is a schematic diagram illustrating a structural example of the human body detection system according to the first embodiment of the present invention. Figure 2(B) is a block diagram showing a structural example of the PC in this embodiment.

[0026] Figure 3 This diagram illustrates the generation of a difference image according to the first embodiment of the present invention.

[0027] Figure 4 This is a flowchart illustrating an example of the processing flow of a PC according to the first embodiment of the present invention.

[0028] Figure 5 This is a diagram illustrating a specific example of the operation of the first embodiment of the present invention.

[0029] Figure 6 This is a flowchart illustrating an example of the PC processing flow according to the second embodiment of the present invention.

[0030] Figure 7 This is a diagram illustrating a specific example of the operation of the second embodiment of the present invention.

[0031] Figure 8 This is a block diagram illustrating a structural example of the PC according to the third embodiment of the present invention.

[0032] Figure 9 This is a flowchart illustrating an example of the processing flow of a PC according to the third embodiment of the present invention.

[0033] Figure 10 This is a diagram illustrating a specific example of the operation of the third embodiment of the present invention.

[0034] Figure 11 This is a block diagram illustrating a structural example of the PC according to the fourth embodiment of the present invention.

[0035] Figure 12 This is a diagram illustrating an example of the range of activity of the fourth embodiment of the present invention.

[0036] Figure 13 This is a flowchart illustrating an example of the PC processing flow according to the fourth embodiment of the present invention.

[0037] Figure 14 This is a diagram illustrating an example of the entry / exit range of the fifth embodiment of the present invention.

[0038] Figure 15 This is a flowchart illustrating an example of the PC processing flow according to the fifth embodiment of the present invention.

[0039] Figure 16 This is a flowchart illustrating an example of the PC processing flow according to the sixth embodiment of the present invention.

[0040] Figure 17 This is a diagram illustrating a specific example of the operation of the sixth embodiment of the present invention. Detailed Implementation

[0041] <Application Example>

[0042] Application examples of the present invention will be described.

[0043] In motion object detection, all moving objects, such as robotic arms, are detected. However, stationary human bodies cannot be detected. Human body detection can detect stationary human bodies, but sometimes it misdetects objects resembling human figures, such as mannequins, as human bodies. Therefore, neither motion object detection nor human body detection can detect human bodies with high accuracy.

[0044] Figure 1 This is a block diagram illustrating a structural example of the human body detection device 100 to which the present invention is applied. The human body detection device 100 includes a human body detection unit 101, a motion body determination unit 102, a condition determination unit 103, a mode setting unit 104, and an information output unit 105. The human body detection unit 101 detects human bodies from captured images. The motion body determination unit 102 determines whether the human body detected by the human body detection unit 101 is a moving body based on the captured images. The condition determination unit 103 determines whether the human body detected by the human body detection unit 101 meets the predetermined conditions that allow the human body to be seen in the captured image. The mode setting unit 104 maintains a first mode until the condition determination unit 103 determines that the predetermined conditions are met, and switches from the first mode to a second mode depending on whether the condition determination unit 103 determines that the predetermined conditions are met. In the first mode, when the motion body determination unit 102 determines that the human body detected by the human body detection unit 101 is a moving body, the information output unit 105 outputs information about the human body. Furthermore, in the second mode, regardless of whether the motion determination unit 102 determines that the human body detected by the human body detection unit 101 is a moving body, the information output unit 105 outputs the information of that human body. The human body detection unit 101 is an example of the human body detection unit of the present invention, the motion determination unit 102 is an example of the motion determination unit of the present invention, and the condition determination unit 103 is an example of the condition determination unit of the present invention. Furthermore, the mode setting unit 104 is an example of the mode setting unit of the present invention, and the information output unit 105 is an example of the information output unit of the present invention. The information of the human body includes, for example, an identifier assigned to that human body, or information indicating the location (or area) where the human body was detected.

[0045] In the above structure, information about a human body is output when it is determined to be a moving body (Mode 1) until the detected human body meets the prescribed conditions that allow it to be seen in the captured image. That is, human bodies are detected through a combination of motion detection and human body detection (the logical product (AND) of the motion detection result and the human body detection result is set as the human body detection result). This allows for high-precision detection of moving human bodies. However, when a human body is stationary, it cannot be detected. Therefore, in the above structure, information about a human body is output after the detected human body meets the prescribed conditions, regardless of whether the detected human body is determined to be a moving body (Mode 2). That is, human bodies are detected through human body detection. This allows for high-precision detection of a human body even when it is stationary (and if the human body has entered the captured image, it can be detected with high precision through human body detection).

[0046] <First Implementation>

[0047] The first embodiment of the present invention will be described.

[0048] Figure 2 (A) is a schematic diagram illustrating a structural example of the human body detection system according to the first embodiment. The human body detection system of the first embodiment includes a camera 10 and a PC 200 (personal computer; human body detection device). The camera 10 and the PC 200 are connected to each other via wired or wireless means. The camera 10 captures images and outputs them to the PC 200. The camera direction of the camera 10 is not particularly limited; in the first embodiment, it is assumed that the images are captured from directly above the floor or ground. The type of images captured is not particularly limited, but in the first embodiment, it is assumed that fisheye images are captured. The PC 200 detects human bodies from the images captured by the camera 10. The PC 200 displays the information of the detected human body (an identifier assigned to the human body, information indicating the location (or area) of the detected human body, etc.) on a display unit, records it in a storage medium, or outputs it to other terminals (such as a smartphone of a remote administrator).

[0049] Furthermore, in the first embodiment, the PC 200 is a separate device from the camera 10, but the PC 200 may also be built into the camera 10. The aforementioned display unit or storage medium may or may not be part of the PC 200. Additionally, the location where the PC 200 is installed is not particularly limited. For example, the PC 200 may or may not be installed in the same room as the camera 10. The PC 200 may or may not be a computer in the cloud. The PC 200 may also be a terminal such as a smartphone carried by the administrator.

[0050] Figure 2 (B) is a block diagram showing a structural example of PC200. PC200 has an input unit 210, a control unit 220, a storage unit 230, and an output unit 240.

[0051] In the first embodiment, the camera 10 is configured to capture moving images. The input unit 210 sequentially acquires the captured images (frames of moving images) from the camera 10 and outputs them to the control unit 220. Alternatively, the camera 10 may sequentially capture still images; in this case, the input unit 210 sequentially acquires the captured still images from the camera 10 and outputs them to the control unit 220.

[0052] The control unit 220 includes a CPU (Central Processing Unit), RAM (Random Access Memory), ROM (Read Only Memory), etc., and performs control of various structural elements or various information processing. In the first embodiment, the control unit 220 detects a human body from the captured image and outputs information about the detected human body (an identifier assigned to the human body, information indicating the location (or area) of the detected human body, etc.) to the output unit 240.

[0053] Storage unit 230 stores programs executed by control unit 220, various data used by control unit 220, etc. For example, storage unit 230 is an auxiliary storage device such as hard disk drive or solid-state drive.

[0054] The output unit 240 displays the information (information about the detected human body) output by the control unit 220 on the display unit, or records it in the storage medium, or outputs it to other terminals (such as a remote administrator's smartphone).

[0055] The control unit 220 will be described in more detail. The control unit 220 includes a human body detection unit 221, a moving body determination unit 222, a condition determination unit 223, a mode setting unit 224, and an information output unit 225.

[0056] The human body detection unit 221 acquires images captured by the camera 10 from the input unit 210 and detects human bodies from the acquired images. Then, the human body detection unit 221 outputs the detected human body information to the motion determination unit 222 and the information output unit 225. The human body detection unit 221 is an example of the human body detection unit of the present invention.

[0057] Furthermore, any algorithm can be used for human detection based on the human detection unit 221. For example, a detector (recognizer) that combines image features such as HoG or Haar-like with enhancements can also be used to detect human bodies. Human bodies can be detected using a fully learned model generated by existing machine learning methods; specifically, a fully learned model generated by deep learning (e.g., R-CNN, Fast R-CNN, YOLO, SSD, etc.) can also be used to detect human bodies.

[0058] The motion determination unit 222 acquires images captured by the camera 10 from the input unit 210 and obtains detection results (human detection results) based on the human detection unit 221 from the human detection unit 221. Based on the acquired images, the motion determination unit 222 determines whether the human body detected by the human detection unit 221 is a moving body. Then, the motion determination unit 222 outputs the motion detection result (the determination of whether the human body detected by the human detection unit is a moving body) to the condition determination unit 223 and the information output unit 225. The motion determination unit 222 is an example of the motion determination unit of the present invention.

[0059] Furthermore, any algorithm can be used for motion determination based on the motion determination unit 222. For example, the motion determination unit 222 can detect moving objects using either background subtraction or inter-frame subtraction. Background subtraction, for example, is a method of detecting moving pixels as pixels whose absolute difference (difference) with a specified background image in the captured image is greater than or equal to a specified threshold. Inter-frame subtraction, for example, is a method of detecting moving pixels as pixels whose absolute difference with a specified background image in the captured current image (current frame) is greater than or equal to a specified threshold. In inter-frame subtraction, for example, the past frame is a frame preceding the current frame by a specified number of times, and the specified number is 1 or more. The specified number (the number of frames from the current frame to the past frames) can also be determined based on the frame rate of the control unit 220's processing (processing to detect a human body and output information about the detected human body), or based on the frame rate of the camera 10's recording, etc.

[0060] In the first embodiment, it is assumed that the motion determination unit 222 generates a difference image during motion detection. If the background subtraction method is used, the difference image represents the difference between the captured image and a predetermined background image. If the inter-frame subtraction method is used, the difference image represents the difference between the captured current image and the captured past image. Figure 3 This illustrates the case where a difference image is generated using the inter-frame differencing method. In Figure 3In the differential image, moving pixels are white pixels, and non-moving pixels (non-moving pixels) are black pixels. Then, assuming it's a moving body determination unit 222, it calculates the amount of motion of the human body detected by the human body detection unit 221 based on the differential image, and determines (detects) human bodies whose calculated motion amount is greater than a predetermined amount as moving bodies. The amount of motion of a human body is, for example, the ratio of the number of moving pixels in the region of the human body to the total number of pixels in the human body. Figure 3 As shown, when calculating motion, the motion determination unit 222 can also perform expansion processing of the region of the motion body (the region composed of moving pixels), or transformation processing of pixels (non-moving pixels) in the region surrounded by moving pixels into moving pixels, etc. Figure 3 In this context, transformation processing can also be described as filling a region (black) surrounded by white with white.

[0061] The condition determination unit 223 obtains the determination result (the result of motion detection) based on the motion determination unit 222 from the motion determination unit 222. Then, based on the obtained determination result, the condition determination unit 223 determines whether the human body detected by the human body detection unit 221 (the human body determined by the motion determination unit 222 to be a moving body) meets the entry conditions. The entry conditions are the prescribed conditions under which the human body entering the captured image can be seen. Then, the condition determination unit 223 outputs the determination result of whether the entry conditions are met to the mode setting unit 224. The condition determination unit 223 is an example of the condition determination unit of the present invention.

[0062] When a human body is captured in an image, it usually indicates that the movement is relatively large and lasts for a relatively long time. Therefore, in the first embodiment, the condition that the human body moves with a greater than predetermined amount of movement and for a longer than predetermined time is used as the condition for being captured in the image.

[0063] The mode setting unit 224 sets the mode (action mode) of the PC 200. In the first embodiment, it is assumed that the mode setting unit 224 sets the mode for the motion determination unit 222 and the information output unit 225. Alternatively, it is assumed that the first mode is set by default. The mode setting unit 224 obtains the determination result based on the condition determination unit 223 from the condition determination unit 223 and maintains the first mode until the condition determination unit 223 determines that the on-camera entry condition is met. Then, the mode setting unit 224 switches from the first mode to the second mode based on the condition determination unit 223 determining that the on-camera entry condition is met. The mode setting unit 224 is an example of the mode setting unit of the present invention.

[0064] In the first embodiment, it is assumed that the human detection unit 221 can detect multiple human bodies from a single image, and the mode setting unit 224 sets a mode for each human body detected by the human detection unit 221. This allows for the detection of multiple human bodies with high precision. Alternatively, if the number of human bodies that the human detection unit 221 can detect from a single image is only one, then associating human bodies with modes may not be necessary.

[0065] The information output unit 225 obtains the detection result (human detection result) based on the human body detection unit 221 from the human body detection unit 221, and obtains the determination result (motion detection result) based on the motion determination unit 222 from the motion determination unit 222. Then, in the first mode, if the motion determination unit 222 determines that the human body detected by the human body detection unit 221 is a motion body, the information output unit 225 outputs the human body information to the output unit 240 as the final human body detection result. Furthermore, in the second mode, regardless of whether the motion determination unit 222 determines that the human body detected by the human body detection unit 221 is a motion body, the information output unit 225 outputs the human body information as the final human body detection result to the output unit 240. Therefore, the first mode can be called the motion body valid mode, and the second mode can also be called the motion body invalid mode. The information output unit 225 is an example of the information output unit of the present invention.

[0066] In the first embodiment, it is assumed that the motion determination unit 222 does not determine whether the human body detected by the human body detection unit 221 is a motion body in the second mode, but determines whether the human body detected by the human body detection unit 221 is a motion body in the first mode. Therefore, the first mode can also be called the motion body detection mode, and the second mode can also be called the motion body non-detection mode. In this way, the processing load of motion body detection (determination of whether the human body detected by the human body detection unit 221 is a motion body) can be reduced. The motion determination unit 222 can perform motion body detection in both the first mode and the second mode. In this case, it can also be assumed that the mode setting unit 224 does not set a mode for the motion body determination unit 222, but sets a mode for the information output unit 225.

[0067] Figure 4 This is a flowchart illustrating an example of the PC200's processing flow. The PC200 executes repeatedly. Figure 4 The processing flow. Figure 4 The repetition cycle of the processing flow is not particularly limited, but in the first embodiment, it is assumed to be repeated at a frame rate (e.g., 30fps) based on the video recording rate of camera 10. Figure 4 The processing flow.

[0068] First, the input unit 210 acquires the captured image (current frame) from the camera 10 (step S401). Next, the motion determination unit 222 acquires previously captured images (past frames) or a predetermined background image from the storage unit 230 (step S402). Then, the motion determination unit 222 generates a difference image from the image acquired in step S401 and the image acquired in step S402 (step S403). Next, the human detection unit 221 detects human bodies from the image acquired in step S401 (step S404). Then, each human body detected in step S404 is processed using steps S405 to S412.

[0069] In step S405, the human detection unit 221 determines whether a human body of the processing target has been detected in the past (the human body detected this time) (same human body determination). If no human body of the processing target has been detected in the past, the human detection unit 221 assigns a new identifier (ID) to the human body of the processing target; if a human body of the processing target has been detected in the past, the human detection unit 221 assigns an identifier identical to the previously assigned identifier to the human body of the processing target. The method for determining the same human body is not particularly limited. For example, if the difference between the position of the human body of the processing target (the center position of the detection rectangle, etc.) and the position of the previously detected human body is below a threshold, it can be determined that the human body of the processing target is the same as the previously detected human body. Similarly, if the IoU (Intersection over Union) between the region of the human body of the processing target (the detection rectangle, etc.) and the region of the previously detected human body is below a threshold, it can also be determined that the human body of the processing target is the same as the previously detected human body. Even in person re-identification, it is possible to determine whether a human body of the processing target has been detected in the past.

[0070] Next, the motion determination unit 222 or the information output unit 225 determines whether a motion valid mode (motion detection mode; mode 1) is set for the human body being processed (step S406). If it is determined that a motion valid mode is set (step S406: YES), the process proceeds to step S407. If it is determined that a motion valid mode is not set (step S406: NO), that is, if it is determined that a motion invalid mode (motion non-detection mode; mode 2) is set, the process proceeds to step S412.

[0071] In step S407, the motion determination unit 222 calculates the motion amount M of the human body of the processing object based on the differential image generated in step S403. Then, the motion determination unit 222 determines whether the motion amount M calculated in step S407 is greater than a predetermined amount Th_M (step S408). If it is determined that the motion amount M is greater than the predetermined amount Th_M (step S408: Yes), the motion determination unit 222 determines that the human body of the processing object is a moving body. Then, the processing proceeds to step S409. If it is determined that the motion amount M is less than the predetermined amount Th_M (step S408: No), the motion determination unit 222 determines that the human body of the processing object is not a moving body. Then, if there are remaining unprocessed human bodies (human bodies detected in step S404 but not processed in steps S405 to S412), the human body of the processing object is switched to an unprocessed human body, and the processing in steps S405 to S412 is performed. If there are no remaining unprocessed human bodies, Figure 4 The processing flow has ended.

[0072] In step S409, the condition determination unit 223 increments the continuous movement frame count F1 of the human body being processed by 1. The continuous movement frame count F1 is the number of frames in which the human body detected by the human body detection unit 221 is determined to be a moving body, and is equivalent to the time during which the human body continues to move with a movement amount greater than a predetermined amount. The initial value of the continuous movement frame count F1 is 0, and if the human body detection unit 221 does not detect a corresponding human body or the human body is determined not to be a moving body, the continuous movement frame count F1 is reset to 0.

[0073] Next, the condition determination unit 223 determines whether the number of consecutive movement frames F1 of the human body being processed is greater than a predetermined number Th_F1 (step S410). The predetermined number Th_F1 is equivalent to the "predetermined time" in the shot entry condition of "the human body moves with a greater than predetermined amount of movement and for a longer than predetermined time". If it is determined that the number of consecutive movement frames F1 is greater than the predetermined number Th_F1 (step S410: Yes), that is, if the shot entry condition is met, the process proceeds to step S411. If it is determined that the number of consecutive movement frames F1 is less than the predetermined number Th_F1 (step S410: No), that is, if the shot entry condition is not met, the process proceeds to step S412.

[0074] In step S411, the mode setting unit 224 switches the mode of the human body being processed from the motion body valid mode to the motion body invalid mode. Then, the process proceeds to step S412.

[0075] In step S412, the information output unit 225 outputs information about the human body of the processing target. Then, if there are remaining unprocessed human bodies (human bodies detected in step S404 but not processed in steps S405-S412), the human body of the processing target is switched to an unprocessed human body, and the processing in steps S405-S412 is performed. If there are no remaining unprocessed human bodies, Figure 4 The processing flow has ended.

[0076] Figure 5 Indicates based on Figure 4 Specific examples of actions in the processing flow. Specifically, Figure 5 This example shows a frame, a human body identifier (ID), the detection result based on the human body detection unit 221, the determination result based on the motion determination unit 222, the number of consecutive moving frames F1, and a mode. Regarding the detection result based on the human body detection unit 221, "O" means a human body was detected, and "×" means no human body was detected. Regarding the determination result based on the motion determination unit 222, "O" means the human body is determined to be a moving body, "×" means the human body is determined not to be a moving body, and "-" indicates no determination was made. The specified number Th_F1 compared with the number of consecutive moving frames F1 is not particularly limited, and is set to 4 here. That is, when the number of consecutive moving frames F1 reaches 5, a switch is made from the motion body valid mode to the motion body invalid mode.

[0077] ID1 is the identifier for the human body. Since the number of consecutive frames F1 in frame 5 reaches 5, a switch to the motion invalidation mode occurs between frame 5 and frame 6. Therefore, even if the human body in ID1 is stationary, its information can still be output.

[0078] ID2 is the identifier of the robot arm that is mistakenly detected as a human by the human detection unit 221. Since the human detection unit 221 misdetects the robot arm as a human infrequently and the number of consecutive movement frames F1 does not reach 5, the motion body valid mode is maintained. Therefore, by continuously applying the combination of motion body detection and human detection, it is possible to suppress the output of robot arm information as human information.

[0079] ID3 is the identifier for a human model that is mistakenly detected as a human by the human detection unit 221. Even if the human detection unit 221 mistakenly detects a human model as a human, the frequency with which the human model is judged as a moving body is relatively low, and the number of consecutive movement frames F1 does not reach 5, thus maintaining the effective moving body mode. Therefore, by continuously applying the combination of moving body detection and human detection, it is possible to suppress the output of human model information as human body information.

[0080] As described above, in the first embodiment, when a detected human body is determined to be a moving body, the information of that human body is output until the detected human body meets the entry conditions (moving body valid mode). That is, the human body is detected by combining moving body detection and human body detection (the logical product (AND) of the results of moving body detection and human body detection is set as the human body detection result). As a result, human bodies that are moving bodies can be detected with high precision. However, when the human body is stationary, it becomes impossible to detect the human body. Therefore, in the first embodiment, after the detected human body meets the entry conditions, the information of that human body is output regardless of whether the detected human body is determined to be a moving body (moving body invalid mode). That is, the human body is detected by human body detection. As a result, even if the human body is stationary, the human body can be detected with high precision (if the human body has entered the captured image, the human body can be detected with high precision by human body detection).

[0081] <Second Implementation Method>

[0082] The second embodiment of the present invention will be described. The structure of the human body detection system in the second embodiment is the same as that in the first embodiment, and the structure of the PC200 (human body detection device) in the second embodiment is the same as that in the first embodiment. In the second embodiment, the entry conditions are different from those in the first embodiment.

[0083] As described in the first embodiment, when a human body is captured in the image, it usually indicates movement with a relatively large amount of motion and a relatively long duration. Therefore, in the second embodiment, the condition for capturing the image is that the total time during which the human body moves with a greater amount of motion than a specified amount within a specified period is longer than a specified time. Therefore, even if the human body repeatedly moves and remains still, it is possible to appropriately switch from an effective movement mode to an ineffective movement mode.

[0084] Figure 6 This is a flowchart illustrating an example of the processing flow of the PC200 in the second embodiment. The PC200 repeatedly executes... Figure 6 The processing flow. Figure 6 The repetition cycle of the processing flow is not particularly limited, but in the second embodiment, it is assumed to be repeated at a frame rate (e.g., 30fps) based on the video recording rate of camera 10. Figure 6 The processing flow.

[0085] The processing in steps S601 to S608 is the same as that in steps S401 to S408 of the first embodiment. If it is determined in step S608 that the amount of motion M is greater than the predetermined amount Th_M (step S608: Yes), the motion determination unit 222 determines that the human body to be processed is a motion body. Then, the processing proceeds to step S609.

[0086] In step S609, the condition determination unit 223 calculates the number of frames (movement frame number F2) in the specified number of frames up to the present where the human body is determined to be a moving body (processing object). The specified number of frames corresponds to the "specified period" in the shot entry condition, which states that "the total time during which the human body moves with a motion amount greater than a specified amount during the specified period up to the present is longer than the specified time." The movement frame number F2 corresponds to the "total time during which the human body moves with a motion amount greater than a specified amount" in the shot entry condition.

[0087] Next, the condition determination unit 223 determines whether the number of movement frames F2 of the human body being processed is greater than a predetermined number Th_F2 (step S610). The predetermined number Th_F2 is equivalent to the "predetermined time" in the shot entry condition, which states that "the total time during which the human body moves with a greater than predetermined amount of movement within the predetermined period up to the present is longer than the predetermined time." If it is determined that the number of movement frames F2 is greater than the predetermined number Th_F2 (step S610: Yes), that is, if the shot entry condition is met, the process proceeds to step S611. If it is determined that the number of movement frames F2 is less than the predetermined number Th_F2 (step S610: No), that is, if the shot entry condition is not met, the process proceeds to step S612.

[0088] The processing of steps S611 and S612 is the same as that of steps S411 and S412 in the first embodiment.

[0089] Figure 7 Indicates based on Figure 6 Specific examples of actions in the processing flow. Specifically, Figure 7 The following is an example showing the frame, the human body identifier (ID), the detection result based on the human body detection unit 221, the determination result based on the moving body determination unit 222, the number of moving frames F2, and the mode. Figure 7 The number of consecutive moving frames F1 in the first embodiment is also shown. The number of frames referenced (a predetermined number of frames up to the present) for calculating the number of moving frames F2 is not particularly limited, and is set to 5 here. The predetermined number Th_F2 compared with the number of moving frames F2 is not particularly limited, and is set to 2 here. That is, when the number of moving frames F2 (the number of frames in the previous 5 frames where the human body detected by the human body detection unit 221 is determined to be a moving body) reaches 3, a switch is performed from the moving body effective mode to the moving body ineffective mode. Everything else is assumed to be the same as in the first embodiment.

[0090] ID1 is the identifier for the human body. Since the number of frames moved (F2) in frame 5 reaches 3, a switch to the motion invalidation mode occurs between frames 5 and 6. Therefore, even if the human body in ID1 is stationary, its information can still be output.

[0091] In the first embodiment, if the human body detected by the human body detection unit 221 is continuously identified as a moving body, and the number of consecutive moving frames F1 does not reach 5, the switch to the moving body invalid mode is not performed. Therefore, the switch to the moving body invalid mode is not performed between frame 5 and frame 6. On the other hand, in the second embodiment, even if the human body detected by the human body detection unit 221 is intermittently identified as a moving body, the switch to the moving body invalid mode can be performed appropriately.

[0092] As described above, in the second embodiment, the switching to human body detection is performed based on a combination of motion detection and human body detection, depending on whether the entry conditions are met, thereby enabling high-precision detection of the human body. Furthermore, by using the condition that the total time during which the human body moves with a greater than predetermined amount of motion over the specified period is longer than the predetermined time as the entry condition, the aforementioned switching can be performed appropriately even if the human body repeatedly moves and remains still.

[0093] <Third Implementation Method>

[0094] The third embodiment of the present invention will be described. The structure of the human body detection system in the third embodiment is the same as that in the first embodiment. In the first or second embodiment, the motion body invalid mode is maintained after switching to the motion body invalid mode, but in the third embodiment, a reset to the motion body valid mode is sometimes performed.

[0095] Figure 8 This is a block diagram illustrating a structural example of the PC200 (human detection device) according to the third embodiment. The PC200 of the third embodiment has the same structural elements as that of the first embodiment. In the third embodiment, the mode setting unit 224 obtains the detection results (human detection results) based on the human detection unit 221 from the human detection unit 221, and resets the mode to the effective mode for the moving body based on the obtained detection results. Other processing is assumed to be the same as in the first embodiment.

[0096] When a human body frames out of the captured image, there is a possibility that another object resembling a human body may be mistakenly detected as such during human body detection. Therefore, if the human body detection unit 221 does not detect a human body, the mode setting unit 224 of the third embodiment resets to the motion body effective mode. Thus, since human bodies are detected through a combination of motion body detection and human body detection, it is possible to suppress the possibility of other objects resembling the undetected human body being mistakenly detected as such. The situation where the human body detection unit 221 does not detect a human body could mean that the human body detection unit 221 has not detected a human body even once; however, in the third embodiment, it is assumed that the state of the human body detection unit 221 not detecting a human body has persisted for a longer than a predetermined time.

[0097] Figure 9 This is a flowchart illustrating an example of the processing flow of PC200 in the third embodiment. PC200 repeatedly executes... Figure 9 The processing flow. Figure 9 The repetition cycle of the processing flow is not particularly limited, but in the third embodiment, it is assumed to be repeated at a frame rate (e.g., 30fps) based on the video recording rate of camera 10. Figure 9 The processing flow.

[0098] The processing in steps S901 to S912 is the same as that in steps S401 to S412 of the first embodiment. After processing each human body detected in step S904 using steps S905 to S912, processing in steps S913 to S915 is performed on each human body not detected in step S904. Here, the human body not detected in step S904 refers to a human body that was detected in the past and assigned an identifier, but was not detected this time.

[0099] In step S913, the mode setting unit 224 increments the number of consecutive non-detection frames F3 of the human body being processed by 1. The number of consecutive non-detection frames F3 is the number of consecutive frames in which the human body detection unit 221 has not detected a human body, and is equivalent to the duration of the period during which the human body detection unit 221 has not detected a human body. The initial value of the number of consecutive non-detection frames F3 is 0, and when the corresponding human body is detected by the human body detection unit 221, the number of consecutive non-detection frames F3 is reset to 0.

[0100] Next, the mode setting unit 224 determines whether the number of consecutive non-detected frames F3 of the human body to be processed is greater than a predetermined number Th_F3 (step S914). The predetermined number Th_F3 is equivalent to a predetermined time. If it is determined that the number of consecutive non-detected frames F3 is greater than the predetermined number Th_F3 (step S914: Yes), it is equivalent to the situation where the human body detection unit 221 has not detected the human body of the processing target for a longer period than the predetermined time. In this case, the process proceeds to step S915. If it is determined that the number of consecutive non-detected frames F3 is less than or equal to the predetermined number Th_F3 (step S914: No), it is equivalent to the situation where the human body detection unit 221 has not detected the human body for a longer period than the predetermined time. In this case, if there are still unprocessed human bodies (human bodies that were not detected in step S904 but were not processed in steps S913 to S915), the human body to be processed is switched to an unprocessed human body, and the processing in steps S913 to S915 is performed. If there are no unprocessed human bodies remaining, Figure 9 The processing flow has ended.

[0101] In step S915, the mode setting unit 224 resets the mode of the human body to be processed to the motion body effective mode. Then, if there are any unprocessed human bodies remaining, the human body to be processed is switched to an unprocessed human body, and the processing in steps S913 to S915 is performed. If there are no unprocessed human bodies remaining, Figure 9 The processing flow has ended.

[0102] Figure 10 express Figure 9 Specific examples of actions in the processing flow. Specifically, Figure 10 This diagram illustrates an example of a frame, a human body identifier (ID), the detection result based on the human body detection unit 221, the determination result based on the moving body determination unit 222, the number of consecutive moving frames F1, the number of consecutive non-detected frames F3, and a pattern. The predetermined number Th_F3 compared to the number of consecutive non-detected frames F3 is not particularly limited, and is set to 2 here. That is, when the number of consecutive non-detected frames F3 reaches 3, a reset to the moving body valid pattern is performed. Everything else is assumed to be the same as in the first embodiment.

[0103] ID1 is the identifier for the human body. Since the number of consecutive moving frames F1 reaches 5 in frame 5, a switch to motion invalidation mode occurs between frames 5 and 6. Therefore, even if the human body in ID1 is stationary, its information can still be output. Since the number of consecutive non-detected frames F3 reaches 3 in frame 10, a switch to motion invalidation mode occurs between frames 10 and 11. This suppresses the output of information about other objects similar to the person in ID1.

[0104] As described above, according to the third embodiment, if the human body detection unit 221 does not detect a human body, the system is reset to the active motion mode. Therefore, since the human body is detected through a combination of active motion detection and human body detection, it is possible to suppress the misdetection of other objects similar to the undetected human body as that human body.

[0105] <Fourth Implementation>

[0106] The fourth embodiment of the present invention will be described. The structure of the human body detection system in the fourth embodiment is the same as that in the first embodiment. However, the entry conditions in the fourth embodiment differ from those in the first to third embodiments.

[0107] Figure 11This is a block diagram illustrating a structural example of the PC200 (human body detection device) according to the fourth embodiment. The PC200 of the fourth embodiment has the same structural elements as that of the first embodiment. In the fourth embodiment, the condition determination unit 223 obtains the detection result (human body detection result) based on the human body detection unit 221 from the human body detection unit 221, and obtains the determination result (moving body detection result) based on the moving body determination unit 222 from the moving body determination unit 222. Then, based on the detection result of the human body detection unit 221, the condition determination unit 223 determines whether the human body detected by the human body detection unit 221 (the human body determined by the moving body determination unit 222 to be a moving body) meets the entry conditions. It is assumed that other processing is the same as in the first embodiment.

[0108] The locations where the human body primarily engages in the shot (e.g., the locations where the human body primarily performs its work) are mostly predetermined. Therefore, in the fourth embodiment, the condition that the human body has moved from the outside to the inside of a predetermined range in the captured image is used as the entry condition. Thus, for objects (not human bodies) that are present within the predetermined range from the beginning, a combination of motion detection and human body detection is applied, thereby suppressing the possibility of the object being mistakenly detected as a human body. Hereinafter, this predetermined range is referred to as the activity range. The activity range is, for example, the central portion of an image captured from directly above the floor or ground. In the fourth embodiment, similar to the first embodiment, it is assumed that a fisheye image is captured from directly above the floor or ground. Furthermore, as... Figure 12 As shown, the range of activity is a circular area within a specified distance from the center of the fisheye image.

[0109] Figure 13 This is a flowchart illustrating an example of the processing flow of PC200 in the fourth embodiment. PC200 repeatedly executes... Figure 13 The processing flow. Figure 13 The processing cycle is not particularly limited, but in the fourth embodiment, it is assumed to be repeated at a frame rate (e.g., 30fps) based on the video recording rate of camera 10. Figure 13 The processing flow.

[0110] The processing in steps S1301 to S1308 is the same as that in steps S401 to S408 of the first embodiment. If it is determined in step S1308 that the amount of motion M is greater than the predetermined amount Th_M (step S1308: Yes), the motion determination unit 222 determines that the human body to be processed is a motion body. Then, the processing proceeds to step S1309.

[0111] In step S1309, the condition determination unit 223 calculates the distance D from the center of the image obtained in step S1301 to the position of the human body of the processing object (such as the center position of the detection rectangle).

[0112] Next, the condition determination unit 223 determines whether the detection flag is ON (step S1310). If the detection flag is determined to be ON (step S1310: Yes), the process proceeds to step S1313. If the detection flag is determined not to be ON (step S1310: No), that is, if the detection flag is determined to be OFF, the process proceeds to step S1311. In the fourth embodiment, the detection flag is a flag indicating whether a human body of the processing object is detected outside the activity range, and it becomes ON based on the detection of a human body of the processing object outside the activity range. That is, the detection flag is a flag indicating whether the precondition of the entry condition, such as the human body moving from the outside to the inside of the activity range, is met, and it becomes ON based on the meeting of the precondition. The initial state of the detection flag is OFF.

[0113] In step S1311, the condition determination unit 223 determines whether the distance D calculated in step S1309 is greater than or equal to a predetermined distance Th_D. The predetermined distance Th_D is the radius of the activity range. If the distance D is greater than or equal to the predetermined distance Th_D, the human body of the processing object is located outside the activity range; if the distance D is less than the predetermined distance Th_D, the human body of the processing object is located inside the activity range. Therefore, the determination in step S1311 can also be described as a determination of whether the aforementioned preconditions are met. If it is determined that the distance D is greater than or equal to the predetermined distance Th_D (step S1311: Yes), that is, if the preconditions are met, the process proceeds to step S1312. If it is determined that the distance D is less than the predetermined distance Th_D (step S1311: No), that is, if the preconditions are not met, the process proceeds to step S1315. Furthermore, in the fourth embodiment, the position where the distance D is the predetermined distance Th_D is set as outside the activity range, but this position can also be set as inside the activity range.

[0114] In step S1312, the condition determination unit 223 sets the detection flag to ON. Then, the process proceeds to step S1315.

[0115] In step S1313, the condition determination unit 223 determines whether the distance D calculated in step S1309 is less than the predetermined distance Th_D. Here, the aforementioned prerequisite condition has already been met. Therefore, the determination in step S1313 can also be described as a determination of whether the entry condition for the human body to move from the outside to the inside of the activity range is met. If it is determined that the distance D is less than the predetermined distance Th_D (step S1313: Yes), that is, if the entry condition is met, the process proceeds to step S1314. If it is determined that the distance D is greater than or equal to the predetermined distance Th_D (step S1313: No), that is, if the entry condition is not met, the process proceeds to step S1315.

[0116] The processing of steps S1314 and S1315 is the same as that of steps S411 and S412 in the first embodiment.

[0117] As described above, in the fourth embodiment, depending on whether the entry condition is met, the system switches to human detection based on a combination of motion detection and human detection, thereby enabling high-precision detection of human bodies. Furthermore, by using the condition that a human body moves from the outside to the inside of the activity range as the entry condition, the system can suppress the misdetection of objects (not human bodies) existing within the activity range as human bodies from the outset.

[0118] <Fifth Implementation>

[0119] The fifth embodiment of the present invention will be described. The structure of the human body detection system in the fifth embodiment is the same as that in the first embodiment, and the structure of the PC200 (human body detection device) in the fifth embodiment is the same as that in the fourth embodiment. In the fifth embodiment, the entry conditions are different from those in the first to fourth embodiments.

[0120] The locations through which a person enters the camera (e.g., room entrances and exits) are mostly predetermined. Therefore, in the fifth embodiment, the condition that the person moves from the inside to the outside of a predetermined range in the captured image is used as the entry condition. Thus, for objects (not human bodies) entering the camera from locations outside the predetermined range, a combination of motion detection and human body detection is applied, thus suppressing the possibility of the object being mistakenly detected as a human body. Hereinafter, this predetermined range will be described as the entry / exit range. The entry / exit range is, for example, as shown below. Figure 14 As shown, this is the end of the path in the direction in which the path extends in the captured image.

[0121] Figure 15 This is a flowchart illustrating an example of the processing flow of PC200 in the fifth embodiment. PC200 repeatedly executes... Figure 15 The processing flow. Figure 15The repetition cycle of the processing flow is not particularly limited, but in the fifth embodiment, it is assumed to be repeated at a frame rate based on the video recording rate of camera 10 (e.g., 30fps). Figure 15 The processing flow.

[0122] The processing in steps S1501 to S1508 is the same as that in steps S401 to S408 of the first embodiment. If it is determined in step S1508 that the amount of motion M is greater than the predetermined amount Th_M (step S1508: Yes), the motion determination unit 222 determines that the human body to be processed is a motion body. Then, the processing proceeds to step S1509.

[0123] In step S1509, the condition determination unit 223 determines whether the detection flag is turned on. If it is determined that the detection flag is turned on (step S1509: Yes), the process proceeds to step S1512. If it is determined that the detection flag is not turned on (step S1509: No), that is, if it is determined that the detection flag is turned off, the process proceeds to step S1510. In the fifth embodiment, the detection flag is a flag indicating whether a human body of the processing object is detected inside the entry / exit range, and it is turned on when a human body of the processing object is detected inside the entry / exit range. That is, the detection flag is a flag indicating whether the prerequisite condition of the entry condition, such as the human body moving from the inside to the outside of the entry / exit range, is met, and it is turned on when the prerequisite condition is met. The initial state of the detection flag is off.

[0124] In step S1510, the condition determination unit 223 determines whether the position of the human body being processed (such as the center position of the detection rectangle) is inside the entry / exit range, i.e., whether the aforementioned prerequisite condition is met. If it is determined that the position of the human body being processed is inside the entry / exit range (step S1510: Yes), i.e., the prerequisite condition is met, the process proceeds to step S1511. If it is determined that the position of the human body being processed is not inside the entry / exit range (on the outside) (step S1510: No), i.e., the prerequisite condition is not met, the process proceeds to step S1514.

[0125] In step S1511, the condition determination unit 223 sets the detection flag to ON. Then, the process proceeds to step S1514.

[0126] In step S1512, the condition determination unit 223 determines whether the position of the human body to be processed is outside the entry / exit range. Here, the aforementioned prerequisite condition has already been met. Therefore, the determination in step S1512 can also be described as a determination of whether the entry condition of the human body moving from the inside to the outside of the entry / exit range is met. If it is determined that the position of the human body to be processed is outside the entry / exit range (step S1512: Yes), that is, if the entry condition is met, the process proceeds to step S1513. If it is determined that the position of the human body to be processed is not outside the entry / exit range (located inside) (step S1512: No), that is, if the entry condition is not met, the process proceeds to step S1514.

[0127] The processing of steps S1513 and S1514 is the same as that of steps S411 and S412 in the first embodiment.

[0128] As described above, in the fifth embodiment, depending on whether the entry conditions are met, a switch to human detection is performed based on a combination of motion detection and human detection, thereby enabling high-precision detection of human bodies. Furthermore, by using the condition that a human body moves from the inside to the outside of the entry / exit range as the entry condition, it is possible to suppress the misdetection of objects (not human bodies) entering the camera from locations outside the entry / exit range as human bodies.

[0129] <Sixth Implementation>

[0130] The sixth embodiment of the present invention will be described. The structure of the human body detection system in the sixth embodiment is the same as that in the first embodiment, and the structure of the PC200 (human body detection device) in the sixth embodiment is the same as that in the fourth or fifth embodiment. In the sixth embodiment, the entry conditions are different from those in the first to fifth embodiments.

[0131] In the sixth embodiment, when detecting a human body from the captured image, the human body detection unit 221 calculates a reliability, which is the probability that the detected human body is a human body. The method for calculating the reliability is not particularly limited; for example, the reliability is the similarity between the features of the detected human body and features predetermined as human body characteristics. The calculated reliability is notified from the human body detection unit 221 to the condition determination unit 223. In the sixth embodiment, a condition is used where the accumulated value of the reliability when the moving body determination unit 222 determines that the human body detected by the human body detection unit 221 is a moving body is higher than a predetermined value. Therefore, since a combination of moving body detection and human body detection is applied to objects with low accumulated values, it is possible to suppress the misdetection of other objects resembling a human body as that human body. Furthermore, even if the human body repeatedly moves and remains stationary, it is possible to appropriately switch from a moving body effective mode to a moving body ineffective mode.

[0132] Figure 16 This is a flowchart illustrating an example of the processing flow of PC200 in the sixth embodiment. PC200 repeatedly executes... Figure 16 The processing flow. Figure 16 The repetition cycle of the processing flow is not particularly limited, but in the sixth embodiment, it is assumed to be repeated at a frame rate based on the video recording rate of camera 10 (e.g., 30fps). Figure 16 The processing flow.

[0133] The processing in steps S1601 to S1608 is the same as that in steps S401 to S408 of the first embodiment. If it is determined in step S1608 that the amount of motion M is greater than the predetermined amount Th_M (step S1608: Yes), the motion determination unit 222 determines that the human body to be processed is a motion body. Then, the processing proceeds to step S1609.

[0134] In step S1609, the condition determination unit 223 adds the reliability of the human body to the cumulative reliability R of the human body. The cumulative reliability R is the cumulative value of the reliability when the moving body determination unit 222 determines that the human body detected by the human body detection unit 221 is a moving body.

[0135] Next, the condition determination unit 223 determines whether the cumulative reliability R of the human body being processed is higher than the predetermined value Th_R, i.e., whether the on-screen entry condition is met (step S1610). If it is determined that the cumulative reliability R is higher than the predetermined value Th_R (step S1610: Yes), i.e., the on-screen entry condition is met, the process proceeds to step S1611. If it is determined that the cumulative reliability R is lower than the predetermined value Th_R (step S1610: No), i.e., the on-screen entry condition is not met, the process proceeds to step S1612.

[0136] The processing of steps S1611 and S1612 is the same as that of steps S411 and S412 in the first embodiment.

[0137] Figure 17 Indicates based on Figure 16 Specific examples of actions in the processing flow. Specifically, Figure 17 The following is an example of a frame, a human body identifier (ID), a detection result based on the human body detection unit 221, a determination result based on the moving body determination unit 222, a cumulative reliability R, and a pattern. Figure 17 The number of consecutive moving frames F1 in the first embodiment is also shown. The predetermined value Th_R, which is compared with the cumulative reliability R, is not particularly limited, but is set to 3000 here. That is, when the cumulative reliability exceeds 3000, a switch is made from the moving body effective mode to the moving body ineffective mode. For other aspects, it is assumed to be the same as in the first embodiment.

[0138] ID1 is the identifier for a human body. Since the accumulated reliability R exceeds 3000 in frame 7, a switch to the motion invalidation mode is performed between frames 7 and 8. Therefore, even if the human body in ID1 is stationary, the information of that human body can still be output. In the first embodiment, if the human body detected by the human body detection unit 221 is continuously identified as a moving body, and the number of consecutive moving frames F1 does not reach 5, a switch to the motion invalidation mode is not performed. Therefore, a switch to the motion invalidation mode is not performed between frames 7 and 8. On the other hand, in the second embodiment, even if the human body detected by the human body detection unit 221 is intermittently identified as a moving body, a switch to the motion invalidation mode can be appropriately performed.

[0139] ID2 is the identifier of the human body model that is mistakenly detected as a human body by the human body detection unit 221. Since the cumulative reliability R does not exceed 3000, the motion body valid mode is maintained. Since the human body model is moved during frames 1 to 6, the number of consecutive frames F1 in frame 5 reaches 5. Therefore, in the first embodiment, a switch to the motion body invalid mode is performed between frames 5 and 6, and the information of the human body model is continuously output as human body information. On the other hand, in the second embodiment, since the motion body valid mode is maintained, it is possible to suppress the output of the information of the human body model as human body information. Specifically, after the human body model comes to a standstill, the information of the human body model is not output as human body information.

[0140] As described above, in the sixth embodiment, based on the condition that the entry criteria are met, a switch to human body detection is performed according to the combination of motion body detection and human body detection, thereby enabling high-precision detection of human bodies. Furthermore, in the sixth embodiment, the probability that the human body detected by the human body detection unit 221 is a human body is calculated. A condition is used where the cumulative reliability value when the motion body determination unit 222 determines that the human body detected by the human body detection unit 221 is a moving body is higher than a predetermined value. Therefore, by applying the combination of motion body detection and human body detection to objects with low cumulative reliability, it is possible to prevent other objects resembling human bodies from being mistakenly detected as human bodies. Additionally, even if the human body repeatedly moves and remains stationary, a suitable switch from a motion body effective mode to a motion body ineffective mode can be performed.

[0141] <Other>

[0142] The above embodiments are merely illustrative examples illustrating the structure of the present invention. The present invention is not limited to the specific embodiments described above, and various modifications can be made within the scope of its technical concept. For example, in the structures using the camera entry conditions in the second, fourth, fifth, and sixth embodiments, the reset of the third embodiment (resetting to the effective mode of the moving body) can also be performed.

[0143] <Postscript 1>

[0144] Human body detection devices (100, 200) are characterized by having:

[0145] Human body detection unit (101, 221) detects human bodies from captured images;

[0146] The moving body determination unit (102, 222) determines, based on the image, whether the human body detected by the human body detection unit is a moving body;

[0147] The condition determination unit (103, 223) determines, based on the determination result of the moving object determination unit, whether the human body detected by the human body detection unit meets the prescribed conditions for the human body to be seen in the image.

[0148] The mode setting unit (104, 224) maintains the first mode until the condition determination unit determines that the specified conditions are met, and switches from the first mode to the second mode based on the condition determination unit's determination that the specified conditions are met; and

[0149] The information output unit (105, 225) outputs information about the human body when the motion determination unit determines that the human body detected by the human body detection unit is a moving body in the first mode, and outputs information about the human body in the second mode regardless of whether the motion determination unit determines that the human body detected by the human body detection unit is a moving body.

[0150] <Appendix 2>

[0151] The human body detection method is characterized by having:

[0152] Human body detection steps (S404, S604, S904, S1304, S1504, S1604) detect human bodies from the captured images;

[0153] The moving body determination step (S408, S608, S908, S1308, S1508, S1608) determines, based on the image, whether the human body detected in the human body detection step is a moving body.

[0154] The condition determination step (S410, S610, S910, S1313, S1512, S1610) determines, based on the determination result in the moving body determination step, whether the human body detected in the human body detection step meets the prescribed conditions for the human body to be seen in the image.

[0155] The mode setting steps (S411, S611, S911, S1314, S1513, S1611) continue until the condition determination step determines that the specified conditions are met, maintaining the first mode, and switching from the first mode to the second mode based on whether the specified conditions are met in the condition determination step; and

[0156] The information output steps (S412, S612, S912, S1315, S1514, S1612) are as follows: In the first mode, if the human body detected in the human body detection step is determined to be a moving body in the moving body determination step, the information of the human body is output; in the second mode, the information of the human body is output regardless of whether the human body detected in the human body detection step is determined to be a moving body in the moving body determination step.

[0157] Label Explanation

[0158] 100: Human body detection device; 101: Human body detection unit; 102: Moving body determination unit

[0159] 103: Condition Determination Unit; 104: Mode Setting Unit; 105: Information Output Unit

[0160] 10: Camera; 200: PC (human detection device)

[0161] 210: Input unit; 220: Control unit; 230: Storage unit; 240: Output unit

[0162] 221: Human Body Detection Department; 222: Movement Assessment Department; 223: Condition Assessment Department

[0163] 224: Mode Setting Unit; 225: Information Output Unit

Claims

1. A human body detection device, characterized in that, have: The human detection unit detects human bodies from the captured images; The moving body determination unit determines, based on the image, whether the human body detected by the human body detection unit is a moving body; The condition determination unit determines whether the human body detected by the human body detection unit meets the prescribed conditions for the human body to be seen in the image. The mode setting unit maintains the first mode until the condition determination unit determines that the specified conditions are met, and switches from the first mode to the second mode according to the condition determination unit's determination that the specified conditions are met. as well as The information output unit, in the first mode, outputs the information of the human body when the human body detection unit detects the human body in the image and the motion determination unit determines that the human body detected by the human body detection unit is a moving body; when the human body detection unit detects the human body in the image but the motion determination unit determines that the human body detected by the human body detection unit is not a moving body, it is considered a false detection by the human body detection unit and the information of the human body is not output. In the second mode, when the human body detection unit detects the human body in the image, the information of the human body is output regardless of whether the motion determination unit determines that the human body detected by the human body detection unit is a moving body.

2. The human body detection device as described in claim 1, characterized in that, The mode setting unit sets a mode for each human body detected by the human body detection unit.

3. The human body detection device as described in claim 1 or 2, characterized in that, In the second mode, the motion determination unit does not determine whether the human body detected by the human body detection unit is a moving body, while in the first mode, the motion determination unit determines whether the human body detected by the human body detection unit is a moving body.

4. The human body detection device as described in claim 1 or 2, characterized in that, When no human body is detected by the human body detection unit, the mode setting unit resets to the first mode.

5. The human body detection device as described in claim 1 or 2, characterized in that, The specified conditions are that the human body exercises with a greater than specified amount of exercise and for a longer than specified time.

6. The human body detection device as described in claim 1 or 2, characterized in that, The specified condition is that the total time during which the human body has exercised with a greater than specified amount of exercise during the specified period is longer than the specified time.

7. The human body detection device as described in claim 1 or 2, characterized in that, The specified condition is that the human body moves from the outside of the specified range in the image to the inside of the specified range.

8. The human body detection device as described in claim 1 or 2, characterized in that, The specified condition is that the human body moves from the inside of the specified range in the image to the outside of the specified range.

9. The human body detection device as described in claim 1 or 2, characterized in that, When detecting a human body from the image, the human body detection unit calculates the reliability of the probability that the detected human body is indeed a human body. The specified condition is that when the moving body determination unit determines that the human body detected by the human body detection unit is a moving body, the cumulative value of the reliability of that human body is higher than a specified value.

10. A method for human body detection, characterized in that, have: Human body detection steps involve detecting human bodies from captured images; The moving body determination step, based on the image, determines whether the human body detected in the human body detection step is a moving body; The condition determination step determines whether the human body detected in the human body detection step meets the prescribed conditions for the human body to be visible in the image. The mode setting step continues until the specified conditions are met in the condition determination step, maintaining the first mode, and switching from the first mode to the second mode based on the condition being met in the condition determination step. as well as In the information output step, in the first mode, if the human body is detected from the image in the human body detection step and the moving body determination step determines that the human body detected in the human body detection step is a moving body, the information of the human body is output. If the human body is detected from the image in the human body detection step, but the moving body determination step determines that the human body detected in the human body detection step is not a moving body, it is considered a false detection in the human body detection step, and the information of the human body is not output. In the second mode, if the human body is detected from the image in the human body detection step, the information of the human body is output regardless of whether the moving body determination step determines that the human body detected in the human body detection step is a moving body.

11. A computer-readable storage medium storing a program for causing a computer to perform the steps of the human body detection method of claim 10.

Images