Human detection device, human detection system, program, and human detection method

Abstract

A human detection device (110) comprises: a human body part detection unit (112) that detects body parts of a person in a predetermined space from image data representing an image of the space, and detects a three-dimensional position of the head of the person in the space; a candidate foot area calculation unit (115) that refers to mapping information, which indicates the three-dimensional size and position of the floor and objects placed in the space, to calculate, if there is an object below the head, a plurality of coordinates below the head as candidate foot areas of the person; a height calculation unit (116) that calculates a plurality of heights as a height of the person on the basis of the plurality of coordinates and the position of the head; and a height assessment unit (118) that specifies the most suitable, single height among the plurality of heights to specify coordinates corresponding to the specified height as the position of the person.

Description

Technical Field

[0001] The present disclosure relates to a human detection device, a human detection system, a program, and a human detection method.

Background Art

[0002] In the field of image monitoring, a human detection device, which is a device for detecting a person from an image, is widely used. The human detection device is utilized for detecting human intrusion, estimating human attributes, measuring congestion level, etc. There are also various human detection methods, and human detection is achieved by searching for edge features of a human silhouette or using deep learning.

[0003] On the other hand, when detecting a person, a part of the person's body is often not detected because it is blocked by an obstacle or the like. As a conventional technique for preventing detection errors due to occlusion, a person determination device described in Patent Document 1 is known.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] In the above conventional technique, depth information of the floor surface is estimated from the installation position information of the camera, the height of a person is calculated from the target person area, and the person is detected. In this technique, even when the feet of the person area are blocked by an obstacle or the like, since the floor surface information is retained, the height of the person can be calculated from the estimated floor surface.

[0006] However, in the conventional technique, the determination is made on the premise that the feet are in contact with the floor surface. When the person is not standing on the floor surface, for example, when the person is on a bed, a higher height than normal is estimated and it is determined as a false detection.

[0007] Therefore, one or more aspects of this disclosure aim to enable the appropriate determination of a person's position based on their height, even when the person is not standing on the floor. [Means for solving the problem]

[0008] A person detection device according to one aspect of the present disclosure is characterized by comprising: a person part detection unit that detects a person part in a space from image data showing an image of a predetermined space and detects the three-dimensional position of the person's head in the space; a foot candidate calculation unit that calculates a plurality of coordinates as candidates for the person's feet below the head by referring to mapping information showing the size and position of the floor surface and objects arranged in the space in three dimensions, if there is an object below the head; a height calculation unit that calculates a plurality of heights as the person's height from the plurality of coordinates and the position of the head; and a height verification unit that identifies the most suitable height from the plurality of heights and identifies the coordinates corresponding to the one height among the plurality of coordinates as the person's position.

[0009] A person detection system according to one aspect of the present disclosure is a person detection system comprising a person detection device and a server, wherein the person detection device comprises: a person part detection unit that detects a person part in a space from image data showing an image of a predetermined space and detects the three-dimensional position of the person's head in the space; a foot candidate calculation unit that calculates a plurality of coordinates as candidates for the person's feet below the head by referring to mapping information showing the size and position of the floor surface and objects arranged in the space in three dimensions, if there is an object below the head; a height calculation unit that calculates a plurality of heights as the person's height from the plurality of coordinates and the position of the head; a height verification unit that identifies the most suitable height from the plurality of heights and identifies the coordinates corresponding to the one height among the plurality of coordinates as the person's position, and the server determines whether or not there is an abnormality from the person's position.

[0010] A program according to one aspect of this disclosure is characterized in that it causes a computer to function as: a human body part detection unit that detects human body parts in a predetermined space from image data showing an image of the space and detects the three-dimensional position of the human head in the space; a foot candidate calculation unit that calculates a plurality of coordinates as candidates for the human feet below the head by referring to mapping information that shows the size and position of the floor surface and objects placed in the space in three dimensions, if there is an object below the head; a height calculation unit that calculates a plurality of heights as the human height from the plurality of coordinates and the position of the head; and a height verification unit that identifies the most suitable height from the plurality of heights and identifies the coordinates corresponding to the one height among the plurality of coordinates as the human position, which is the position of the human.

[0011] A person detection method relating to one aspect of this disclosure is: A method for detecting a person that is performed by a computer, The method is characterized by detecting a person's body part within a predetermined space from image data showing an image of the space, detecting the three-dimensional position of the person's head in the space, referring to mapping information that shows the size and position of the floor surface and objects placed in the space in three dimensions, calculating multiple coordinates as candidates for the person's feet below the head if there is an object below the head, calculating multiple heights as the person's height from the multiple coordinates and the position of the head, identifying the most suitable height from the multiple heights, and identifying the coordinates corresponding to the one height among the multiple coordinates as the person's position. [Effects of the Invention]

[0012] According to one or more aspects of this disclosure, even if a person is not standing on the floor, their position can be appropriately determined from their height. [Brief explanation of the drawing]

[0013] [Figure 1] This is a block diagram schematically showing the configuration of a human detection system including a human detection device according to Embodiment 1. [Figure 2] It is a block diagram schematically showing the configuration of the human detection device according to Embodiment 1. [Figure 3] It is a schematic diagram for explaining the relationship between a person and an object. [Figure 4] It is a flowchart showing the operation in the human detection device. [Figure 5] It is a block diagram schematically showing the configuration of a human detection system including the human detection device according to Embodiment 2. [Figure 6] It is a block diagram schematically showing the configuration of the human detection device according to Embodiment 2. [Figure 7] It is a block diagram schematically showing the configuration of a server. [Figure 8] (A) and (B) are block diagrams showing hardware configuration examples.

MODE FOR CARRYING OUT THE INVENTION

[0014] Embodiment 1. FIG. 1 is a block diagram schematically showing the configuration of a human detection system 100 including a human detection device 110 according to Embodiment 1. The human detection system 100 includes a camera 101 that functions as an imaging unit, a human detection device 110, a display 102 as a display unit, and a speaker 103 as an audio output unit.

[0015] The camera 101 captures an image of a predetermined space and provides image data indicating the image to the human detection device 110. The camera 101 may use, for example, a CCD (Charge Coupled Device) or the like, and the image obtained from the camera 101 may be an RGB (Red Green Blue) image, an IR (infrared) image, or a distance image. <"

[0016] The human detection device 110 detects a person from the image indicated by the image data provided from the camera 101. Then, the human detection device 110 determines whether the position of the detected person is worthy of reporting. If it is determined that it is worthy of reporting, it causes at least one of the display 102 and the speaker 103 to give a notification.

[0017] The display 102 displays information indicating an abnormality detected by the human detection device 110. The speaker 103 outputs information indicating an abnormality detected by the human detection device 110 as voice.

[0018] FIG. 2 is a block diagram schematically showing the configuration of the human detection device 110 according to Embodiment 1. The human detection device 110 includes an input interface unit (hereinafter referred to as input I / F unit) 111, a human part detection unit 112, a layout information storage unit 113, a layout estimation unit 114, a foot candidate calculation unit 115, a height calculation unit 116, a threshold information storage unit 117, a height verification unit 118, an abnormality determination unit 119, and an output interface unit (hereinafter referred to as output I / F unit) 120.

[0019] The input I / F unit 111 receives input of image data from the camera 101. The input image data is provided to the human part detection unit 112 and the layout estimation unit 114.

[0020] The human part detection unit 112 acquires image data from the input I / F unit 111 and detects the parts of a person in the space where the image is captured from the image shown by the image data. As a method for detecting the parts of a person, a known method may be adopted. For example, it is preferable to use skeleton detection using deep learning represented by the following Document 1. Document 1: Zhe Cao, Gines Hidalgo, Tomas Simon, Shih-En Wei, Yaser Sheikh, “OpenPose: Realtime Multi-Person 2D Pose Estimation using Parts Sffinity Fields”, arXiv:1812.08008v2, 30 May, 2019

[0021] Specifically, the human body part detection unit 112 may detect 17 joint points such as the eyes, shoulders, hands, elbows, waist, and feet calculated using a skeletal detector as described in the above-mentioned reference 1, or it may detect a part of the human silhouette estimated from background subtraction or optical flow.

[0022] The human body part detection unit 112 then identifies the head from the detected body parts and detects the three-dimensional position of the identified head in the space captured in the image data. The human body part detection unit 112 provides the head position information indicating the detected position to the foot candidate calculation unit 115 and the height calculation unit 116. Furthermore, three-dimensional object position information can be detected even from images captured by a monocular camera using a technique called 3D Object Detection, as shown in reference 2 below. Document 2: Thomas Roddick, Alex Kendall, Roberto Cipolla, “Ohthographic Feature Transform for Monocular 3D Obhect Detection”, arXiv:1811.0818v1, 20 November 2018 The human body part detection unit 112 may also estimate the posture of a person from the image.

[0023] Here, "head" refers to the reference point for measuring height. In other words, the head is the top of the head (or, to put it another way, the crown of the head). When detecting from an image, the top of the head may be detected as the head, or the upper edge of the bounding box where the head was detected may be considered the head.

[0024] The layout information storage unit 113 stores layout information indicating the size and position of the floor surface and objects in the image captured by the camera 101.

[0025] The layout estimation unit 114 identifies the size and position of objects such as furniture and the floor surface within the field of view from the image data shown by the input I / F unit 111 through object detection. The layout estimation unit 114 then generates mapping information indicating the relative positions of the floor surface and objects with respect to the camera 101, in other words, the three-dimensional positions in space. Three-dimensional object position information can be identified, for example, by a method called 3D Object Detection as described in the above-mentioned reference 2.

[0026] Alternatively, instead of the above processing, the layout estimation unit 114 may refer to the size and position of the floor surface and objects indicated by the layout information stored in the layout information storage unit 113 and generate mapping information indicating the relative position of the floor surface and objects included in the image data with respect to the camera 101. In such cases, the layout estimation unit 114 needs to update the layout information each time an object is placed. The layout estimation unit 114 may also estimate the relative position of the object to the camera 101 and the floor surface based on the placement position and size of the object, such as furniture, which have been input in advance.

[0027] In other words, the layout estimation unit 114 may estimate the size and position of the floor and objects from an image of a space without people and generate mapping information, or it may generate mapping information by referring to layout information that shows the size and position of the floor and objects in that space. The layout estimation unit 114 provides the above mapping information to the foot candidate calculation unit 115.

[0028] The foot candidate calculation unit 115 refers to the mapping information generated by the layout estimation unit 114 and calculates the foot coordinates corresponding to the human head, indicated by the head position information provided by the human body part detection unit 112, as foot candidate coordinates. For example, the foot candidate calculation unit 115 refers to the mapping information generated by the layout estimation unit 114 and calculates where a person's feet are located in the mapping information, based on the position of the person's head indicated by the head position information provided by the person part detection unit 112.

[0029] Specifically, the foot candidate calculation unit 115 draws a perpendicular line from the position of the head detected by the human body part detection unit 112 to the plane corresponding to the floor surface indicated by the mapping information, and defines the point where that perpendicular line intersects the floor surface as the foot candidate coordinate. In this case, situations can arise where multiple cases are possible, such as when a person 105 is detected at the edge of the bed 106, as shown in Figure 3, where it is possible to determine whether the person is standing on an object or standing on the floor surface at the back of the bed.

[0030] As described above, if there is a possibility that an object such as furniture exists below the position of the person's head detected by the person body part detection unit 112, for example, if the object is within a predetermined range from the intersection of the perpendicular line from the position of the person's head and the floor surface or the floor surface obscured by the object, the foot candidate calculation unit 115 calculates the coordinates of that intersection and the coordinates of the point on the upper surface of the object that is closest to that intersection as foot candidate coordinates.

[0031] In other words, the foot candidate calculation unit 115, by referring to the mapping information, calculates multiple coordinates as candidates for the person's feet below the head's position if there is an object below the head. Here, the foot candidate calculation unit 115 calculates multiple foot candidate coordinates, including the coordinate of the area below the head on the floor surface and the coordinate of the area below the head on the upper surface of an object located below the head.

[0032] The foot coordinate information, which indicates the coordinates of the candidate feet calculated by the foot candidate calculation unit 115, is provided to the height calculation unit 116.

[0033] The height calculation unit 116 calculates the person's height using the head detected by the human body part detection unit 112 and the candidate foot coordinates calculated by the candidate foot coordinate calculation unit 115. For example, the height calculation unit 116 calculates a person's height based on the length of the line segment between the person's head and the candidate coordinates of their feet. If multiple candidate coordinates of feet are calculated, the height calculation unit 116 calculates multiple heights based on the person's head and each of the multiple candidate coordinates of feet. In other words, the height calculation unit 116 calculates multiple heights as the person's height based on the multiple candidate coordinates of feet and the position of the head.

[0034] As described above, since the candidate coordinates of the feet are mapped into three-dimensional space using mapping information, the actual height can be calculated from the line segments in the image. The height calculation unit 116 generates height information indicating the calculated height and provides this height information to the height verification unit 118.

[0035] The height calculation unit 116 may also calculate height while taking into account the person's posture. For example, if the human body part detection unit 112 detects a person's posture, information indicating the detected posture is also notified to the height calculation unit 116. If the posture is seated, the height calculation unit 116 can calculate the height by doubling the value obtained from the length between the person's head and the candidate coordinates of their feet. If the posture is lying down, the height calculation unit 116 can calculate the height by multiplying the value obtained from the length between the person's head and the candidate coordinates of their feet by ten. If the posture is standing, the height calculation unit 116 can use the value obtained from the length between the person's head and the candidate coordinates of their feet as the person's height. As described above, the height calculation unit 116 can calculate a person's height that takes their posture into account by multiplying the value obtained from the length between the person's head and the candidate coordinates of their feet by a predetermined coefficient according to the detected posture.

[0036] The threshold information storage unit 117 stores threshold information indicating height thresholds. For example, the threshold information indicates the lower and upper limits of height as thresholds.

[0037] The height verification unit 118 identifies the most suitable height from among multiple heights, and identifies the foot candidate coordinates corresponding to that height from among multiple foot candidate coordinates as the person's position. For example, the height verification unit 118 identifies a height by comparing a predetermined threshold with multiple heights.

[0038] Specifically, the height verification unit 118 determines whether the calculated height is appropriate by comparing the height calculated by the height calculation unit 116 for each candidate foot coordinate with the threshold indicated by the threshold information stored in the threshold information storage unit 117. The height verification unit 118 then generates detection position information indicating the detected position, using the candidate foot coordinates corresponding to the height determined to be appropriate as the person's detection position. The height verification unit 118 then provides this detection position information to the anomaly determination unit 119. If multiple heights are deemed suitable, the height verification unit 118 only needs to identify one height using a predetermined method, such as the height closest to the median of the thresholds, the height closest to the representative value of past heights, or the height closest to the height of the inhabitants of that space. The detection position information is person position information indicating the detected person's location.

[0039] Here, the height verification unit 118 compares the height calculated for each candidate foot position by the height calculation unit 116 with the threshold information pre-stored in the threshold information storage unit 117, and deletes candidate foot position coordinates where the height exceeds the upper limit and candidate foot position coordinates where the height falls below the lower limit.

[0040] For example, in a residence for an elderly person living alone, if a candidate coordinate for the feet of a person taller than 1.8m appears, or if a candidate coordinate for the feet of a person taller than 1m appears, the height verification unit 118 deletes the corresponding candidate coordinate for the feet as an abnormal detection value.

[0041] Furthermore, even when multiple potential foot coordinates are calculated for a single head, such as when a person's head is detected at the edge of a bed, the height is calculated for each potential foot coordinate, and by comparing that height with a threshold, the person's position can be accurately determined. The threshold only needs to be predetermined. For example, if the height of a resident is known, the threshold can be predetermined based on that height. Specifically, the upper limit can be set by adding a predetermined value (e.g., 2.5 cm) to the height, and the lower limit by subtracting a predetermined value (e.g., 2.5 cm) from the height.

[0042] Alternatively, the height of residents may be determined based on past height estimation results. For example, the average, median, or mode of heights calculated over a recent predetermined period, specifically a 24-hour period, can be used as the representative value for that resident's height.

[0043] Furthermore, for example, by storing data in the threshold information storage unit 117 that links a person's face and other features with that person's height, the height verification unit 118 can detect the person's features from the image data and determine that person's height from that data. The height verification unit 118 can then determine a threshold from the determined height.

[0044] Furthermore, the height verification unit 118 can update the threshold based on the height of a person detected after the installation of the person detection device 110. For example, the height verification unit 118 may pre-determine a default value for the threshold and update the threshold using representative values ​​such as the average, median, or mode of height calculated over the most recent predetermined period, specifically, a 24-hour period. However, if, for example, a person of a specific height does not appear at a frequency exceeding a predetermined threshold during a predetermined period, the height verification unit 118 may use the default value as the threshold, or it may determine the threshold from the average height detected in the past.

[0045] Furthermore, if the height verification unit 118 obtains the height of an inhabitant in the space via an input unit (not shown) or the like after a predetermined threshold has been set, it may update the threshold using that height.

[0046] The abnormality determination unit 119 determines whether or not there is an abnormality based on the person's position indicated by the detected position information provided by the height verification unit 118. For example, the anomaly detection unit 119 detects whether there is a person in a location that warrants reporting, based on the person's location indicated by the detected location information. The anomaly detection unit 119 then determines that there is an anomaly if the person remains in that location for a predetermined amount of time or longer. A location that warrants reporting refers to, for example, a situation where a person is detected on a bed or furniture where there is a risk of falling, or a situation where a person is detected in a location where they appear to be wandering.

[0047] Here, the abnormality detection unit 119 may determine whether an abnormal state exists by taking into account the person's posture. For example, if the human body part detection unit 112 detects a person's posture that is not standing, sitting, or lying down, but also includes postures such as a violent posture, a convulsive posture, an unsteady posture, a vomiting posture, a defecation posture, or other postures that warrant rushing over or assistance, it may determine that the person is in an abnormal state. The human body part detection unit 112 may also detect posture by pre-learning or modeling human postures using AI (Artificial Intelligence).

[0048] If the abnormality detection unit 119 determines that an abnormality exists, it will notify an external device that an abnormality has occurred. For example, if the abnormality detection unit 119 determines that there is an abnormality, it will notify the abnormality via the output I / F unit 120. For example, the abnormality detection unit 119 will display information indicating an abnormality on the display 102 via the output I / F unit 120, or output a sound indicating an abnormality from the speaker 103. The abnormality detection unit 119 may also display information indicating an abnormality on the display 102 and output a sound indicating an abnormality from the speaker 103.

[0049] The output I / F unit 120 communicates with connected external devices. For example, the output I / F unit 120 is connected to the display 102 and the speaker 103, and communicates with them.

[0050] Figure 4 is a flowchart showing the operation of the human detection device 110. First, the input I / F unit 111 acquires image data of residents from an imaging device such as a camera 101 installed at a high place in a room (S10). The acquired image data is provided to the human body part detection unit 112 and the layout estimation unit 114. The imaging range can be any space where people are likely to be captured, and the installation location is not limited as long as it is possible to shoot from an angle that looks down on public facilities or elderly care facilities, etc. The installation position of the camera 101 is also stored in advance. The layout estimation unit 114 performs mapping including depth information using a monocular camera by estimating the positions of the floor surface and objects from the image captured by the camera 101.

[0051] When the human body part detection unit 112 receives image data, it performs detection of human body parts from the image data and detects the head of the person (S11). The human body part detection unit 112 may also estimate the posture of the person in the image.

[0052] Furthermore, the layout estimation unit 114 estimates the size and position of objects such as furniture and the floor surface in the image in order to estimate the layout of the room, or it refers to the layout information stored in the layout information storage unit 113 to map the floor surface and objects in the room in 3D (three dimensions) and generates mapping information that shows the relative position of the floor surface and objects with respect to the camera 101 (S12). The mapping information is provided to the foot-level candidate calculation unit 115.

[0053] As shown in Figure 4, the layout estimation step S12 is performed in parallel with the human body part estimation step S11, but Embodiment 1 is not limited to this example. For example, the layout estimation step S12 may be performed before or after the human body part estimation step S11.

[0054] The foot candidate calculation unit 115 calculates the foot candidate coordinates from the mapping information estimated in the room layout estimation step S13 and the head estimated in the human body part estimation step S11. The calculated foot candidate coordinates are provided to the height calculation unit 116.

[0055] The height calculation unit 116 calculates the person's height for each candidate coordinate of the feet, based on the length to the corresponding head (S14). The calculated height is then provided to the height verification unit 118.

[0056] The height verification unit 118 identifies a suitable height by comparing the height for each alternating foot coordinate with the threshold indicated by the threshold information stored in the threshold information storage unit 117, and identifies the candidate foot coordinates corresponding to the identified height as the person's position (S15). The person's position is then provided to the abnormality determination unit 119.

[0057] The abnormality detection unit 119 determines whether or not there is an abnormality based on the person's position (S16). Then, if the abnormality detection unit 119 determines that there is an abnormality, it notifies the system via the output I / F unit 120 (S17).

[0058] As described above, according to Embodiment 1, false detection of people can be suppressed, and the position of a person can be accurately estimated even when a person is standing on furniture or the like. Therefore, accidents can be prevented.

[0059] Embodiment 2. Figure 5 is a schematic block diagram showing the configuration of a human detection system 200 including a human detection device 210 according to Embodiment 2. The human detection system 200 comprises a camera 101, a human detection device 210, a display 102 as a display unit, a speaker 103 as an audio output unit, and a server 230.

[0060] The camera 101, display 102, and speaker 103 of the human detection system 200 in Embodiment 2 are the same as those of the human detection system 100 in Embodiment 1. However, in Embodiment 2, the display 102 and speaker 103 are connected to the server 230.

[0061] Figure 6 is a block diagram schematically showing the configuration of the human detection device 210 according to Embodiment 2. The human detection device 210 includes an input I / F unit 111, a human body part detection unit 112, a layout information storage unit 113, a layout estimation unit 114, a foot candidate calculation unit 115, a height calculation unit 116, a threshold information storage unit 117, a height verification unit 118, and a communication unit 221.

[0062] The input I / F unit 111, human body part detection unit 112, layout information storage unit 113, layout estimation unit 114, foot candidate calculation unit 115, height calculation unit 116, threshold information storage unit 117, and height verification unit 118 of the human detection device 210 according to Embodiment 2 are the same as those of the input I / F unit 111, human body part detection unit 112, layout information storage unit 113, layout estimation unit 114, foot candidate calculation unit 115, height calculation unit 116, threshold information storage unit 117, and height verification unit 118 of the human detection device 110 according to Embodiment 1. However, in Embodiment 2, the height verification unit 118 provides the generated detected position information to the communication unit 221.

[0063] The communication unit 221 communicates with the server 230. For example, the communication unit 221 functions as a transmitter that sends detected position information from the height verification unit 118 to the server 230.

[0064] Figure 7 is a block diagram illustrating the configuration of server 230. The server 230 includes a communication unit 231, an anomaly detection unit 232, and an output I / F unit 233.

[0065] The communication unit 231 communicates with the human detection device 210. For example, the communication unit 231 receives detection location information from the human detection device 210. The received detection location information is provided to the anomaly determination unit 232.

[0066] The abnormality determination unit 232 determines whether or not there is an abnormality based on the location indicated by the detection location information provided by the communication unit 231. The processing here is the same as the processing in the abnormality determination unit 119 in the person detection device 110 in Embodiment 1. If the abnormality detection unit 232 determines that an abnormality exists, it notifies the abnormality via the output I / F unit 233. For example, the abnormality detection unit 232 may display information indicating an abnormality on the display 102 via the output I / F unit 233, or output a sound indicating an abnormality from the speaker 103. The abnormality detection unit 232 may also display information indicating an abnormality on the display 102 and output a sound indicating an abnormality from the speaker 103.

[0067] The abnormality determination unit 232 may also accumulate determination results obtained from multiple human detection devices 210 and make a comprehensive determination as to whether or not an abnormal state exists based on those determination results. For example, by installing multiple cameras 101 and human detection devices 210 in a given space, multiple detection results can be obtained for each person in that space. By estimating from multiple viewpoints, the blind spots of the cameras are reduced, and the anomaly determination unit 232 can accurately estimate whether or not there is an anomaly. Furthermore, the abnormality determination unit 232 can reduce detection omissions by scrutinizing the abnormality determination results based on detection location information obtained from multiple person detection devices 210 using OR conditions. Furthermore, the abnormality detection unit 232 can suppress false positives by scrutinizing the abnormality detection results based on detection location information obtained from multiple person detection devices 210 using an AND condition.

[0068] The output I / F unit 233 communicates with connected external devices. For example, the output I / F unit 233 is connected to the display 102 and the speaker 103, and communicates with them.

[0069] Some or all of the human body part detection unit 112, layout estimation unit 114, foot candidate calculation unit 115, height calculation unit 116, height verification unit 118, and abnormality determination unit 119 described above can be configured, for example, with a memory 10 and a processor 191 such as a CPU (Central Processing Unit) that executes a program stored in the memory 10, as shown in Figure 8(A). Such a program may be provided via a network or recorded on a recording medium. That is, such a program may be provided, for example, as a program product. In other words, the human detection devices 110 and 210 can be implemented using a so-called computer.

[0070] Furthermore, some or all of the human body part detection unit 112, layout estimation unit 114, foot candidate calculation unit 115, height calculation unit 116, height verification unit 118, and abnormality determination unit 119 can also be composed of processing circuits 12 such as a single circuit, a composite circuit, a program-operated processor, a program-operated parallel processor, an ASIC (Application Specific Integrated Circuit), or an FPGA (Field Programmable Gate Array), as shown in Figure 8(B). As described above, the human body part detection unit 112, layout estimation unit 114, foot candidate calculation unit 115, height calculation unit 116, height verification unit 118, and abnormality determination unit 119 can be realized by a processing circuit network.

[0071] The layout information storage unit 113 and the threshold information storage unit 117 can be implemented using a storage device such as an HDD (Hard Disc Drive) or SSD (Solid State Drive). The input I / F section 111 can be implemented using an input interface such as a communication interface like a NIC (Network Interface Card) or a connection interface like USB (Universal Serial Bus). The output I / F section 120 can be implemented using an output interface such as a communication interface (NIC) or a connection interface such as USB. Furthermore, the communication unit 221 can be implemented using a communication interface such as a NIC.

[0072] Furthermore, part or all of the abnormality detection unit 232 can also be configured, for example, as shown in Figure 8(A), with a memory 10 and a processor 191 such as a CPU that executes the program stored in the memory 10. Such a program may be provided via a network, or it may be provided by being recorded on a recording medium. That is, such a program may be provided, for example, as a program product. In other words, the server 230 can be implemented by a so-called computer.

[0073] Furthermore, part or all of the abnormality detection unit 232 can also be composed of a processing circuit 12 such as a single circuit, a composite circuit, a program-operated processor, a program-operated parallel processor, an ASIC, or an FPGA, as shown in Figure 8(B). As described above, the abnormality detection unit 232 can be implemented by a processing circuit network.

[0074] The communication unit 231 can be implemented using a communication interface such as a NIC. The output I / F section 233 can be implemented using an output interface such as a communication interface (NIC) or a connection interface such as USB. [Explanation of Symbols]

[0075] 100,200 person detection system, 101 camera, 102 display, 103 speaker, 110,210 person detection device, 111 input I / F unit, 112 person body part detection unit, 113 layout information storage unit, 114 layout estimation unit, 115 foot candidate calculation unit, 116 height calculation unit, 117 threshold information storage unit, 118 height verification unit, 119 anomaly determination unit, 120 output I / F unit, 221 communication unit, 230 server, 231 communication unit, 232 anomaly determination unit, 233 output I / F unit.

Claims

1. A human body part detection unit detects human body parts within a predetermined space from image data showing an image of the space, and detects the three-dimensional position of the human head in the space. By referring to mapping information that shows the size and position of the floor surface and objects arranged in the space in three dimensions, if the object is below the head, a foot candidate calculation unit calculates multiple coordinates as candidates for the person's feet below the head, A height calculation unit that calculates multiple heights as the height of the person based on the aforementioned multiple coordinates and the position of the head, The system includes a height verification unit that identifies the most suitable height from the aforementioned multiple heights, and identifies the coordinates corresponding to the aforementioned height from among the aforementioned multiple coordinates as the person's position. A human detection device characterized by the following.

2. The foot candidate calculation unit calculates the coordinates of the area below the head on the floor surface and the coordinates of the area below the head on the upper surface of the object located below the head as the plurality of coordinates. A person detection device according to claim 1, characterized by the following:

3. The height verification unit identifies one height by comparing a predetermined threshold with the multiple heights. A person detection device according to claim 1, characterized by the following:

4. The height verification unit identifies one height by comparing a predetermined threshold with the multiple heights. A person detection device according to claim 2, characterized by the following:

5. The height verification unit updates the predetermined threshold using a representative height value calculated in the past. A human detection device according to claim 3 or 4, characterized by the above.

6. The height verification unit determines or updates the predetermined threshold using the height of the person residing in the space. A human detection device according to claim 3 or 4, characterized by the above.

7. The system further includes a layout estimation unit that estimates the size and position of the floor surface and the objects from an image of the empty space and generates the mapping information. A human detection device according to any one of claims 1 to 4, characterized by the following:

8. The system further includes a layout estimation unit that generates the mapping information by referring to layout information indicating the size and position of the floor surface and the objects in the space. A human detection device according to any one of claims 1 to 4, characterized by the following:

9. The system further includes an abnormality determination unit that determines whether or not there is an abnormality based on the aforementioned human position. A human detection device according to any one of claims 1 to 4, characterized by the following:

10. If the abnormality detection unit determines that an abnormality exists, it will cause an external device to notify the abnormality. A person detection device according to claim 9, characterized by the following:

11. The system further includes a transmission unit that transmits the aforementioned person location information to an external device. A human detection device according to any one of claims 1 to 4, characterized by the following:

12. A human detection system comprising a human detection device and a server, The aforementioned human detection device is A human body part detection unit detects human body parts within a predetermined space from image data showing an image of the space, and detects the three-dimensional position of the human head in the space. By referring to mapping information that shows the size and position of the floor surface and objects arranged in the space in three dimensions, if the object is below the head, a foot candidate calculation unit calculates multiple coordinates as candidates for the person's feet below the head, A height calculation unit that calculates multiple heights as the height of the person based on the aforementioned multiple coordinates and the position of the head, The system includes a height verification unit that identifies the most suitable height from the aforementioned multiple heights, and identifies the coordinates corresponding to the aforementioned height from among the aforementioned multiple coordinates as the person's position, which is the person's location. The server determines whether or not there is an abnormality based on the location of the person. A human detection system characterized by the following.

13. Computers, A human body part detection unit detects human body parts within a predetermined space from image data showing an image of the space, and detects the three-dimensional position of the human head in the space. By referring to mapping information that shows the size and position of the floor surface and objects arranged in the aforementioned space in three dimensions, if the object is below the head, a foot candidate calculation unit calculates multiple coordinates as candidates for the person's feet below the head. A height calculation unit that calculates multiple heights as the height of the person from the aforementioned multiple coordinates and the position of the head, and This height verification unit functions to identify the most suitable height from the aforementioned multiple heights, and to identify the coordinates corresponding to the aforementioned height from among the aforementioned multiple coordinates as the person's position. A program characterized by the following.

14. A method for detecting a person, which is performed by a computer. From image data showing an image of a predetermined space, a person's body part is detected within the space, and the three-dimensional position of the person's head in the space is detected. By referring to mapping information that shows the size and position of the floor surface and objects placed in the aforementioned space in three dimensions, if the object is below the head, multiple coordinates are calculated below the head as candidates for the person's feet. From the aforementioned multiple coordinates and the position of the head, multiple heights are calculated as the height of the person. Identify the most suitable height from the aforementioned multiple heights, and identify the coordinates corresponding to the aforementioned height from among the aforementioned multiple coordinates as the person's position. A person detection method characterized by the following.

Images