Method for identifying personal attributes, device for identifying personal attributes, and program
The system uses posture estimation and skeletal information to identify headwear attributes at work sites, addressing the challenge of automating personal attribute recognition for obscured faces, enhancing safety management efficiency and reducing supervisor workload.
Patent Information
- Authority / Receiving Office
- JP · JP
- Patent Type
- Patents
- Current Assignee / Owner
- NAKAYO INC
- Filing Date
- 2023-02-16
- Publication Date
- 2026-07-29
AI Technical Summary
Existing safety management systems at work sites, such as manufacturing and construction sites, face challenges in automating the identification of personal attributes of individuals, particularly for unregistered visitors and those wearing protective gear like dust masks or goggles, which hinders efficient safety management.
A system using a camera and a person attribute identification device performs posture estimation to acquire skeletal information, identifies the head region, and determines the attributes of headwear, such as helmets and hats, based on positional coordinates of body landmarks like shoulders, ankles, and ears, enabling automatic identification of personal attributes.
The system efficiently and automatically identifies personal attributes of workers and visitors, even when face information is obscured, reducing the workload of supervisors and enhancing safety management by providing visual cues and alarms.
Smart Images
Figure 0007896511000005 
Figure 0007896511000006 
Figure 0007896511000007
Abstract
Description
Technical Field
[0001] The present invention relates to a technology for automatically identifying the personal attributes of on-site personnel, such as workers, in work sites such as manufacturing sites and construction sites.
Background Art
[0002] Conventionally, various efforts have been made to promote safety management in work sites such as manufacturing sites and construction sites. For example, cameras are installed at work sites, and administrators may remotely perform safety management according to the attributes and levels of the people shown in the images of these cameras.
[0003] Also, Patent Document 1 discloses a person estimation technology that extracts face-related information from a photographed image including a subject person, identifies the face-related information with the highest degree of matching from the face-related information of pre-registered persons, and estimates the subject person as the person associated with this identified face-related information.
Prior Art Documents
Patent Documents
[0004]
Patent Document 1
Summary of the Invention
Problems to be Solved by the Invention
[0005] Automation of the above-mentioned safety management operations is desired for reasons such as reducing the burden on administrators. Here, the technology described in Patent Document 1 is not suitable for application to the above-mentioned automation of safety management because it cannot estimate unregistered persons such as visitors to a factory tour, and it is difficult to identify face-related information when a person is wearing a dust mask or work goggles.
[0006] Meanwhile, in recent years, efforts have been made to improve the efficiency of safety management at work sites by color-coding the helmets worn by workers, visitors, and other individuals entering the work site, thereby making the attributes of the helmet wearers visible.
[0007] This invention has been made in view of the above circumstances, and aims to provide a technology for automatically identifying the personal attributes of a person captured in video footage from a camera installed at a work site. [Means for solving the problem]
[0008] To solve the above problems, the present invention uses a camera installed at the work site and a person attribute identification device connected to this camera. The person attribute identification device performs posture estimation processing on the person displayed in the camera image to acquire the person's skeletal information and identifies the person's head region based on the acquired skeletal information. Then, by identifying the features of the person's head region from the camera image, it determines the features of headwear such as helmets and hats worn on the person's head and identifies the person's attributes associated with the determined features.
[0009] For example, the present invention is A method for identifying the personal attributes of a person displayed in the image of a camera, using a camera and a person attribute identification device connected to the camera, The aforementioned person attribute identification device, A posture estimation process is performed on the person captured in the image of the aforementioned camera to obtain the person's skeletal information. The skeletal information acquired as described above The positional coordinates of both ankles, both shoulders, both eyes, and both ears are included. Based on this, the head region of the person shown in the image from the camera is identified, The characteristics of the identified head region are determined from the image of the aforementioned camera. The person attributes that are pre-associated with the characteristics determined above are identified as the person attributes of the person shown in the camera's video. It is, If the horizontal direction of the camera image is defined as the x-direction and the vertical direction as the y-direction, and the x-distance between the two shoulders is a, the y-distance from the right eye to the right ear is b1, and the y-distance from the left eye to the left ear is b2, then the four coordinate values (x1, y1), (x2, y2), (x3, y3), and (x4, y4) that define the head region as a rectangle are: x1 is the x-coordinate value of the right eye, y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is the x-coordinate value of the left eye, y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the right eye, y3 is a coordinate value that is b1 away from the y coordinate value of the right eye, in the y direction away from the ankles relative to the right eye. x4 is the x-coordinate value of the left eye, and, y4 is the y-coordinate value at a position b2 away from the y-coordinate value of the left eye, in the y-direction opposite to both ankles relative to the left eye. Alternatively, the present invention is A method for identifying the personal attributes of a person displayed in the image of a camera, using a camera and a person attribute identification device connected to the camera, The aforementioned person attribute identification device, A posture estimation process is performed on the person captured in the image of the aforementioned camera to obtain the person's skeletal information. Based on the position coordinates of both ankles, both shoulders, and both ears included in the skeletal information of the person obtained by the posture estimation process, the head region of the person displayed in the camera's image is identified. The characteristics of the identified head region are determined from the image of the aforementioned camera. The person attributes that are pre-associated with the characteristics determined above are identified as the person attributes of the person shown in the camera's video. If the horizontal direction of the image from the aforementioned camera is defined as the x-direction and the vertical direction as the y-direction, and the x-direction distance between the two shoulders is defined as a, then the four coordinate values (x1, y1), (x2, y2), (x3, y3), and (x4, y4) that define the head region as a rectangle are: x1 is the x-coordinate value of the right ear. y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is the x-coordinate value of the left ear. y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the right ear, y3 is the y-coordinate value of the right ear. x4 is the x-coordinate value of the left ear, and, y4 is the y-coordinate value of the left ear. Alternatively, the present invention is A method for identifying the personal attributes of a person displayed in the image of a camera, using a camera and a person attribute identification device connected to the camera, The aforementioned person attribute identification device, A posture estimation process is performed on the person captured in the image of the aforementioned camera to obtain the person's skeletal information. Based on the position coordinates of both ankles, both shoulders, and one ear included in the skeletal information of the person obtained by the posture estimation process, the head region of the person displayed in the camera's image is identified. The characteristics of the identified head region are determined from the image of the aforementioned camera. The person attributes that are pre-associated with the characteristics determined above are identified as the person attributes of the person shown in the camera's video. If the horizontal direction of the camera image is defined as the x-direction and the vertical direction as the y-direction, and the x-distance between the two shoulders is defined as a, and the x-distance from one ear to the midpoint between the two shoulders is defined as c, then the four coordinate values (x1, y1), (x2, y2), (x3, y3), and (x4, y4) that define the head region as a rectangle are: x1 is the x-coordinate value of the aforementioned ear, y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is a coordinate value c away from the x-coordinate value of the midpoint between the two shoulders, in the x-direction toward the opposite side of the ear with respect to the midpoint between the two shoulders. y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the aforementioned ear, y3 is the y-coordinate value of the aforementioned ear, x4 is a coordinate value c away from the midpoint of the x-coordinates of the two shoulders, in the x-direction toward the opposite side of the ear, and, y4 is the y-coordinate value of the aforementioned ear. [Effects of the Invention]
[0010] According to the present invention, posture estimation processing is performed on a person captured in the camera image of a camera installed at the work site to acquire skeletal information. From this skeletal information, the head region of the person is identified, and the characteristics of the head region of the person are determined from the camera image. This makes it possible to automatically and efficiently identify the person's attributes from headwear such as helmets and hats. [Brief explanation of the drawing]
[0011] [Figure 1] Figure 1 is a schematic diagram of a work site monitoring system according to one embodiment of the present invention. [Figure 2] Figure 2 is a schematic diagram of the functional configuration of the work site monitoring device 1. [Figure 3] Figure 3 is a schematic diagram showing an example of the registered contents of the person attribute memory unit 12. [Figure 4] Figure 4 is a schematic diagram showing an example of the registered contents of the alarm condition storage unit 13. [Figure 5] Figure 5 is a flowchart illustrating the operation of the work site monitoring device 1. [Figure 6] Figure 6 is a flowchart illustrating the head region identification process S14 shown in Figure 5. [Figure 7] Figure 7 is a diagram illustrating the principle of identifying the head region using the positional coordinates of both shoulders, both ankles, both ears, and both eyes in S141 shown in Figure 6. [Figure 8] Figure 8 is a diagram illustrating the principle of identifying the head region using the positional coordinates of both shoulders, both ankles, and both ears in S143 shown in Figure 6. [Figure 9] Figure 9 is a diagram illustrating the principle of identifying the head region using the positional coordinates of both shoulders, both ankles, and one ear in S145 shown in Figure 6. [Modes for carrying out the invention]
[0012] An embodiment of the present invention will be described below.
[0013] Figure 1 is a schematic diagram of a work site monitoring system according to one embodiment of the present invention.
[0014] As shown in the figure, the work site monitoring system according to this embodiment comprises a work site monitoring device 1, a plurality of cameras 2-1, 2-2 (hereinafter also simply referred to as camera 2), a warning light 3, and a speaker 4, which are wirelessly connected to the work site monitoring device 1 via a wireless AP (access point) 5.
[0015] Camera 2-1 is installed to photograph people entering and exiting work sites 6 such as manufacturing sites and construction sites through entrances 60, and Camera 2-2 is installed to photograph people approaching designated machinery and equipment installed within work sites 6 (for example, machinery and equipment whose operation is permitted only to skilled workers). Warning lights 3 and speakers 4 are installed in conspicuous locations within work sites 6. Furthermore, people entering work sites 6 (people entering the site) are required to wear headgear such as helmets or hats 7 color-coded according to their personal attributes (visualization of the personal characteristics of people entering the site).
[0016] The work site monitoring device 1 identifies the attributes of a person entering the work site by determining the color of the headgear 7 worn by the person as captured in the image from camera 2. Based on the identified attributes and the installation location of camera 2 that captured the person entering the work site, the device determines whether a warning is necessary for the person entering the work site, and controls the illumination of the warning light 3 and the output of the warning sound from speaker 4 based on that determination.
[0017] Next, we will explain the details of the work site monitoring device 1.
[0018] Furthermore, since existing cameras, warning lights, and speakers with wireless connectivity or connected wireless adapters can be used as the camera 2, warning light 3, and speaker 4 that constitute the work site monitoring system according to this embodiment together with the work site monitoring device 1, a detailed explanation of them will be omitted.
[0019] Figure 2 is a schematic diagram of the functional configuration of the work site monitoring device 1.
[0020] As shown in the figure, the work site monitoring device 1 includes a wireless interface unit 10, a man-machine interface unit 11, a person attribute storage unit 12, an alarm condition storage unit 13, a camera image acquisition unit 14, a posture estimation processing unit 15, a head region identification unit 16, a color determination unit 17, a person attribute identification unit 18, and an alarm control unit 19.
[0021] The wireless interface unit 10 is an interface for wirelessly connecting to the camera 2, warning light 3, and speaker 4 via the wireless AP 5.
[0022] The man-machine interface unit 11 is an interface for displaying information to the operator of the work site monitoring device 1 (the person monitoring the work site) and for receiving various operations from this operator, and includes a display device such as an LCD (Liquid Crystal Display) and an input device such as a mouse, keyboard, or touch panel.
[0023] The person attribute memory unit 12 stores the correspondence between the color of the head-mounted device 7 and the person attributes of the wearer, in accordance with the operation received from the operator via the man-machine interface unit 11.
[0024] Figure 3 is a schematic diagram showing an example of the registered contents of the person attribute memory unit 12.
[0025] As shown in the figure, the person attribute storage unit 12 stores a person attribute record 120 for each color of the headgear 7. The person attribute record 120 has a field 121 in which the color of the headgear 7 is registered and a field 122 in which the person attribute is registered. In this embodiment, the person attributes of people entering the site are classified into visitors, skilled workers, apprentice workers, and supervisors, but the classification of the person attributes of people entering the site can be determined according to the actual situation and environment of the work site.
[0026] The alarm condition storage unit 13 stores alarm conditions for individuals entering the site for each camera 2, in accordance with the operation received from the operator via the human-machine interface unit 11.
[0027] Figure 4 is a schematic diagram showing an example of the registered contents of the alarm condition storage unit 13.
[0028] As shown in the figure, the alarm condition storage unit 13 stores an alarm condition record 130 for each camera 2 for persons entering the site. The alarm condition record 130 includes a field 131 in which the camera ID, which is the identification information of camera 2, is registered; a field 132 in which the person attributes of the person entering the site who will be the target of the warning using the warning light 3 and speaker 4 are registered as alarm conditions; and a field 133 in which the alarm message to be output by sound from speaker 4 is registered.
[0029] The camera image acquisition unit 14 acquires camera images from each camera 2 via the wireless interface unit 10 and displays them on the human-machine interface unit 11. The camera image acquisition unit 14 also determines whether or not there are people on site in the camera images acquired from camera 2. If there are people on site, it captures the camera image and obtains a still image. The acquired still image, along with the camera ID assigned to that camera 2, is then passed to the posture estimation processing unit 15 and the color determination unit 17.
[0030] The posture estimation processing unit 15 performs existing posture estimation processing on the person present in the still image received from the camera image acquisition unit 14 along with the camera ID, and obtains the skeletal information of the person present. One example of posture estimation processing is AI (Artificial Intelligence) processing using Google's pre-trained model "CoralPoseNet™". According to this posture estimation processing, although it differs depending on the posture of the person present in the still image (whether they are facing forward, with their back to camera 2, facing sideways, crouching, etc.), it is possible to obtain skeletal information including the position coordinates of up to 17 parts such as both eyes, both ears, both shoulders, and both ankles.
[0031] The head region identification unit 16 identifies the head region of the person present at the site based on the skeletal information acquired by the posture estimation processing unit 15.
[0032] The color determination unit 17 determines the color of the head region identified by the head region identification unit 16 in the still image received from the camera image acquisition unit 14 along with the camera ID (i.e., the color of the head device 7 worn on the head of the person on site shown in the still image). It then notifies the person attribute identification unit 18 of the determination result along with the camera ID.
[0033] The person attribute identification unit 18 refers to the person attribute storage unit 12 and identifies the person attribute associated with the color determination result received from the color determination unit 17 along with the camera ID. It then notifies the alarm control unit 19 of the identified person attribute along with the camera ID.
[0034] The alarm control unit 19 displays the camera image of camera 2, which is identified by the camera ID received from the person attribute identification unit 18, along with the person attribute received from the person attribute identification unit 18, on the camera image displayed on the man-machine interface unit 11. The alarm control unit 19 also controls the illumination of the warning light 3 and the output of the alarm sound from the speaker 4.
[0035] The functional configuration of the work site monitoring device 1 shown in Figure 2 may be implemented in hardware using integrated logic ICs such as ASICs (Application Specific Integrated Circuits) and FPGAs (Field Programmable Gate Arrays), or it may be implemented in software using a computer such as a DSP (Digital Signal Processor). Alternatively, it may be implemented as a process in a general-purpose computer such as a PC equipped with a CPU (Central Processing Unit), memory, auxiliary storage devices such as SSDs (Solid State Drives) and HDDs (Hard Disk Drives), and communication devices such as a wireless LAN (Local Area Network) adapter, by having the CPU load a predetermined program from the auxiliary storage device into memory and execute it.
[0036] Figure 5 is a diagram illustrating the operation flow of the work site monitoring device 1.
[0037] The camera image acquisition unit 14 acquires camera images sent from each camera 2 via the wireless interface unit 10 and displays them on the human-machine interface unit 11, while also monitoring whether or not there are people on site in the camera images (S10). If a person on site is detected in the camera image of any of the cameras 2 (YES in S11), the camera image is captured to obtain a still image (S12). Then, the acquired still image, along with the camera ID assigned to that camera 2, is passed to the posture estimation processing unit 15 and the color determination unit 17.
[0038] In response, the posture estimation processing unit 15 performs the existing posture estimation process on the person on site shown in the still image received from the camera image acquisition unit 14 along with the camera ID, and obtains the skeletal information of the person on site (S13).
[0039] Next, the head region identification unit 16 performs the head region identification process described later based on the skeletal information acquired by the posture estimation processing unit 15 to identify the head region of the person who entered the site as shown in the still image received by the posture estimation processing unit 15 (S14).
[0040] Next, the color determination unit 17 determines the color of the head region identified by the head region identification unit 16 in the still image received from the camera image acquisition unit 14 along with the camera ID, that is, the color of the head device 7 worn by the person on site shown in the still image. If the head region contains multiple colors, the unit determines that the color occupying the largest area is the color of the head device 7. The color determination result is then notified to the person attribute identification unit 18 along with the camera ID (S15).
[0041] In response, the person attribute identification unit 18 refers to the person attribute storage unit 12 and identifies the person attribute associated with the color determination result (color of the head-mounted device 7) received from the color determination unit 17. If the person attribute associated with the color determination result does not exist in the person attribute storage unit 12, the person attribute identification unit 18 identifies the person attribute as unknown. The person attribute identification result is then notified to the alarm control unit 19 along with the camera ID received from the color determination unit 17 along with the determination result (S16).
[0042] Next, the alarm control unit 19 displays the camera image of camera 2, which is identified by the camera ID received from the person attribute identification unit 18, along with the camera ID, on the human-machine interface unit 11 (S17).
[0043] Furthermore, the alarm control unit 19 determines whether an alarm output is necessary based on the camera ID and the results of the identification of person attributes received from the person attribute identification unit 18 (S18). Specifically, it determines that an alarm is necessary if (1) the results of the identification of person attributes received from the person attribute identification unit 18 indicate that the person attribute is unknown, and (2) the results of the identification of person attributes received from the person attribute identification unit 18 are included in the alarm conditions stored in the alarm condition storage unit 13, which are linked to the camera ID received from the person attribute identification unit 18, along with this camera ID. In all other cases, it determines that an alarm is not necessary.
[0044] Then, if the alarm control unit 19 determines that an alarm is necessary (YES in S18), it controls the illumination of the warning light 3 and the output of an alarm sound from the speaker 4 to issue an alarm (S19). Specifically, if the result of identifying the person's attributes is that the person's attributes are unknown (the alarm in the case of (1) above), the warning light 3 is illuminated and an alarm message indicating that a person with unknown attributes has entered the work site 6 is output from the speaker 4. Also, if the result of identifying the person's attributes is that the alarm is included in the alarm conditions stored in the alarm condition storage unit 13 linked to the corresponding camera ID (the alarm in the case of (2) above), the warning light 3 is illuminated and the warning message stored in the alarm condition storage unit 13 linked to this camera ID is output from the speaker 4. Then, the process returns to S11.
[0045] On the other hand, if the alarm control unit 19 determines that an alarm is not necessary (NO in S18), it immediately returns to S11.
[0046] Figure 6 is a diagram illustrating the operation flow of the head region identification process S14 shown in Figure 5.
[0047] First, the head region identification unit 16 determines whether the skeletal information acquired by the posture estimation processing unit 15 includes the position coordinates of both shoulders, both ankles, both ears, and both eyes (S140). If all of these position coordinates are included (YES in S140), the head region is identified using these position coordinates (S141). Then, the process proceeds to S15 in Figure 5.
[0048] Figure 7 is a diagram illustrating the principle of identifying the head region using the positional coordinates of both shoulders, both ankles, both ears, and both eyes in S141 shown in Figure 6.
[0049] This diagram shows the camera view when a person is facing camera 2 directly, with Rs indicating the right shoulder, Ls the left shoulder, Ra the right ankle, La the left ankle, Rea the right ear, Lea the left ear, Rey the right eye, and Ley the left eye.
[0050] Furthermore, this camera image is defined in an XY coordinate system where the upper left corner is the origin O, the direction to the right from the origin O is the positive X-axis, and the direction downward from the origin O is the positive Y-axis.
[0051] According to the Vitruvian Man, shoulder width is equal to 1 / 4 of height. Therefore, the coordinates of each vertex of the rectangular head region 70 are determined as follows.
[0052] The difference between the x-coordinates of the right shoulder Rs and the left shoulder Ls (the x-axis distance between both shoulders Rs and Ls), a, and the y-coordinate of the midpoint of the line segment with the left and right ankles La and Ra as its endpoints (the average value ya of the y-coordinates of the right ankle Ra and the left ankle La) are determined. Then, the y-coordinate of the position obtained by translating the midpoint of the line segment with the left and right ankles La and Ra by a distance of 4a, which is equivalent to four times the difference a between the x-coordinates a of the left and right shoulders Ls and Rs, in the negative Y-axis direction (direction from the ankle toward the ear) is determined as the y-coordinate of the upper end of the head region 70 (the end on the origin O side). Furthermore, the difference b1 between the y-coordinate of the right ear Rea and the y-coordinate of the right eye Rey (the y-direction distance from the right eye Rea to the right ear Rey) is calculated, and the y-coordinate of the position obtained by translating the distance equivalent to this difference b1 in the negative Y-axis direction (the direction opposite to the ankle relative to the right eye Rey) is determined as the y-coordinate of the lower right vertex of the head region 70. Similarly, the difference b2 between the y-coordinate of the left ear Lea and the y-coordinate of the left eye Ley (the y-direction distance from the left ear Lea to the left eye Ley) is calculated, and the y-coordinate of the position obtained by translating the distance equivalent to this difference b2 in the negative Y-axis direction (the direction opposite to the ankle relative to the left eye Ley) is determined as the y-coordinate of the lower left vertex of the head region 70. In addition, the x-coordinate of the right eye Rey is determined as the x-coordinate of the right edge of the head region 70, and the x-coordinate of the left eye Ley is determined as the x-coordinate of the left edge of the head region 70.
[0053] If the coordinates of the right eye Rey are (xrey, yrey) and the coordinates of the left eye Rey are (xley, yley), then in the XY coordinate system of the camera image shown in Figure 7, the coordinates (x1, y1), (x2, y2), (x3, y3), and (x4, y4) of the upper right, upper left, lower right, and lower left vertices of the head region 70 can be calculated using the following equation 1.
[0054]
number
[0055] On the other hand, the head region identification unit 16 determines whether the position coordinates of both shoulders, both ankles, both ears, and both eyes are all included in the skeletal information acquired by the posture estimation processing unit 15 (NO in S140) (S142). If all of these position coordinates are included (YES in S142), the head region is identified using these position coordinates (S143). Then, the process proceeds to S15 in Figure 5.
[0056] Figure 8 is a diagram illustrating the principle of identifying the head region using the positional coordinates of both shoulders, both ankles, and both ears in S143 shown in Figure 6.
[0057] This figure shows the camera image when the person has their back to camera 2. Similar to Figure 7, Rs represents the right shoulder, Ls the left shoulder, Ra the right ankle, La the left ankle, Rea the right ear, and Lea the left ear. Furthermore, an XY coordinate system is defined for this camera image, with the upper left corner as the origin O, the direction to the right of the origin O being the positive X-axis, and the direction downwards from the origin O being the positive Y-axis.
[0058] As mentioned above, according to the Vitruvian Man, shoulder width is equal to 1 / 4 of height. Therefore, the coordinates of each vertex of the rectangular head region 70 are determined as follows.
[0059] The difference between the x-coordinates of the right shoulder Rs and the left shoulder Ls (the x-axis distance between both shoulders Rs and Ls), a, and the y-coordinate of the midpoint of the line segment with the left and right ankles La and Ra as its endpoints (the average value ya of the y-coordinates of the right ankle Ra and the left ankle La) are determined. From the midpoint of the line segment with the left and right ankles La and Ra as its endpoints, the y-coordinate of the position obtained by translating 4a, which is four times the difference a between the x-coordinates a of the left and right shoulders Ls and Rs, in the negative Y-axis direction (direction from the ankle towards the shoulder) is determined to be the y-coordinate of the upper end of the head region 70 (the end on the origin O side). In addition, the y-coordinate of the right ear Rea is determined to be the y-coordinate of the lower right vertex of the head region 70, and the y-coordinate of the left ear Lea is determined to be the y-coordinate of the lower left vertex of the head region 70. Furthermore, the x-coordinate value of the right ear Rea is determined to be the x-coordinate value of the right edge of the head region 70, and the x-coordinate value of the left ear Lea is determined to be the x-coordinate value of the left edge of the head region 70.
[0060] If the coordinates of the right ear Rea are (xrea, yrea) and the coordinates of the left ear Lea are (xlea, ylea), then in the XY coordinate system of the camera image shown in Figure 8, the coordinates (x1, y1), (x2, y2), (x3, y3), and (x4, y4) of the upper right, upper left, lower right, and lower left vertices of the head region can be calculated using the following equation 2.
[0061]
number
[0062] On the other hand, if the skeletal information acquired by the posture estimation processing unit 15 does not include the position coordinates of either of the shoulders, ankles, or ears (NO in S142), the head region identification unit 16 determines whether the position coordinates of both shoulders, both ankles, and one ear are all included in the skeletal information (S144). If all of these position coordinates are included (YES in S144), the head region is identified using these position coordinates (S145). Then, the process proceeds to S15 in Figure 5.
[0063] Figure 9 is a diagram illustrating the principle of identifying the head region using the positional coordinates of both shoulders, both ankles, and one ear in S145 shown in Figure 6.
[0064] This figure shows the camera image when a person is facing right with their back to camera 2. Similar to Figure 7, Rs represents the right shoulder, Ls the left shoulder, Ra the right ankle, La the left ankle, and Rea the position of the right ear. This camera image also has an XY coordinate system defined, with the upper left corner as the origin O, the direction to the right from the origin O being the positive X-axis, and the direction downward from the origin O being the positive Y-axis.
[0065] As mentioned above, according to the Vitruvian Man, shoulder width is equal to 1 / 4 of height. Therefore, the coordinates of each vertex of the rectangular head region 70 are determined as follows.
[0066] The difference between the x-coordinates of the right shoulder Rs and the left shoulder Ls (the x-axis distance between both shoulders Rs and Ls), a, and the y-coordinate of the midpoint of the line segment with the left and right ankles La and Ra as its endpoints (the average value ya of the y-coordinates of the right ankle Ra and the left ankle La) are determined. Then, the y-coordinate of the position obtained by translating the midpoint of the line segment with the left and right ankles La and Ra by a distance of 4a, which is equivalent to four times the difference a between the x-coordinates a of the left and right shoulders Ls and Rs, in the negative Y-axis direction (direction from the ankle towards the shoulder) is determined as the y-coordinate of the upper end of the head region 70 (the end on the origin O side). Furthermore, the x-coordinate of the position obtained by translating from the right shoulder Rs by a distance a / 2, which is half the difference a between the x-coordinates a of the left shoulders Ls and Rs, toward the left shoulder Ls in the negative X-axis direction (the midpoint of the line segment with the left shoulders Ls and Rs as endpoints), is defined as the x-coordinate of the position obtained by translating from the left shoulder Ls by a distance a / 2 toward the right shoulder Rs in the positive X-axis direction (the x-coordinate of the position obtained by translating from the left shoulder Ls by a distance a / 2 toward the right shoulder Rs in the positive X-axis direction), and the difference c between the x-coordinate of the right ear Rea and this x-coordinate of the head central axis is calculated. Then, the x-coordinate value of the right ear Rea is determined as the x-coordinate value of the right edge of the head region 70, and the x-coordinate value of the position obtained by translating the x-coordinate value of the right ear Rea to the left (negative X-axis direction) by a distance 2c, which is equivalent to twice the difference between the x-coordinate values of the right ear Rea and the head central axis, is determined as the x-coordinate value of the position obtained by translating the head central axis to the left by a distance c, which is equivalent to the difference between the x-coordinate values of the head central axis and the right ear Rea, and is determined as the x-coordinate value of the left edge of the head region 70.
[0067] If the coordinates of the right ear Rey are (xrea, yrea), then in the XY coordinate system of the camera image shown in Figure 9, the coordinates (x1, y1), (x2, y2), (x3, y3), and (x4, y4) of the upper right, upper left, lower right, and lower left vertices of the head region 70 can be calculated using the following equation 3.
[0068]
number
[0069] Figure 9 illustrates the camera image when the person is facing right with their back to camera 2 (where the position coordinates of the right ear Rea can be obtained, but the position coordinates of the left ear Rea are missing). For the camera image when the person is facing left with their back to camera 2 (where the position coordinates of the left ear Rea can be obtained, but the position coordinates of the right ear Rea are missing), the difference c between the x-coordinate of the head's central axis and the x-coordinate of the left ear Rea is calculated. The x-coordinate of the left ear Rea is then determined as the x-coordinate of the left edge of the head region 70, and the x-coordinate of the position obtained by translating the x-coordinate of the left ear Rea to the right (positive X-axis direction) by a distance of 2c, which is equivalent to twice the difference c between the head's central axis and the x-coordinate of the left ear Rea, is determined as the x-coordinate of the right edge of the head region 70.
[0070] If the coordinates of the left ear Ley are (xlea, ylea), then in the XY coordinate system of the camera image shown in Figure 9, the coordinates (x1, y1), (x2, y2), (x3, y3), and (x4, y4) of the upper right, upper left, lower right, and lower left vertices of the head region 70 can be calculated using the following equation 4.
[0071]
number
[0072] On the other hand, if the skeletal information acquired by the posture estimation processing unit 15 does not include the position coordinates of either the shoulders, both ankles, or one ear (NO in S144), the head region identification unit 16 determines that the head region cannot be identified and returns to S11.
[0073] One embodiment of the present invention has been described above.
[0074] In this embodiment, the work site monitoring device 1 performs posture estimation processing on a person entering the work site as captured by the camera 2 installed at the work site 6, and acquires the skeletal information of the person entering the work site. Based on this skeletal information, it identifies the head region of the person entering the work site, and by determining the color of the identified head region from the camera image, it identifies the color of the headgear 7 worn by the person entering the work site as captured by the camera image. Based on this color, it identifies the person's attributes associated with the color of the headgear 7. Therefore, according to this embodiment, even when it is difficult to identify the face information of a person entering the work site as captured by the camera image, such as when the person has their back to the camera 2 or is wearing a dust mask or work goggles, the person's attributes can be efficiently and automatically identified.
[0075] Furthermore, in this embodiment, the work site monitoring device 1 can identify the head region of a person entering the work site if the skeletal information acquired by the posture estimation process includes the position coordinates of both shoulders, both ankles, and one ear. This increases the likelihood of identifying the head region of a person entering the work site from camera footage. Additionally, if the skeletal information includes the position coordinates of both shoulders, both ankles, and both ears, the head region of a person entering the work site can be identified with higher accuracy by using these position coordinates compared to identifying the head region using only the position coordinates of both shoulders, both ankles, and one ear. Moreover, if the skeletal information includes the position coordinates of both shoulders, both ankles, both ears, and both eyes, the head region of a person entering the work site can be identified with even higher accuracy by using these position coordinates compared to identifying the head region using only the position coordinates of both shoulders, both ankles, and both ears.
[0076] Furthermore, in this embodiment, the work site monitoring device 1 displays the person attributes associated with the color of the head area of the person entering the work site overlaid on the camera image showing that person. Therefore, the operator can easily determine the person attributes of the person entering the work site as shown in the camera image at a glance. This reduces the workload of the operator (monitor) in safety management of the work site 6.
[0077] Furthermore, in this embodiment, the work site monitoring device 1 outputs an alarm using the warning light 3 and speaker 4 installed at the work site 6 when the characteristics of a person entering the work site as captured in the camera image are included in the alarm conditions associated with that camera 2. This further reduces the workload of the supervisor in safety management at the work site 6.
[0078] It should be noted that the present invention is not limited to the embodiments described above, and numerous modifications are possible within the scope of its essence.
[0079] For example, in the above embodiment, the camera 2, warning light 3, and speaker 4 are wirelessly connected to the work site monitoring device 1 via a wireless AP (access point) 5. However, the present invention is not limited to this. The camera 2, warning light 3, and speaker 4 may also be wired to the work site monitoring device 1 via a wired network device such as a wired hub.
[0080] Furthermore, in the above embodiment, the color of the headwear 7 worn by the person shown in the camera image is determined by determining the color of the head region identified from the camera image, and the person's attributes associated with the color of the headwear 7 are identified. However, the present invention is not limited to this. The characteristics of the helmet, hat, etc. worn on the head of the person shown in the camera image may be determined by determining the characteristics of the headwear 7 worn by the person shown in the camera image, and the person's attributes associated with the characteristics of the headwear 7 worn by the person shown in the camera image may be identified.
[0081] Furthermore, the present invention can be broadly applied to a person attribute identification device that identifies the person attributes of a person captured in camera footage from a camera installed at a work site. [Explanation of Symbols]
[0082] 1: Work site monitoring device 2-1, 2-2: Camera 3: Warning light 4: Speaker 5: Wireless AP 6: Work site 7: Head-mounted device 10: Wireless interface unit 11: Human-machine interface unit 12: Person attribute memory unit 13: Alarm condition memory unit 14: Camera image acquisition unit 15: Pose estimation processing unit 16: Head region identification unit 17: Color determination unit 18: Person Attribute Identification Unit 19: Alarm Control Unit 60: Entrance / exit 61: Mechanical equipment
Claims
1. A method for identifying the personal attributes of a person displayed in the image of a camera, using a camera and a person attribute identification device connected to the camera, The aforementioned person attribute identification device, A posture estimation process is performed on the person captured in the image of the aforementioned camera to obtain the person's skeletal information. Based on the position coordinates of both ankles, both shoulders, both eyes, and both ears included in the skeletal information of the person obtained by the posture estimation process, the head region of the person displayed in the camera's image is identified. The characteristics of the identified head region are determined from the image of the aforementioned camera. The person attributes that are pre-associated with the characteristics determined above are identified as the person attributes of the person shown in the camera's video. If the horizontal direction of the camera image is defined as the x-direction and the vertical direction as the y-direction, and the x-distance between the two shoulders is a, the y-distance from the right eye to the right ear is b1, and the y-distance from the left eye to the left ear is b2, then the four coordinate values (x1, y1), (x2, y2), (x3, y3), and (x4, y4) that define the head region as a rectangle are: x1 is the x-coordinate value of the right eye, y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is the x-coordinate value of the left eye, y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the right eye, y3 is a coordinate value that is b1 away from the y coordinate value of the right eye, in the y direction away from both ankles relative to the right eye. x4 is the x-coordinate value of the left eye, and, y4 is the y-coordinate value at a position b2 away from the y-coordinate value of the left eye, in the y-direction away from both ankles relative to the left eye. A method for identifying the attributes of a person, characterized by the following features.
2. A method for identifying the personal attributes of a person displayed in the image of the camera, using a camera and a person attribute identification device connected to the camera, The aforementioned person attribute identification device, A posture estimation process is performed on the person captured in the image of the aforementioned camera to obtain the person's skeletal information. Based on the position coordinates of both ankles, both shoulders, and both ears included in the skeletal information of the person obtained by the posture estimation process, the head region of the person displayed in the camera's image is identified. The characteristics of the identified head region are determined from the image of the aforementioned camera. The person attributes that are pre-associated with the characteristics determined above are identified as the person attributes of the person shown in the camera's video. If the horizontal direction of the image from the camera is defined as the x-direction and the vertical direction as the y-direction, and the x-direction distance between the two shoulders is defined as a, then the four coordinate values that define the head region as a rectangle are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), x1 is the x-coordinate value of the right ear. y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is the x-coordinate value of the left ear. y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the right ear, y3 is the y-coordinate value of the right ear. x4 is the x-coordinate value of the left ear, and, y4 is the y-coordinate value of the left ear. A method for identifying the attributes of a person, characterized by the following features.
3. A method for identifying the personal attributes of a person displayed in the image of a camera, using a camera and a person attribute identification device connected to the camera, The aforementioned person attribute identification device, A posture estimation process is performed on the person captured in the image of the aforementioned camera to obtain the person's skeletal information. Based on the position coordinates of both ankles, both shoulders, and one ear included in the skeletal information of the person obtained by the posture estimation process, the head region of the person displayed in the camera's image is identified. The characteristics of the identified head region are determined from the image of the aforementioned camera. The person attributes that are pre-associated with the characteristics determined above are identified as the person attributes of the person shown in the camera's video. If the horizontal direction of the camera image is defined as the x-direction and the vertical direction as the y-direction, and the x-direction distance between the two shoulders is denoted as a, and the x-direction distance from one ear to the midpoint between the two shoulders is denoted as c, then the four coordinate values that define the head region as a rectangle are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), x1 is the x-coordinate value of the aforementioned ear, y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is a coordinate value c away from the x-coordinate value of the midpoint between the two shoulders, in the x-direction toward the opposite side of one ear with respect to the midpoint between the two shoulders. y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the aforementioned ear, y3 is the y-coordinate value of the aforementioned ear, x4 is a coordinate value c away from the midpoint of the x-coordinates of the two shoulders, in the x-direction toward the opposite side of one ear, and, y4 is the y-coordinate value of the aforementioned ear. A method for identifying the attributes of a person, characterized by the following features.
4. A method for identifying a person's attributes according to any one of claims 1 to 3, The aforementioned person attribute identification device, The identified person attributes are overlaid on the camera's video feed and displayed on the display device. A method for identifying the attributes of a person, characterized by the following features.
5. A method for identifying a person's attributes according to any one of claims 1 to 3, The aforementioned person attribute identification device, If the identified person attributes match the predetermined person attributes, the alarm device will output an alarm to the person shown in the camera's video. A method for identifying the attributes of a person, characterized by the following features.
6. A method for identifying a person's attributes according to claim 4, The aforementioned person attribute identification device, If the identified person attributes match the predetermined person attributes, the alarm device will output an alarm to the person shown in the camera's video. A method for identifying the attributes of a person, characterized by the following features.
7. A person attribute identification device that identifies the person attributes of a person shown in a camera image, A video acquisition means for acquiring images from the aforementioned camera, A posture estimation processing implementation means performs posture estimation processing on a person shown in the camera image acquired by the aforementioned image acquisition means to acquire skeletal information of the person, A head region identification means identifies the head region of a person displayed in the camera image based on the position coordinates of both ankles, both shoulders, both eyes, and both ears included in the skeletal information acquired by the posture estimation processing means, A feature determination means for determining the features of the head region of a person identified by the head region identification means from the image of the camera, The system includes a person attribute identification means that identifies person attributes pre-associated with the features determined by the feature determination means as person attributes of the person shown in the camera's video, If the horizontal direction of the camera image is defined as the x-direction and the vertical direction as the y-direction, and the x-distance between the two shoulders is a, the y-distance from the right eye to the right ear is b1, and the y-distance from the left eye to the left ear is b2, then the four coordinate values (x1, y1), (x2, y2), (x3, y3), and (x4, y4) that define the head region as a rectangle are: x1 is the x-coordinate value of the right eye, y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is the x-coordinate value of the left eye, y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the right eye, y3 is a coordinate value that is b1 away from the y coordinate value of the right eye, in the y direction away from both ankles relative to the right eye. x4 is the x-coordinate value of the left eye, and, y4 is the y-coordinate value at a position b2 away from the y-coordinate value of the left eye, in the y-direction away from both ankles relative to the left eye. A person attribute identification device characterized by the following features.
8. A person attribute identification device for identifying the person attributes of a person displayed in a camera image, A video acquisition means for acquiring images from the aforementioned camera, A posture estimation processing implementation means performs posture estimation processing on a person shown in the camera image acquired by the aforementioned image acquisition means to acquire skeletal information of the person, A head region identification means identifies the head region of a person displayed in the camera's image based on the position coordinates of both ankles, both shoulders, and both ears included in the skeletal information acquired by the posture estimation processing means, A feature determination means for determining the features of the head region of a person identified by the head region identification means from the image of the camera, The system includes a person attribute identification means that identifies person attributes pre-associated with the features determined by the feature determination means as person attributes of the person shown in the camera's video, If the horizontal direction of the image from the camera is defined as the x-direction and the vertical direction as the y-direction, and the x-direction distance between the two shoulders is defined as a, then the four coordinate values that define the head region as a rectangle are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), x1 is the x-coordinate value of the right ear. y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is the x-coordinate value of the left ear. y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the right ear, y3 is the y-coordinate value of the right ear. x4 is the x-coordinate value of the left ear, and, y4 is the y-coordinate value of the left ear. A person attribute identification device characterized by the following features.
9. A person attribute identification device for identifying the person attributes of a person displayed in a camera image, A video acquisition means for acquiring images from the aforementioned camera, A posture estimation processing implementation means performs posture estimation processing on a person shown in the camera image acquired by the aforementioned image acquisition means to acquire skeletal information of the person, A head region identification means identifies the head region of a person displayed in the camera image based on the position coordinates of both ankles, both shoulders, and one ear included in the skeletal information acquired by the posture estimation processing means, A feature determination means for determining the features of the head region of a person identified by the head region identification means from the image of the camera, The system includes a person attribute identification means that identifies person attributes pre-associated with the features determined by the feature determination means as person attributes of the person shown in the camera's video, If the horizontal direction of the camera image is defined as the x-direction and the vertical direction as the y-direction, and the x-direction distance between the two shoulders is denoted as a, and the x-direction distance from one ear to the midpoint between the two shoulders is denoted as c, then the four coordinate values that define the head region as a rectangle are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), x1 is the x-coordinate value of the aforementioned ear, y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is a coordinate value c away from the x-coordinate value of the midpoint between the two shoulders, in the x-direction toward the opposite side of one ear with respect to the midpoint between the two shoulders. y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the aforementioned ear, y3 is the y-coordinate value of the aforementioned ear, x4 is a coordinate value c away from the midpoint of the x-coordinates of the two shoulders, in the x-direction toward the opposite side of one ear, and, y4 is the y-coordinate value of the aforementioned ear. A person attribute identification device characterized by the following features.
10. A program that causes a computer to function as a person attribute identification device that identifies the person attributes of a person shown in a camera image, Video acquisition means for acquiring images from the aforementioned camera, A posture estimation processing implementation means performs posture estimation processing on a person shown in the camera image acquired by the aforementioned image acquisition means to acquire skeletal information of the person. Based on the position coordinates of both ankles, both shoulders, both eyes, and both ears included in the skeletal information acquired by the posture estimation processing means, a head region identification means identifies the head region of a person displayed in the camera's image. A feature determination means for determining the features of the head region of a person identified by the head region identification means from the image of the camera, and The computer is used as a person attribute identification means to identify the person attributes that are pre-associated with the features determined by the feature determination means as the person attributes of the person shown in the camera's video. If the horizontal direction of the camera image is defined as the x-direction and the vertical direction as the y-direction, and the x-distance between the two shoulders is a, the y-distance from the right eye to the right ear is b1, and the y-distance from the left eye to the left ear is b2, then the four coordinate values (x1, y1), (x2, y2), (x3, y3), and (x4, y4) that define the head region as a rectangle are: x1 is the x-coordinate value of the right eye, y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is the x-coordinate value of the left eye, y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the right eye, y3 is a coordinate value that is b1 away from the y coordinate value of the right eye, in the y direction away from both ankles relative to the right eye. x4 is the x-coordinate value of the left eye, and, y4 is the y-coordinate value at a position b2 away from the y-coordinate value of the left eye, in the y-direction away from both ankles relative to the left eye. A program characterized by the following features.
11. A program that causes a computer to function as a person attribute identification device for identifying the person attributes of a person shown in a camera image, Video acquisition means for acquiring images from the aforementioned camera, A posture estimation processing implementation means performs posture estimation processing on a person shown in the camera image acquired by the aforementioned image acquisition means to acquire skeletal information of the person. Based on the position coordinates of both ankles, both shoulders, and both ears included in the skeletal information acquired by the posture estimation processing means, a head region identification means identifies the head region of a person displayed in the camera's image. A feature determination means for determining the features of the head region of a person identified by the head region identification means from the image of the camera, and The computer is used as a person attribute identification means to identify the person attributes that are pre-associated with the features determined by the feature determination means as the person attributes of the person shown in the camera's video. If the horizontal direction of the image from the camera is defined as the x-direction and the vertical direction as the y-direction, and the x-direction distance between the two shoulders is defined as a, then the four coordinate values that define the head region as a rectangle are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), x1 is the x-coordinate value of the right ear. y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is the x-coordinate value of the left ear. y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the right ear, y3 is the y-coordinate value of the right ear. x4 is the x-coordinate value of the left ear, and, y4 is the y-coordinate value of the left ear. A program characterized by the following features.
12. A program that causes a computer to function as a person attribute identification device for identifying the person attributes of a person shown in a camera image, Video acquisition means for acquiring images from the aforementioned camera, A posture estimation processing implementation means performs posture estimation processing on a person shown in the camera image acquired by the aforementioned image acquisition means to acquire skeletal information of the person. Head region identification means that identifies the head region of a person displayed in the camera image based on the position coordinates of both ankles, both shoulders, and one ear included in the skeletal information acquired by the posture estimation processing means. A feature determination means for determining the features of the head region of a person identified by the head region identification means from the image of the camera, and The computer is used as a person attribute identification means to identify the person attributes that are pre-associated with the features determined by the feature determination means as the person attributes of the person shown in the camera's video. If the horizontal direction of the camera image is defined as the x-direction and the vertical direction as the y-direction, and the x-direction distance between the two shoulders is denoted as a, and the x-direction distance from one ear to the midpoint between the two shoulders is denoted as c, then the four coordinate values that define the head region as a rectangle are (x1, y1), (x2, y2), (x3, y3), and (x4, y4), x1 is the x-coordinate value of the aforementioned ear, y1 is a coordinate value that is 4a away from the average y coordinate value of both ankles, in the y direction from both ankles toward both shoulders. x2 is a coordinate value c away from the x-coordinate value of the midpoint between the two shoulders, in the x-direction toward the opposite side of one ear with respect to the midpoint between the two shoulders. y2 is a coordinate value that is 4a away from the average y coordinate value of both ankles in the y direction from both ankles toward both shoulders. x3 is the x-coordinate value of the aforementioned ear, y3 is the y-coordinate value of the aforementioned ear, x4 is a coordinate value c away from the midpoint of the x-coordinates of the two shoulders, in the x-direction toward the opposite side of one ear, and, y4 is the y-coordinate value of the aforementioned ear. A program characterized by the following features.