Image recognition system, image recognition method and program

Abstract

An image recognition system, an image recognition method, and a program are provided that can determine with high accuracy whether a target person is using a handrail. [Solution] The image recognition system 100 includes an image acquisition unit (control unit 21) that acquires an image (captured image 50) of a target person 60 captured by an imaging device 10, a skeleton detection unit (control unit 21) that detects the skeletal position of the target person 60 based on the image acquired by the image acquisition unit, an entry determination unit (control unit 21) that determines whether the target person 60 is present in a specified area of ​​the image based on the skeletal position detected by the skeleton detection unit, and a handrail use determination unit (control unit 21) that determines whether the target person 60 is using a handrail based on the determination result by the entry determination unit.

Description

[Technical Field]

[0001] The present invention relates to an image recognition system, an image recognition method, and a program. [Background technology]

[0002] BACKGROUND ART Conventionally, there is known a technique for detecting whether a passenger is holding onto a moving handrail of a passenger conveyor such as an escalator or a moving walkway, and for issuing a warning to passengers who are not holding onto the moving handrail. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Laid-Open No. 2009-196733 Summary of the Invention [Problem to be solved by the invention]

[0004] However, with conventional technology, it was only possible to detect whether a passenger was holding onto the handrail at the detection point of the detection sensor, and it was not possible to determine whether an object detected by the detection sensor near the grip of the handrail was a passenger's hand. As a result, it was possible that it was not possible to accurately determine whether a passenger was holding onto the handrail over the entire area of ​​the handrail.

[0005] Therefore, in order to solve the above problem, an object of the present invention is to provide an image recognition system, an image recognition method, and a program that can determine with high accuracy whether a target person is using a handrail. [Means for solving the problem]

[0006] In order to solve the above problem, the image recognition system according to claim 1 comprises: an image acquisition unit that acquires an image of a target person captured by an imaging device; a skeleton detection unit that detects a skeleton position of the target person based on the image acquired by the image acquisition unit; an intrusion determination unit that determines whether or not the target person is present in a predetermined area of ​​the image based on the skeleton position detected by the skeleton detection unit; a handrail use determination unit that determines whether or not the target person is using a handrail based on the determination result by the entry determination unit; Equipped with.

[0007] The invention described in claim 2 is the image recognition system described in claim 1, The entry determination unit a human intrusion determination unit that determines whether or not the target person is present in the first area of ​​the image; and a hand entry determination unit that determines whether or not the target person has a hand in the second area of ​​the image.

[0008] The invention described in claim 3 is the image recognition system described in claim 2, The entry determination unit displays the first area and the second area on the display unit in different display modes so as to be superimposed on the image.

[0009] The invention described in claim 4 is the image recognition system described in claim 2, The entry determination unit displays the position of the hand in the image on a display unit by displaying the skeleton position or a bounding box.

[0010] The invention described in claim 5 is the image recognition system described in claim 2, The handrail use determination unit determines whether the handrail is being used when the person entry determination unit determines that the target person is present in the first area and the hand entry determination unit determines that the hand is present in the second area.

[0011] The invention described in claim 6 is the image recognition system described in claim 1, A warning control unit is provided that issues a warning depending on the determination result by the handrail use determination unit.

[0012] The invention described in claim 7 is the image recognition system described in claim 6, The warning control unit notifies the warning to a terminal device used by a manager of a facility where the handrail is installed.

[0013] The invention described in claim 8 is the image recognition system described in claim 1, A display control unit is provided that displays on a display unit according to the determination result by the handrail use determination unit.

[0014] The invention described in claim 9 is the image recognition system described in claim 1, The device further includes a counting unit that counts the number of people who used the handrail and the number of people who did not use the handrail according to the result of the determination by the handrail use determination unit.

[0015] The invention described in claim 10 is the image recognition system described in claim 9, The counting unit performs personal authentication of the target person, and counts the number of times the handrail is used and / or the number of times the handrail is not used for each authenticated person.

[0016] The invention described in claim 11 is the image recognition system described in claim 9, The counting unit assigns an identification number to each of the plurality of target persons by object detection, and simultaneously counts the number of people who used the handrail and the number of people who did not use the handrail.

[0017] The invention described in claim 12 is the image recognition system described in claim 1, The handrail use determination unit determines that the handrail is being used when the state in which the handrail is being used continues for a predetermined time.

[0018] The invention described in claim 13 is the image recognition system described in claim 12, The handrail use determination unit receives settings of parameters related to determining whether or not a handrail is being used, including the predetermined time period.

[0019] The invention described in claim 14 is the image recognition system described in claim 1, A recording control unit is provided to store the image in association with the determination result by the handrail use determination unit.

[0020] The invention described in claim 15 is the image recognition system described in claim 1, The entry determination unit receives a setting of the range of the predetermined area.

[0021] The invention described in claim 16 is the image recognition system described in claim 1, The image capturing device includes an attribute determining unit that determines at least one of the age, sex, and clothing of the target person.

[0022] The invention described in claim 17 is the image recognition system described in claim 1, The entry determination unit determines the direction of travel of the target person in the specified area based on at least one of the body orientation of the target person based on the skeletal position, time series data of the detection time of the target person in the specified area, and tracking of the target person.

[0023] The invention described in claim 18 is the image recognition system described in claim 1, The apparatus further includes a warning control unit that changes the content of the warning depending on the determination result by the handrail use determination unit and the moving speed of the target person in the specified area.

[0024] The invention described in claim 19 is the image recognition system described in claim 18, The warning control unit issues a warning when the handrail use determination unit determines that the handrail is being used and the moving speed is faster than a predetermined threshold.

[0025] The invention described in claim 20 is the image recognition system described in claim 18, The warning control unit changes the display content of the display unit according to the determination result by the handrail use determination unit and the moving speed of the target person in the predetermined area.

[0026] The image recognition method according to claim 21 comprises: an image acquisition step of acquiring an image of a target person captured by an imaging device; a skeleton detection step of detecting a skeleton position of the target person based on the image acquired in the image acquisition step; an intrusion determination step of determining whether or not the target person is present in a predetermined area of ​​the image based on the skeleton position detected in the skeleton detection step; a handrail use determination unit step of determining whether or not the target person is using a handrail based on the determination result in the entry determination step; Includes.

[0027] The program according to claim 22 comprises: The computer for the image recognition system an image acquisition unit that acquires an image of a target person captured by an imaging device; a skeleton detection unit that detects a skeleton position of the target person based on the image acquired by the image acquisition unit; an intrusion determination unit that determines whether or not the target person is present in a predetermined area of ​​the image based on the skeleton position detected by the skeleton detection unit; a handrail use determination unit that determines whether or not the target person is using a handrail based on the determination result by the entry determination unit; Function as.

[0028] The image recognition system according to claim 23 comprises: an image acquisition unit that acquires an image of a target person captured by an imaging device; a skeleton detection unit that detects a skeleton position of the target person based on the image acquired by the image acquisition unit; an image identification unit that identifies a handrail portion based on the image acquired by the image acquisition unit; a handrail use determination unit that determines whether or not the target person is using a handrail based on the position of the handrail unit identified by the image identification unit and the skeleton position detected by the skeleton detection unit; and Equipped with.

[0029] The invention described in claim 24 is the image recognition system described in claim 23, The handrail usage determination unit determines whether the target person is using the handrail based on the positional relationship between the position of at least one of the back of the hand, palm, or arm of the target person as the skeletal position and the position of the handrail unit.

[0030] The invention described in claim 25 is the image recognition system described in claim 23, The handrail usage determination unit determines that the target person is not using the handrail based on the position of the handrail section and the skeletal position, and determines that an abnormality has occurred if it determines that the distance between the position of a specified part of the target person and the position of the handrail section is less than a specified distance.

[0031] The invention described in claim 26 is the image recognition system described in claim 23, The handrail usage determination unit determines that the target person is not using the handrail based on the position of the handrail section and the skeletal position, and if it determines that the distance between the position of a specified part of the target person and the position of the handrail section is greater than a specified distance, it does not determine that there is an abnormality.

[0032] The image recognition method according to claim 27 comprises: an image acquisition step of acquiring an image of a target person captured by an imaging device; a skeleton detection step of detecting a skeleton position of the target person based on the image acquired in the image acquisition step; an image identification step of identifying a handrail portion based on the image acquired in the image acquisition step; a handrail use determination step of determining whether or not the target person is using a handrail based on the position of the handrail portion identified in the image identification step and the skeleton position detected in the skeleton detection step; Includes.

[0033] The program according to claim 28 comprises: The computer for the image recognition system an image acquisition unit that acquires an image of a target person captured by an imaging device; a skeleton detection unit that detects a skeleton position of the target person based on the image acquired by the image acquisition unit; an image identification unit that identifies a handrail portion based on the image acquired by the image acquisition unit; a handrail use determination unit that determines whether or not the target person is using a handrail based on the position of the handrail unit identified by the image identification unit and the skeleton position detected by the skeleton detection unit; Function as. [Effects of the Invention]

[0034] According to the present invention, it is possible to determine with high accuracy whether a target person is using a handrail. [Brief explanation of the drawings]

[0035] [Figure 1] 1 is a block diagram showing the functional configuration of an image recognition system according to a first embodiment. [Figure 2] 1A to 1C are diagrams showing examples of images captured by the imaging device according to the first embodiment. [Figure 3] 5 is a flowchart showing a control procedure of a determination process according to the first embodiment. [Figure 4] 10 is a flowchart showing a control procedure of a determination process according to the second embodiment. [Figure 5] 11 is a flowchart showing a control procedure of a determination process according to the third embodiment. [Figure 6] 10A and 10B are diagrams showing examples of images captured by an imaging device according to a third embodiment. [Figure 7] 10 is a flowchart showing a control procedure for a process of determining whether or not a handrail is in use according to the fourth embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0036] Hereinafter, an embodiment of the present invention will be described with reference to the drawings.

[0037] (First embodiment) <Image recognition system configuration> FIG. 1 is a block diagram showing the functional configuration of an image recognition system 100. As shown in FIG. The image recognition system 100 includes an imaging device 10, an information processing device 20, and an alarm device 30. The information processing device 20 is communicatively connected to the imaging device 10 and the alarm device 30 wirelessly or via a wire, and is capable of transmitting and receiving data to and from the imaging device 10 and the alarm device 30. The imaging device 10 transmits image data relating to the captured moving image to the information processing device 20. The image data relating to the moving image includes image data of a plurality of captured images that make up the moving image.

[0038] FIG. 2 is a diagram showing an example of an image 50 captured by the imaging device 10. As shown in FIG. The imaging device 10 is installed near a passage A, such as a staircase, within a facility. In the example shown in FIG. The installation angle and elevation angle of the imaging device 10 are set so that at least the passage A, the handrail B provided in the passage A, the hand 61 of the target person 60 on the handrail B side while the target person 60 is holding the handrail B, and the state of the target person 60 passing by are included in the field of view. The imaging device 10 captures a moving image of a target person 60 passing through the passage A at a predetermined frame rate. The imaging device 10 may capture color moving images or monochrome moving images. The imaging resolution of the imaging device 10 is not particularly limited as long as it is within a range that allows appropriate detection of the presence or absence of the ankles of the target person 60 in a person entry area 51 (first area) described below, and the presence or absence of the wrists of the target person 60 in a hand entry area 52 (second area).

[0039] The information processing device 20 includes a control unit 21, a storage unit 22, a communication unit 23, an operation unit 24, a display unit 25, etc. The units of the information processing device 20 are connected via a bus.

[0040] The control unit 21 includes a CPU (Central Processing Unit), a RAM (Random Access Memory), etc. The control unit 21 reads and executes programs stored in the storage unit 22 and performs various arithmetic processing, thereby controlling the operation of the information processing device 20.

[0041] The storage unit 22 is a non-transitory recording medium readable by the control unit 21 as a computer, and stores programs, various data, etc. The storage unit 22 includes a non-volatile memory such as an HDD (Hard Disk Drive) or an SSD (Solid State Drive).

[0042] The storage unit 22 stores a skeleton detection model 221. The skeleton detection model 221 is a machine learning model that, when image data of a captured image 50 of a target person 60 is input, detects a plurality of analysis points 70 corresponding to a plurality of body parts of the target person 60 and outputs position (coordinate) information of the analysis points 70. The position information of the analysis points 70 is skeleton position information. Hereinafter, "detecting a plurality of analysis points 70 corresponding to a plurality of body parts of the target person 60" will also be referred to as "detecting the skeleton of the target person 60." When a plurality of target persons 60 are captured in the captured image 50, the skeleton detection model 221 detects the skeleton of each target person 60.

[0043] 2 shows an example of analysis points 70. The skeleton detection model 221 detects analysis points 70 for, for example, 18 body parts (plural body parts) of the target person 60. The 18 body parts are, for example, both eyes, both ears, nose, throat, both shoulders, both elbows, both wrists, the base of the left and right legs, both knees, and both ankles. However, the positions and number of analysis points 70 are not limited to these. The skeleton detection model 221 omits detection of analysis points 70 for body parts that are not shown in the captured image 50 out of the above 18 body parts. The analysis points 70 include an analysis point 71 corresponding to the wrist of the target person 60, an analysis point 72 corresponding to the ankle of the target person 60, and the like.

[0044] The storage unit 22 has a determination data area 222 . In the determination data area 222, data relating to the determination result as to whether or not the target person 60 used the handrail B while passing through the passage A is recorded in association with the captured image 50 used for the determination.

[0045] The communication unit 23 includes a network card, a communication module, etc., and transmits and receives data to and from the imaging device 10 and the alarm device 30 in accordance with a predetermined communication standard.

[0046] The operation unit 24 includes a keyboard having cursor keys, character input keys, various function keys, etc., and a pointing device such as a mouse. The operation unit 24 outputs operation signals input by key operations on the keyboard, mouse operations, etc., to the control unit 21. The operation unit 24 may include a touch panel or the like, and output operation signals to the control unit 21 according to the position of an operation by the operator's finger, etc.

[0047] The display unit 25 includes a liquid crystal display or the like, and performs various displays in accordance with a display control signal from the control unit 21 of the information processing device 20 .

[0048] When the alarm device 30 receives a predetermined control signal from the information processing device 20, it issues an alarm near the passage A. The alarm may be visual information such as a lamp, audio information such as a buzzer, or a combination thereof.

[0049] In the image recognition system 100, the imaging device 10 captures a video of a target person 60 passing through passage A, and transmits image data of captured images 50 constituting the video to the information processing device 20. Next, the information processing device 20 analyzes the image data of the received captured images 50 to determine whether the target person 60 has used handrail B. The information processing device 20 records the determination result in association with the captured images 50.

[0050] <Image recognition system operation> Next, the operation of the image recognition system 100 will be described. 3 is a flowchart showing the flow of the determination process executed by the image recognition system 100. The image recognition system 100 determines whether or not the target person 60 used the handrail B while passing through the passage A through the determination process.

[0051] (Determination process of the first embodiment) The control unit 21 of the information processing device 20 acquires image data of the latest captured image 50 that constitutes a moving image captured by the imaging device 10 (step S1). Hereinafter, the "image data of the captured image 50" may also be simply referred to as the "captured image 50." That is, an image (captured image 50) of the target person 60 captured by the imaging device 10 is acquired. The control unit 21 functions as an image acquisition unit. Step S1 is an image acquisition step.

[0052] Next, the control unit 21 acquires the shooting time of the captured image 50 acquired in step S1 (step S2). When the control unit 21 imports the captured image 50 from a moving image in real time, the control unit 21 may use the time when the captured image 50 is acquired as the shooting time of the captured image 50. Alternatively, the control unit 21 may acquire the shooting time of the captured image 50 from the imaging device 10. Next, the control unit 21 inputs the captured image 50 acquired in step S1 into the skeleton detection model 221 and detects the skeleton of the target person 60 (step S3). In other words, the control unit 21 detects a plurality of analysis points 70 of the target person 60 using the skeleton detection model 221. That is, the control unit 21 detects the skeleton position of the target person 60 based on the captured image 50. The control unit 21 functions as a skeleton detection unit. Step S3 is a skeleton detection step.

[0053] Next, the control unit 21 determines whether or not the skeleton of the target person 60 has been detected in step S3 (step S4). The case where the skeleton of the target person 60 is detected in the captured image 50 (step S4; YES) will be described. In this case, the control unit 21 determines whether or not analysis points 72 corresponding to both ankles of the target person 60 detected in step S3 exist in the human entry area 51 provided in the captured image 50 acquired in step S1. The human entry area 51 is an area in the captured image 50 that includes the entire passage A. If analysis points 72 corresponding to both ankles of the target person 60 exist in the human entry area 51 , the control unit 21 determines that the target person 60 exists in the human entry area 51 . If there are no analysis points 72 corresponding to both ankles of the target person 60 in the human entry area 51, the control unit 21 determines that the target person 60 is not present in the human entry area 51. As described above, the control unit 21 determines whether or not the target person 60 is present in the human entry area 51 (first area) (step S5). The control unit 21 functions as a human entry determination unit.

[0054] Next, the control unit 21 determines whether or not it has been determined in step S5 that the target person 60 is present in the person entry area 51 (step S6). If it is determined that the target person 60 is present in the person entry area 51 (step S6; YES), the control unit 21 determines whether or not the analysis point 71 corresponding to the wrist of the target person 60 on the handrail B side detected in step S3 is present in the hand entry area 52 provided in the captured image 50 acquired in step S1. The hand entry area 52 is an area in the captured image 50 that includes the entire handrail B. That is, the control unit 21 determines whether the target person 60 is present in a predetermined area (hand entry area 52) of the captured image 50 based on the skeletal position of the target person 60. The control unit 21 functions as an entry determination unit. The control unit 21 determines whether or not the hand 61 of the target person 60 is present in the second area (hand entry area 52) of the captured image 50. The control unit 21 functions as a hand entry determination unit.

[0055] When an analysis point 71 corresponding to the wrist of the target person 60 on the handrail B side is present in the hand entry area 52, the control unit 21 determines that the target person 60 is gripping the handrail B. In other words, the control unit 21 determines that the target person 60 is using the handrail B. If there is no analysis point 71 in the hand access area 52 that corresponds to the wrist of the target person 60 on the handrail B side, the control unit 21 determines that the target person 60 is not gripping the handrail B. In other words, the control unit 21 determines that the target person 60 is not using the handrail B. As described above, the control unit 21 determines whether or not the target person 60 is using the handrail B (step S7). That is, the control unit 21 determines whether the target person 60 is using the handrail B based on the determination result as the entry determination unit. The control unit 21 functions as a handrail use determination unit. Step S7 is an entry determination step and a handrail use determination step.

[0056] The control unit 21 may receive in advance via the operation unit 24 a setting as to whether the wrist of the target person 60 on the handrail B side is the right wrist or the left wrist of the target person 60. The setting as to whether the wrist of the target person 60 on the handrail B side is the right wrist or the left wrist is a setting of a parameter related to the determination of whether the handrail is being used.

[0057] Next, the control unit 21 displays on the display unit 25 the captured image 50, on which are superimposed displays such as a frame indicating the person entry area 51, a frame indicating the hand entry area 52, and red dots indicating the multiple analysis points 70 detected in step S3 (step S8). The red dots indicating the analysis points 70 are displays of skeletal positions, and include red dots indicating analysis point 71 corresponding to the wrist of the target person 60 on the handrail B side. The control unit 21 may superimpose a bounding box surrounding the hand 61 of the target person 60 on the handrail B side on the captured image 50, instead of red dots indicating the analysis points 71. 2, the control unit 21 displays the frame line (dashed line) indicating the person entry area 51 and the frame line (dash line) indicating the hand entry area 52 in different colors. Alternatively, the control unit 21 may display the frame color indicating the person entry area 51 and the frame color indicating the hand entry area 52 in different colors. In other words, the control unit 21 displays the person entry area 51 and the hand entry area 52 in different display modes in the captured image 50.

[0058] Next, the control unit 21 stores the determination result of whether the target person 60 is using the handrail B in step S7 in the determination data area 222 in association with the captured image 50 acquired in step S1 (step S9). The control unit 21 functions as a recording control unit. Next, the control unit 21 determines whether or not a predetermined first time has elapsed since it was first determined in step S5 that the target person 60 was present in the human entry area 51. In other words, the control unit 21 determines whether or not a predetermined first time has elapsed since the target person 60 entered the human entry area 51 (step S10). If the first time or more has passed since the target person 60 entered the human entry area 51 (step S10; YES), the control unit 21 shifts the determination process to step S13.

[0059] On the other hand, if it has been less than the first time since the target person 60 entered the human entry area 51 (step S10; NO), the control unit 21 transitions the judgment process to step S1 and acquires image data of the next captured image 50 in the video captured by the imaging device 10. Steps S1 to S10 are repeatedly executed at a cycle corresponding to the frame rate of image capture by the image capture device 10. For example, if the frame rate is 10 fps, 10 frames of captured images 50 are captured per second, and therefore steps S1 to S10 are repeatedly executed every 1 / 10 of a second.

[0060] On the other hand, a case where the skeleton of the target person 60 is not detected in the captured image 50 (step S4; YES) or where it is determined that the target person 60 does not exist in the human entry area 51 (step S6; NO) will be described. In this case, the control unit 21 determines whether or not it has been determined that the target person 60 exists in the human entry area 51 in the captured image 50, which is the previous frame image in the moving image captured by the imaging device 10. If the control unit 21 determines that the target person 60 is present in the human entry area 51 in the previous captured image 50 in the video captured by the imaging device 10, the control unit 21 determines that the target person 60 has exited the human entry area 51. If the control unit 21 determines that the target person 60 is not present in the human entry area 51 in the previous captured image 50 in the video captured by the imaging device 10, it determines that the target person 60 has not entered the human entry area 51 and has not exited the human entry area 51. As described above, the control unit 21 determines whether the target person 60 has exited the human entry area 51 (step S11).

[0061] Next, the control unit 21 determines whether or not it is determined in step S11 that the target person 60 has exited the person entry area 51 (step S12). If it is determined that the target person 60 has not exited the human entry area 51 (step S12; NO), the control unit 21 transitions the determination process to step S1 and acquires image data of the next captured image 50 in the video captured by the imaging device 10.

[0062] On the other hand, a case will be described where it is determined that the target person 60 has exited the human entry area 51 (step S12; YES), or where a first time or more has passed since the target person 60 entered the human entry area 51 (step S10; YES). In this case, the control unit 21 determines whether the target person 60 used the handrail B while passing through the passage A (step S13). For example, the control unit 21 calculates the duration of the state in which the target person 60 holds onto the handrail B based on the shooting time of the captured image 50 acquired in step S2. Next, the control unit 21 determines that the target person 60 used the handrail B if the duration of the state in which the target person 60 held onto the handrail B while passing through the passage A is equal to or longer than a predetermined second time. The control unit 21 determines that the target person 60 did not use the handrail B if the duration of the state in which the target person 60 held onto the handrail B while passing through the passage A is less than the predetermined second time. The control unit 21 may accept the setting of the second time via the operation unit 24. The second time is a parameter related to the determination of whether or not the handrail was used.

[0063] If it is determined that the target person 60 did not use the handrail B while passing through the passage A (step S13; NO), the control unit 21 issues a warning (step S14). That is, the control unit 21 issues a warning in accordance with the determination result as the handrail use determination unit 21. The control unit 21 functions as a warning control unit. Specifically, the control unit 21 transmits, as a warning, an instruction signal to issue an alarm to the alarm device 30. Upon receiving the instruction signal from the information processing device 20, the alarm device 30 issues, as an alarm, visual information such as a lamp and / or auditory information such as a buzzer toward the vicinity of the passage A. The control unit 21 may display, as a warning, on the display unit 25, a message indicating that it has been determined that the target person 60 did not use the handrail B while passing through the passage A. As a warning, the control unit 21 may send an instruction signal to issue an alarm to another alarm device provided in a control room or the like of the facility where the handrail B is installed, so that an alarm is issued in the control room. As a warning, the control unit 21 may send a message to an external terminal device used by the manager of the facility where the handrail B is installed, stating that it has determined that the target person 60 did not use the handrail B while passing through the passage A.

[0064] In step S14, the control unit 21 may display on the display unit 25 the determination result of whether or not the target person 60 used the handrail B while passing through the passage A in step S13. That is, the control unit 21 performs display on the display unit 25 according to the determination result as the handrail use determination unit. The control unit 21 functions as a display control unit.

[0065] On the other hand, if it is determined that the target person 60 used the handrail B while passing through the passage A (step S13; YES), the control unit 21 shifts the determination process to step S15.

[0066] Next, the control unit 21 counts the total number of target persons 60 who have passed through passage A. Furthermore, based on the determination result of step S13, the control unit 21 counts the number of target persons 60 who used handrail B while passing through passage A, or the number of target persons 60 who did not use handrail B while passing through passage A. The control unit 21 stores the counted number of people in the memory unit 22 (step S15), and ends the determination process. That is, the control unit 21 counts the number of people who used the handrail B and the number of people who did not use the handrail B, depending on the determination result as the handrail use determination unit. The control unit 21 functions as a counting unit.

[0067] In step S7 of the above-described determination process, the control unit 21 may determine whether or not the target person 60 is using the handrail B as follows. Specifically, the control unit 21 determines whether or not an analysis point 71 corresponding to the wrist of the target person 60 on the handrail B side is present in the person entry area 51. When there is no analysis point 71 corresponding to the wrist of the target person 60 on the handrail B side in the person entry area 51, the control unit 21 determines that the target person 60 is holding the handrail B. In other words, the control unit 21 determines that the target person 60 is using the handrail B. If an analysis point 71 corresponding to the wrist of the target person 60 on the handrail B side is present in the person entry area 51, the control unit 21 determines that the target person 60 is not holding onto the handrail B. In other words, the control unit 21 determines that the target person 60 is not using the handrail B.

[0068] In the above determination process, the control unit 21 may omit steps S5 and S6. In this case, in step S7, if an analysis point 71 corresponding to the wrist of the target person 60 on the handrail B side is present in the hand entry area 52, the control unit 21 determines that the target person 60 is passing through passage A and is holding onto handrail B. In other words, the control unit 21 determines that the target person 60 is passing through passage A and is using handrail B. If there is no analysis point 71 in the hand entry area 52 that corresponds to the wrist of the target person 60 on the handrail B side, the control unit 21 determines that the target person 60 is passing through passage A and is not holding onto handrail B. In other words, the control unit 21 determines that the target person 60 is passing through passage A and is not using handrail B.

[0069] In the above determination process, a case will be described in which a plurality of target persons 60 are captured in the captured image 50. In this case, the control unit 21 detects the skeleton of each target person 60 using the skeleton detection model 221. Furthermore, the control unit 21 detects each target person 60 using object detection technology and assigns an identification number to each detected target person 60. Next, the control unit 21 determines whether or not each target person 60 used handrail B while passing through passage A. Next, the control unit 21 stores the determination result of whether or not handrail B was used while passing through passage A in association with the identification number in the storage unit 22. In this case, the control unit 21 simultaneously counts the number of people who used handrail B while passing through passage A and the number of people who did not use handrail B.

[0070] Alternatively, if the target person 60 is wearing a helmet or the like to which a QR code (registered trademark) or the like containing personal information is attached, the control unit 21 may acquire the personal information of the target person 60 by reading the QR code and identify the target person 60 (personal authentication). In this case as well, the control unit 21 determines whether or not each target person 60 used handrail B while passing through passage A. Next, the control unit 21 stores the determination result of whether or not handrail B was used while passing through passage A in association with the personal information in the storage unit 22. Next, the control unit 21 counts the number of times handrail B was used while passing through passage A and / or the number of times handrail B was not used while passing through passage A for each authenticated individual and stores the count in the storage unit 22.

[0071] In the above determination process, the control unit 21 may determine attributes such as age, sex, clothing, etc. of the target person 60 from the captured image 50. The control unit 21 functions as an attribute determination unit. In this case, the control unit 21 stores the determination result of whether or not handrail B was used while passing through passage A in association with the attributes of the target person 60 in the storage unit 22. Next, the control unit 21 counts, for each attribute of the target person 60, the number of people who used handrail B while passing through passage A, the number of people who did not use handrail B while passing through passage A, the total number of people who passed through passage A, etc., and stores these in the storage unit 22. For example, the control unit 21 counts the number of men who did not use handrail B while passing through passage A, the number of women who did not use handrail B while passing through passage A, etc.

[0072] In the above determination process, the control unit 21 may, for example, accept the setting of a predetermined time period (such as 9:00 AM to 10:00 AM) via the operation unit 24. In this case, the control unit 21 counts the number of people who used handrail B while passing through passage A, the number of people who did not use handrail B while passing through passage A, the total number of people who passed through passage A, etc. during the set time period.

[0073] In the above determination process, the control unit 21 may receive settings for the range of the person entry area 51 and the range of the hand entry area 52 provided in the captured image 50 via the operation unit 24.

[0074] In the above-described determination process, steps S1 to S10 are repeatedly executed at a cycle corresponding to the frame rate of image capture by the image capture device 10, but this is not limitative. The control unit 21 of the information processing device 20 may receive a setting of the cycle for executing steps S1 to S10 via the operation unit 24. In this case, the control unit 21 repeatedly executes steps S1 to S10 at the set cycle.

[0075] (Second embodiment) Next, a second embodiment will be described, focusing on the differences from the first embodiment. FIG. 4 is a flowchart showing the flow of the determination process in the second embodiment.

[0076] (Determination process of the second embodiment) In the second embodiment, the control unit 21 performs steps S21 to S33 similar to steps S1 to S9, S11 to S13, and S15 in the first embodiment. Next, the control unit 21 calculates the movement speed of the target person 60 in the human entry area 51 based on the shooting time of the captured image 50 acquired in step S22 (step S34). Specifically, the control unit 21 calculates the movement speed of the target person 60 in the human entry area 51 from the time when the target person 60 entered the human entry area 51 and the time when the target person 60 exited the human entry area 51.

[0077] Next, the control unit 21 determines whether or not it is determined in step S32 that the target person 60 did not use the handrail B while passing through the passage A (step S35). A case will be described where it is determined that the target person 60 did not use the handrail B while passing through the passage A (step S35; YES). In this case, the control unit 21 performs step S36, which is the same as step S14 in the first embodiment, and ends the determination process.

[0078] On the other hand, if it is determined that the target person 60 used the handrail B while passing through the passage A (step S35; NO), the control unit 21 determines whether the movement speed of the target person 60 calculated in step S34 exceeds a predetermined threshold value (step S37). The control unit 21 may accept the setting of the predetermined threshold value via the operation unit 24. If the moving speed of the target person 60 exceeds the predetermined threshold (step S37; YES), the control unit 21 issues a warning (step S38) and ends the determination process. Specifically, the control unit 21 transmits an instruction signal to issue a warning to the alarm device 30. When the alarm device 30 receives the instruction signal from the information processing device 20, it issues auditory information such as "Please walk through" as a warning to people near the passage A. The control unit 21 may display a warning message such as "Please walk through" on the display unit 25. In other words, the control unit 21 changes the display content of the display unit 25 according to the determination result as the handrail use determination unit and the moving speed of the target person 60 in the passage A.

[0079] On the other hand, if the moving speed of the target person 60 is equal to or less than the predetermined threshold (step S37; NO), the control unit 21 ends the determination process.

[0080] As described above, in the determination process of the second embodiment, the control unit 21 issues a warning or caution depending on the determination result as the handrail use determination unit and the moving speed of the target person 60 in the passage A. In other words, the control unit 21 changes the content of the warning depending on the determination result as the handrail use determination unit and the moving speed of the target person 60 in the passage A.

[0081] (Third embodiment) Next, a third embodiment will be described, focusing on the differences from the first embodiment. FIG. 5 is a flowchart showing the flow of the determination process according to the third embodiment. In the third embodiment, the direction of travel in the passage A is predetermined.

[0082] In the determination process of the third embodiment, as shown in FIG. 6, a first person entry area 51a and a second person entry area 51b are provided in place of the person entry area 51 in the captured image 50. The first person entry area 51a is an area in the captured image 50 where one end of the passage A is captured. The second person entry area 51b is an area in the captured image 50 where the other end of the passage A is captured.

[0083] (Determination process of the third embodiment) In the third embodiment, the control unit 21 performs steps S41 to S44 similar to steps S1 to S4 in the first embodiment. A case where the skeleton of the target person 60 is detected in the captured image 50 (step S44; YES) will be described. In this case, the control unit 21 determines whether or not analysis points 72 corresponding to both ankles of the target person 60 detected in step S43 exist in the first person access area 51a or the second person access area 51b provided in the captured image 50 acquired in step S41. When analysis points 72 corresponding to both ankles of the target person 60 exist in the first-person access area 51a, the control unit 21 determines that the target person 60 exists in the first-person access area 51a. When analysis points 72 corresponding to both ankles of the target person 60 exist in the second-person access area 51b, the control unit 21 determines that the target person 60 exists in the second-person access area 51b. If there are no analysis points 72 corresponding to both ankles of the target person 60 in the first person entry area 51a and the second person entry area 51b, the control unit 21 determines that the target person 60 is not present in the first person entry area 51a and the second person entry area 51b. As described above, the control unit 21 determines whether or not the target person 60 is present in the first person access area 51a and the second person access area 51b (step S45).

[0084] Next, the control unit 21 determines whether or not it has been determined in step S45 that the target person 60 is present in the first person access area 51a or the second person access area 51b (step S46). If it is determined that the target person 60 is not present in the first person entry area 51a or the second person entry area 51b (step S46; NO), the control unit 21 transitions the determination process to step S41 and acquires image data of the next captured image 50 in the video captured by the imaging device 10.

[0085] On the other hand, if it is determined that the target person 60 is present in the first person access area 51a or the second person access area 51b (step S46; YES), the control unit 21 performs steps S47 to S49 similar to steps S7 to S9 in the first embodiment. In step S48, the control unit 21 displays on the display unit 25 the captured image 50, on which a frame or the like indicating the first person entry area 51a, a frame or the like indicating the second person entry area 51b, a frame or the like indicating the hand entry area 52, and red dots or the like indicating the multiple analysis points 70 detected in step S43 are superimposed.

[0086] Next, the control unit 21 acquires image data of the next captured image 50 in the moving image captured by the imaging device 10 (step S50). Next, the control unit 21 performs steps S51 and S52 similar to steps S42 and S43.

[0087] Next, the control unit 21 determines whether or not analysis points 72 corresponding to both ankles of the target person 60 detected in step S52 are present in the opposite person entry area (described later) provided in the captured image 50 acquired in step S50. In step S45, when it is determined that the target person 60 is present in the first person entry area 51a, the person entry area on the opposite side is the second person entry area 51b. On the other hand, if it is determined in step S45 that the target person 60 is present in the second person entry area 51b, the person entry area on the opposite side is the first person entry area 51a. If analysis points 72 corresponding to both ankles of the target person 60 exist in the person entry area on the opposite side, the control unit 21 determines that the target person 60 exists in the person entry area on the opposite side. If there are no analysis points 72 corresponding to both ankles of the target person 60 in the person entry area on the opposite side, the control unit 21 determines that the target person 60 is not present in the person entry area on the opposite side. As described above, the control unit 21 determines whether or not the target person 60 is present in the person entry area on the opposite side (step S53).

[0088] Next, the control unit 21 determines whether or not it has been determined in step S53 that the target person 60 is present in the person entry area on the opposite side (step S54). If it is determined that the target person 60 does not exist in the person entry area on the opposite side (step S54; NO), the control unit 21 shifts the determination process to step S47.

[0089] On the other hand, if it is determined that the target person 60 is present in the person entry area on the opposite side (step S54; YES), the control unit 21 performs steps S55 and S56 similar to steps S13 and S15 in the first embodiment. Next, the control unit 21 determines the direction of travel of the target person 60 in the passage A based on the shooting time of the captured image 50 in which the target person 60 present in the first person entry area 51a is captured and the shooting time of the captured image 50 in which the target person 60 present in the second person entry area 51b is captured (step S57). That is, the control unit 21 determines the traveling direction of the target person 60 in the passage A based on the time series data of the detection time of the target person 60 in the predetermined areas (the first person entry area 51a and the second person entry area 51b).

[0090] Next, the control unit 21 determines whether or not it is determined in step S55 that the target person 60 did not use the handrail B while passing through the passage A (step S58). A case will be described where it is determined that the target person 60 did not use the handrail B while passing through the passage A (step S58; YES). In this case, the control unit 21 performs step S59, which is the same as step S14 in the first embodiment, and ends the determination process.

[0091] On the other hand, if it is determined that the target person 60 used handrail B while passing through passage A (step S58; NO), the control unit 21 determines whether the direction of travel of the target person 60 determined in step S57 is opposite to the predetermined direction of travel through passage A (step S60). If the target person 60 is moving in the opposite direction (step S60; YES), the control unit 21 issues a warning (step S61) and ends the determination process. Specifically, the control unit 21 transmits an instruction signal to issue a warning to the alarm device 30. When the alarm device 30 receives the instruction signal from the information processing device 20, it issues auditory information such as "Please go in the opposite direction" as a warning toward the vicinity of passage A. On the other hand, if the moving direction of the target person 60 is not reversed (step S60; NO), the control unit 21 ends the determination process.

[0092] In the third embodiment, the passage A may be a two-way passage. In this case, in the determination process, a person entry area is set in the captured image 50 in an area where one end of the right side of the passage A is imaged, an area where the other end of the right side of the passage A is imaged, an area where one end of the left side of the passage A is imaged, and an area where the other end of the left side of the passage A is imaged.

[0093] In step S57 of the determination processing of the third embodiment described above, the control unit 21 may determine the direction of travel of the target person 60 in the passage A by determining the body orientation of the target person 60 from the skeleton of the target person 60 detected in steps S43 and S52. In step S57 of the determination process in the third embodiment, the control unit 21 may determine the traveling direction of the target person 60 in the passage A based on tracking of the target person 60 in the captured image 50.

[0094] (Fourth embodiment) Next, a fourth embodiment will be described, focusing on the differences from the first embodiment. In the fourth embodiment, the control unit 21 executes a handrail use determination process shown in FIG. 7 in step S7 of the determination process in the first embodiment.

[0095] (Handrail use determination process) The control unit 21 identifies and sets a handrail portion that is a portion corresponding to the handrail B in the captured image 50 acquired in step S1 (step S71). The control unit 21 functions as an image identification unit. Step S71 is an image identification step. In step S71, the control unit 21 displays the captured image 50 acquired in step S1 on the display unit 25. Thereafter, the control unit 21 accepts an operation by a user, such as a facility manager, to specify a handrail section in the displayed captured image 50 via the operation unit 24. Thereafter, the control unit 21 sets the handrail section specified by the user in the captured image 50.

[0096] Alternatively, in step S71, the control unit 21 may identify a handrail portion included in the captured image 50 by image recognition AI (Artificial Intelligence) and set the identified handrail portion in the captured image 50. The control unit 21 pre-registers images including handrails with labeled parts corresponding to the handrails in the image recognition AI as learning data, calculates the features of the handrails included in the images, and stores them in the memory unit 22. In step S71, the control unit 21 identifies the handrail portion by evaluating the captured image 50 acquired in step S1 based on the features of the handrail previously stored in the memory unit 22, and sets the identified handrail portion in the captured image 50.

[0097] Next, the control unit 21 calculates the coordinates of the first part of the target person 60 in the captured image 50 acquired in step S1 based on the detection result of the skeleton of the target person 60 in step S3 (step S72). The first part of the target person 60 is at least one of the back of the hand, the palm, and the arm of the target person 60.

[0098] Next, the control unit 21 calculates the coordinates of the handrail section set in step S71 in the captured image 50 acquired in step S1 (step S73). Next, the control unit 21 determines whether or not the coordinates of the first part of the target person 60 calculated in step S72 match the coordinates of the handrail calculated in step S73 (step S74).

[0099] If the coordinates of the first part of the target person 60 and the coordinates of the handrail part match (step S74; YES), the control unit 21 determines that the target person 60 is holding onto the handrail B. In other words, the control unit 21 determines that the target person 60 is using the handrail B (step S75). Thereafter, the control unit 21 ends the handrail use determination process.

[0100] On the other hand, if the coordinates of the first part of the target person 60 do not match the coordinates of the handrail part (step S74; NO), the control unit 21 calculates the coordinates of the shoulders of the target person 60 in the captured image 50 acquired in step S1 (step S76). In step S76, the control unit 21 inputs the captured image 50 acquired in step S1 to the image recognition AI. The image recognition AI has previously learned the height (number of pixels) of the person in the captured image captured by the imaging device 10. Thereafter, the control unit 21 uses an image recognition AI to calculate the position (coordinates) of the shoulders that is assumed from the positions of the ankles of the target person 60. Alternatively, the control unit 21 uses an image recognition AI to identify the shoulders of the target person 60 and calculate the position (coordinates) of the shoulders.

[0101] Next, the control unit 21 calculates the distance between the shoulder of the target person 60 and the handrail in the captured image 50 based on the coordinate of the shoulder of the target person 60 calculated in step S76 and the coordinate of the handrail calculated in step S73 (step S77). Next, the control unit 21 determines whether the distance between the shoulder of the target person 60 and the handrail calculated in step S77 is less than the arm length of an average person estimated from the height of the target person 60 based on the skeleton detection results (step S78).

[0102] If the distance between the shoulder of the target person 60 and the handrail is equal to or shorter than the length of the arm (step S78; YES), the control unit 21 determines that the target person 60 is not holding onto the handrail B, that is, is not using the handrail B, and determines that this is abnormal (step S79). Thereafter, the control unit 21 ends the handrail use determination process.

[0103] On the other hand, if the distance between the shoulder of the target person 60 and the handrail is longer than the length of the arm (step S78; NO), the control unit 21 determines that the target person 60 has not entered a range where he or she can grasp the handrail B (step S80). Thereafter, the control unit 21 ends the handrail use determination process.

[0104] In the handrail use determination process, the control unit 21 determines whether the target person 60 is using the handrail B based on the position of the handrail part and the skeletal position of the target person 60. The control unit 21 functions as a handrail use determination unit.

[0105] In the above-mentioned handrail use determination process, if the coordinates of the first part of the target person 60 do not match the coordinates of the handrail part (step S74; NO), the control unit 21 may determine that the target person 60 is not holding onto the handrail B, i.e., is not using the handrail B. Thereafter, the control unit 21 executes steps S76 to S78. In this case, if the distance between the shoulder of the target person 60 and the handrail is equal to or shorter than the length of the arm (step S78; YES), the control unit 21 determines that an abnormality has occurred. On the other hand, if the distance between the shoulder of the target person 60 and the handrail is longer than the length of the arm (step S78; NO), the control unit 21 does not determine that there is an abnormality.

[0106] <Effects> As described above, the image recognition system 100 according to this embodiment includes an image acquisition unit (control unit 21) that acquires an image (captured image 50) of the target person 60 captured by the imaging device 10. The image recognition system 100 includes a skeleton detection unit (control unit 21) that detects the skeleton position of the target person 60 based on the image acquired by the image acquisition unit. The image recognition system 100 includes an intrusion determination unit (control unit 21) that determines the presence or absence of a target person 60 in a predetermined area of ​​the image based on the skeleton position detected by the skeleton detection unit. The image recognition system 100 includes a handrail use determination unit (control unit 21) that determines whether or not the target person 60 is using the handrail based on the determination result by the entry determination unit. Therefore, it is possible to determine whether or not the target person 60 continuously used the handrail while passing through the passage A captured in the captured image 50. By detecting the skeleton of the target person 60, it is possible to determine whether or not the hand 61 of the target person 60 is gripping the handrail. This makes it possible to determine with high accuracy whether the target person is using the handrail.

[0107] In the image recognition system 100 according to this embodiment, the intrusion determination unit (control unit 21) includes a human intrusion determination unit (control unit 21) that determines whether or not a target person 60 is present in a first area (human intrusion area 51) of an image (captured image 50). The entry determination unit (control unit 21) includes a hand entry determination unit (control unit 21) that determines whether or not a hand 61 of a target person 60 is present in a second area (hand entry area 52) of the image. Therefore, the presence or absence of the target person 60 in the first area (person entry area 51) can be used to determine with high accuracy whether the target person 60 is passing through the passage A. The presence or absence of the target person 60's hand 61 in the second area (hand entry area 52) can be used to determine with high accuracy whether the target person 60 is using the handrail.

[0108] In the image recognition system 100 according to this embodiment, the entry determination unit (control unit 21) displays the first area (person entry area 51) and the second area (hand entry area 52) on the display unit 25 in different display modes, superimposed on the image (captured image 50). Therefore, it is possible to easily visually confirm whether the target person 60 is passing through the passage A and whether the target person 60 is using the handrail.

[0109] In the image recognition system 100 according to this embodiment, the entry determination unit (control unit 21) displays the position of the hand 61 in the image (captured image 50) on the display unit 25 by displaying the skeleton position or a bounding box. Therefore, it is possible to easily visually check whether the target person 60 is using the handrail.

[0110] In the image recognition system 100 according to this embodiment, the handrail use determination unit (control unit 21) determines whether the handrail is being used when the person entry determination unit (control unit 21) determines that a target person 60 is present in the first area and the hand entry determination unit (control unit 21) determines that a hand 61 of the target person 60 is present in the second area. Therefore, it is possible to eliminate cases where the target person 60 is just holding onto the handrail without climbing the stairs, for example, at the beginning of climbing the stairs. This makes it possible to determine with high accuracy whether the target person 60 is using the handrail while passing through passage A.

[0111] The image recognition system 100 according to this embodiment includes a warning control unit (control unit 21) that issues a warning in accordance with the result of the determination by the handrail use determination unit (control unit 21). Therefore, if the target person 60 does not use the handrail while passing through the passage A, a warning can be issued.

[0112] In the image recognition system 100 according to this embodiment, the warning control unit (control unit 21) issues a warning to a terminal device used by a manager of a facility where a handrail is installed. Therefore, the manager can know that someone passed through aisle A without using the handrail.

[0113] The image recognition system 100 according to this embodiment includes a display control unit (control unit 21) that displays on the display unit 25 according to the determination result by the handrail use determination unit (control unit 21). Therefore, it is possible to easily determine whether the target person 60 is using the handrail.

[0114] The image recognition system 100 according to this embodiment includes a counting unit (control unit 21) that counts the number of people who used the handrail and the number of people who did not use the handrail according to the determination result by the handrail use determination unit (control unit 21). Therefore, data such as the percentage of people who did not use the handrail can be obtained.

[0115] In the image recognition system 100 according to this embodiment, the counting unit (control unit 21) performs personal authentication of the target person 60, and counts the number of times the handrail is used and / or the number of times the handrail is not used for each authenticated individual. Therefore, data such as the percentage of cases where the handrail was not used can be obtained for each individual.

[0116] In the image recognition system 100 according to this embodiment, the counting unit (control unit 21) assigns an identification number to each of multiple target persons 60 through object detection, and simultaneously counts the number of people who used the handrail and the number of people who did not use the handrail. Therefore, even if a plurality of target persons 60 are captured in the captured image 50, it is possible to quickly count the number of people who used the handrail and the number of people who did not use the handrail.

[0117] In the image recognition system 100 according to this embodiment, the handrail use determination unit (control unit 21) determines that the handrail is being used when the state in which the handrail is being used continues for a predetermined time (second time). Therefore, it can be determined with high accuracy that the target person 60 has been continuously using the handrail while passing through the passage A.

[0118] In the image recognition system 100 according to this embodiment, the handrail use determination unit (control unit 21) accepts the setting of parameters related to the determination of whether or not a handrail is being used, including the predetermined time period (second time period). Therefore, it is possible to determine whether or not a handrail is being used based on criteria desired by the user of the image recognition system 100.

[0119] The image recognition system 100 according to this embodiment includes a recording control unit (control unit 21) that stores an image (captured image 50) in association with the determination result by the handrail use determination unit (control unit 21). Therefore, after the determination process, it is possible to check in the captured image 50 whether the target person 60 is using the handrail.

[0120] In the image recognition system 100 according to this embodiment, the entry determination unit (control unit 21) receives settings of the ranges of the predetermined areas (person entry area 51, hand entry area 52). Therefore, it is possible to determine whether or not a handrail is being used within a range desired by the user of the image recognition system 100.

[0121] The image recognition system 100 according to this embodiment includes an attribute determination unit (control unit 21) that determines at least one of the age, sex, and clothing of the target person 60. Therefore, it is possible to determine whether or not the handrail is used for each attribute of the target person 60.

[0122] In the image recognition system 100 according to this embodiment, the entry determination unit (control unit 21) determines the direction of travel of the target person 60 in a specified area based on at least one of the body orientation of the target person 60 based on the skeletal position, the time series data of the detection time of the target person 60 in the specified area, and the tracking of the target person 60. Therefore, it is possible to determine whether or not the target person 60 is using the handrail for each direction of travel.

[0123] The image recognition system 100 according to this embodiment includes a warning control unit (control unit 21) that changes the content of the warning depending on the determination result by the handrail use determination unit (control unit 21) and the moving speed of the target person 60 in a predetermined area. Therefore, a warning or caution can be given depending on the moving speed of the target person 60 in a predetermined area.

[0124] In the image recognition system 100 according to this embodiment, the warning control unit (control unit 21) issues a warning when the handrail use determination unit (control unit 21) determines that the handrail is being used and the moving speed is faster than a predetermined threshold. Therefore, if the target person 60 runs through the passage A, a warning can be issued.

[0125] In the image recognition system 100 according to this embodiment, the warning control unit (control unit 21) changes the display content of the display unit 25 according to the judgment result by the handrail use judgment unit (control unit 21) and the movement speed of the target person 60 in a specified area. Therefore, a warning or caution can be given depending on the moving speed of the target person 60 in a predetermined area.

[0126] The image recognition system 100 according to this embodiment includes an image acquisition unit (control unit 21) that acquires an image (captured image 50) of a target person 60 captured by the imaging device 10. The image recognition system 100 includes a skeleton detection unit (control unit 21) that detects the skeleton position of the target person 60 based on the image acquired by the image acquisition unit. The image recognition system 100 includes an image specifying unit (control unit 21) that specifies a handrail portion based on the image acquired by the image acquisition unit. The image recognition system 100 is equipped with a handrail usage determination unit (control unit 21) that determines whether or not the target person 60 is using the handrail B based on the position of the handrail identified by the image identification unit and the skeleton position detected by the skeleton detection unit. Therefore, by detecting the skeleton of the target person 60, it is possible to determine whether the hand 61 of the target person 60 is gripping the handrail. This makes it possible to determine with high accuracy whether the target person is using the handrail.

[0127] In the image recognition system 100 according to this embodiment, the handrail usage determination unit (control unit 21) determines whether or not the target person 60 is using the handrail based on the positional relationship between the position of the handrail unit and at least one of the back of the hand, palm, or arm of the target person 60, which is the skeletal position of the target person 60. Therefore, it is possible to determine with high accuracy whether or not the target person is using the handrail.

[0128] In the image recognition system 100 according to this embodiment, the handrail usage determination unit (control unit 21) determines that the target person 60 is not using the handrail based on the position of the handrail and the skeletal position of the target person 60, and determines that an abnormality has occurred if it determines that the distance between the position of a predetermined part of the target person 60 and the position of the handrail is less than a predetermined distance. Therefore, it is possible to determine with high accuracy whether the target person is not using the handrail or has not entered a range where the target person can grab the handrail.

[0129] In the image recognition system 100 according to this embodiment, the handrail usage determination unit (control unit 21) determines that the target person 60 is not using the handrail based on the position of the handrail and the skeletal position of the target person 60, and if it determines that the distance between the position of a predetermined part of the target person 60 and the position of the handrail is greater than a predetermined distance, it does not determine that there is an abnormality. Therefore, it is possible to determine with high accuracy whether the target person is not using the handrail or has not entered a range where the target person can grab the handrail. If the target person has not entered a range where the target person can grab the handrail, it is possible not to determine that there is an abnormality.

[0130] The description in the above embodiment is an example of the image recognition system, image recognition method, and program according to the present invention, and the present invention is not limited to this. For example, at least a part of the processing performed by the control unit 21 of the information processing device 20 in the above embodiment may be shared and executed by one or more devices other than the information processing device 20 in the image recognition system 100. In this case, the one or more devices may include the imaging device 10. For example, a control unit provided in the imaging device 10 may execute at least a part of the processing executed by the control unit 21 in the above embodiment. In this case, the skeleton detection model 221 may be stored in a storage unit provided in the imaging device 10.

[0131] The information processing device 20 may include an imaging device 10.

[0132] In the above embodiment, the information processing device 20 has been described as analyzing image data of a moving image captured by the imaging device 10 in real time, but the present invention is not limited to this. The information processing device 20 may also perform analysis using image data of a past moving image.

[0133] In addition, the detailed configuration and operation of each device constituting the image recognition system 100 may be modified as appropriate without departing from the spirit of the present invention. [Explanation of symbols]

[0134] 1. Image Recognition System 10. Imaging device 20 Information processing equipment 21 control unit (image acquisition unit, skeleton detection unit, entry determination unit, handrail use determination unit, human entry determination unit, hand entry determination unit, warning control unit, display control unit, count unit, recording control unit, attribute determination unit, image identification unit) 22 Memory section 23 Communications Department 24 Control section 25 Display section 30 Alarm device 50 Captured images 51 Person entry area (first area) 51a 1st person entry area 51b 2nd person entry area 52 Hand entry area (second area) 60 Target Person 61 moves 70,71,72 analysis points

Claims

1. an image acquisition unit that acquires an image of a target person captured by an imaging device; a skeleton detection unit that detects a skeleton position of the target person based on the image acquired by the image acquisition unit; an intrusion determination unit that determines whether or not the target person is present in a predetermined area of ​​the image based on the skeleton position detected by the skeleton detection unit; a handrail use determination unit that determines whether or not the target person is using a handrail based on the determination result by the entry determination unit; An image recognition system comprising:

2. The entry determination unit a human intrusion determination unit that determines whether or not the target person is present in the first area of ​​the image; The image recognition system according to claim 1 , further comprising: a hand entry determination unit that determines whether or not the target person has a hand in the second region of the image.

3. The image recognition system according to claim 2 , wherein the entry determination unit displays the first area and the second area on the display unit in different display modes so as to be superimposed on the image.

4. The image recognition system according to claim 2 , wherein the entry determination unit displays the position of the hand in the image on a display unit by displaying the skeletal position or by displaying a bounding box.

5. The image recognition system of claim 2, wherein the handrail usage determination unit determines that the handrail is being used when the human entry determination unit determines that the target person is present in the first area and the hand entry determination unit determines that a hand is present in the second area.

6. The image recognition system according to claim 1 , further comprising a warning control unit that issues a warning in accordance with a determination result by the handrail use determination unit.

7. The image recognition system according to claim 6 , wherein the warning control unit issues the warning to a terminal device used by a manager of a facility in which the handrail is installed.

8. The image recognition system according to claim 1 , further comprising a display control unit that displays on a display unit according to a determination result made by the handrail use determination unit.

9. The image recognition system according to claim 1 , further comprising a counting unit that counts the number of people who used the handrail and the number of people who did not use the handrail according to a determination result by the handrail use determination unit.

10. The image recognition system according to claim 9 , wherein the counting unit performs personal authentication of the target person and counts the number of times the handrail is used and / or the number of times the handrail is not used for each authenticated individual.

11. The image recognition system according to claim 9 , wherein the counting unit assigns an identification number to each of the plurality of target persons by object detection, and simultaneously counts the number of people who used the handrail and the number of people who did not use the handrail.

12. The image recognition system according to claim 1 , wherein the handrail use determination unit determines that the handrail is being used when the state in which the handrail is being used continues for a predetermined time.

13. The image recognition system according to claim 12 , wherein the handrail use determination unit receives settings of parameters related to the determination of whether or not the handrail is being used, the parameters including the predetermined time period.

14. The image recognition system according to claim 1 , further comprising a recording control unit that stores the image in association with a determination result made by the handrail use determination unit.

15. The image recognition system according to claim 1 , wherein the entry determination unit receives a setting of the range of the predetermined area.

16. The image recognition system according to claim 1 , further comprising an attribute determination unit that determines at least one of an age, a sex, and a clothing of the target person.

17. 2. The image recognition system according to claim 1, wherein the entry determination unit determines the direction of travel of the target person in the specified area based on at least one of the body orientation of the target person based on the skeletal position, time series data of the detection time of the target person in the specified area, and tracking of the target person.

18. The image recognition system according to claim 1 , further comprising a warning control unit that changes the content of the warning depending on the determination result by the handrail use determination unit and the moving speed of the target person in the predetermined area.

19. The image recognition system according to claim 18 , wherein the warning control unit issues a warning when the handrail use determination unit determines that the handrail is being used and the moving speed is faster than a predetermined threshold.

20. The image recognition system according to claim 18 , wherein the warning control unit changes the display content of the display unit according to the determination result by the handrail use determination unit and the moving speed of the target person in the predetermined area.

21. an image acquisition step of acquiring an image of a target person captured by an imaging device; a skeleton detection step of detecting a skeleton position of the target person based on the image acquired in the image acquisition step; an intrusion determination step of determining whether or not the target person is present in a predetermined area of ​​the image based on the skeleton position detected in the skeleton detection step; a handrail use determination unit step of determining whether or not the target person is using a handrail based on the determination result in the entry determination step; An image recognition method comprising:

22. The computer for the image recognition system an image acquisition unit that acquires an image of a target person captured by an imaging device; a skeleton detection unit that detects a skeleton position of the target person based on the image acquired by the image acquisition unit; an intrusion determination unit that determines whether or not the target person is present in a predetermined area of ​​the image based on the skeleton position detected by the skeleton detection unit; a handrail use determination unit that determines whether or not the target person is using a handrail based on the determination result by the entry determination unit; A program that functions as a

23. an image acquisition unit that acquires an image of a target person captured by an imaging device; a skeleton detection unit that detects a skeleton position of the target person based on the image acquired by the image acquisition unit; an image identification unit that identifies a handrail portion based on the image acquired by the image acquisition unit; a handrail use determination unit that determines whether or not the target person is using a handrail based on the position of the handrail unit identified by the image identification unit and the skeleton position detected by the skeleton detection unit; and An image recognition system comprising:

24. The image recognition system of claim 23, wherein the handrail usage determination unit determines whether or not the target person is using a handrail based on the positional relationship between the position of at least one of the back of the hand, the palm, or the arm of the target person as the skeletal position and the position of the handrail unit.

25. The image recognition system of claim 23, wherein the handrail usage determination unit determines that the target person is not using a handrail based on the position of the handrail portion and the skeletal position, and determines that an abnormality has occurred if it determines that the distance between the position of a predetermined part of the target person and the position of the handrail portion is less than a predetermined distance.

26. The image recognition system of claim 23, wherein the handrail usage determination unit determines that the target person is not using a handrail based on the position of the handrail portion and the skeletal position, and does not determine that an abnormality exists if it determines that the distance between the position of a predetermined part of the target person and the position of the handrail portion is greater than a predetermined distance.

27. an image acquisition step of acquiring an image of a target person captured by an imaging device; a skeleton detection step of detecting a skeleton position of the target person based on the image acquired in the image acquisition step; an image identification step of identifying a handrail portion based on the image acquired in the image acquisition step; a handrail use determination step of determining whether or not the target person is using a handrail based on the position of the handrail portion identified in the image identification step and the skeleton position detected in the skeleton detection step; An image recognition method comprising:

28. The computer for the image recognition system an image acquisition unit that acquires an image of a target person captured by an imaging device; a skeleton detection unit that detects a skeleton position of the target person based on the image acquired by the image acquisition unit; an image identification unit that identifies a handrail portion based on the image acquired by the image acquisition unit; a handrail use determination unit that determines whether or not the target person is using a handrail based on the position of the handrail unit identified by the image identification unit and the skeleton position detected by the skeleton detection unit; A program that functions as a

Images