Gate system, gate device control method, and gate device control program

Abstract

To provide a gate system capable of allowing an entering person to appropriately visually confirm a matching result.SOLUTION: The gate system includes: an imaging unit capable of picking up images of a person moving in the first direction in a given area of a passage and a person moving in a second direction opposite to the first direction; a display unit that has a screen capable of being seen by the person moving in the first direction; and a processor. The processor detects the movement direction of each person. The imaging unit picks up images of faces of people moving through the passages. The display unit displays matching results of the face images of people moving in the first direction but does not display the matching results of face images of people moving in the second direction.SELECTED DRAWING: Figure 1

Description

[[Technical Field]]

[0001] The present disclosure relates to a gate system, a control method for a gate device, and a control program for a gate device. [[Background Art]]

[0002] Patent Document 1 discloses a human monitoring system in which a plurality of imaging devices are installed within a predetermined area, collation is performed on a plurality of subjects existing within a monitorable area where each imaging device can capture images, and the collation result is displayed to an administrator in real time. [[Prior Art Literature]] [[Patent Literature]]

[0003] [[Patent Document 1]] Japanese Unexamined Patent Publication No. 2008-85874 [[Summary of the Invention]] [[Problem to be Solved by the Invention]]

[0004] For example, it is conceivable to install, on the entrance side of a facility or the like, an imaging device capable of capturing images of passers entering the facility or the like and a display device, capture an image of the face of the passer with the imaging device, perform face collation on the captured image, and display the collation result on the display device. However, since the imaging device can also capture images of passers exiting from the facility or the like, in the conventional technology, there is a risk that the collation result of the exiting passer (for example, an error message such as unauthorized passage or the back view of the passer) may also be displayed on the display device. This display is useless in the operation of the collation system, and causes unnecessary confusion for entering passers. Therefore, it is desired to determine the traveling direction of the passer so that the collation result of the passer can be appropriately displayed on the display device.

[0005] An object of the present disclosure is to provide a technique that allows an entering person to appropriately visually recognize a collation result. [[Means for Solving the Problem]]

[0006] One aspect of the present disclosure comprises a photographing device capable of photographing a person moving in a first direction and a person moving in a second direction opposite to the first direction within a predetermined area of ​​a passageway, a display device having a screen on which the person moving in the first direction is visible, and a processor, wherein the processor detects the direction of movement of each person, and the photographing device captures a facial image of the person moving in the passageway. The display device provides a gate system that displays the matching result of a person's face image moving in the first direction, but does not display the matching result of a person's face image moving in the second direction.

[0007] One aspect of the present disclosure provides a control method for a gate device comprising: a shooting device capable of shooting a person moving in a first direction and a person moving in a second direction opposite to the first direction within a predetermined area of ​​a passageway; and a display device having a screen on which a person moving in the first direction can be seen, wherein the gate device detects the direction of movement of each person, the shooting device captures a facial image of a person moving in the passageway, and the display device displays the matching result of the facial image of a person moving in the first direction but does not display the matching result of the facial image of a person moving in the second direction.

[0008] One aspect of the present disclosure is a control program for a gate device comprising: a camera capable of photographing a person moving in a first direction and a person moving in a second direction opposite to the first direction within a predetermined area of ​​a passageway; and a display device having a screen on which the person moving in the first direction can be seen. The present invention provides a control program for a gate device that causes the gate device to detect the direction of movement of each person, to cause the imaging device to capture a facial image of a person moving along the passage, to display the matching result of the facial image of a person moving in the first direction on the display device, and not to display the matching result of the facial image of a person moving in the second direction.

[0009] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0010] According to this disclosure, the verification results can be appropriately viewed by the person entering the premises. [Brief explanation of the drawing]

[0011] [Figure 1] A diagram illustrating the outline of the gate system according to Embodiment 1. [Figure 2] This figure shows an example of the display of the matching result in the first display device or the second display device according to Embodiment 1. [Figure 3] Block diagram showing an example configuration of the gate system according to Embodiment 1. [Figure 4] Diagram illustrating the facial and body images according to Embodiment 1. [Figure 5] A flowchart showing an example of the detection process performed by the first edge device according to Embodiment 1. [Figure 6] Example of the configuration of person information according to Embodiment 1 [Figure 7] A flowchart showing an example of the overall processing performed by the management device according to Embodiment 1. [Figure 8] Figure 7 shows a flowchart illustrating an example of face detection information processing. [Figure 9] Figure 7 shows a flowchart illustrating an example of body detection information processing. [Figure 10] Figure 7 shows a flowchart illustrating an example of post-transfer processing. [Figure 11] Figure 7 shows a flowchart illustrating an example of facial recognition result information processing. [Figure 12] A flowchart showing an example of the display processing performed by the first edge device according to Embodiment 1. [Figure 13]Diagram showing a passage determination zone and a matching zone in a detection region according to Embodiment 1 [Figure 14] Diagram for explaining an example of a display pattern in the first display device according to Embodiment 1 [Figure 15A] Side view for explaining a method of detecting, from a person information table, person information estimated to belong to the same person as the person indicated by body detection information according to Embodiment 1 [Figure 15B] Front view for explaining a method of detecting, from a person information table, person information estimated to belong to the same person as the person indicated by body detection information according to Embodiment 1 [Figure 16] Diagram for explaining a case where a person walks in line immediately behind a person walking forward according to Embodiment 1 [Figure 17] Diagram for explaining a method of detecting the moving direction of a person according to Embodiment 1 [Figure 18] Diagram for explaining a case where there is a person moving backward in a first direction according to Embodiment 2 [Figure 19] Flowchart showing an example of face matching result information processing according to Embodiment 2 [Figure 20] Diagram showing an example where an adult holds a child according to Embodiment 3 [Figure 21] Configuration example of person information according to Embodiment 3 [Figure 22] Flowchart showing an example of face detection information processing according to Embodiment 3 [Figure 23] Flowchart showing a first example of face matching result information processing according to Embodiment 3 [Figure 24] Flowchart showing a second example of face matching result information processing according to Embodiment 3 [Figure 25] Diagram showing an example where a person's face is printed on the clothes worn by a person according to Embodiment 4 [Figure 26] Configuration example of person information according to Embodiment 4 [Figure 27] Flowchart showing an example of face detection information processing according to Embodiment 4 [Figure 28]A flowchart showing an example of face matching result information processing according to Embodiment 4. [Figure 29] A diagram showing a first modified example of the gate system relating to this disclosure. [Figure 30] A diagram showing a second modified example of the gate system relating to this disclosure. [Figure 31] A diagram showing a third modified example of the gate system relating to this disclosure. [Figure 32] A diagram showing an example of the computer hardware configuration related to this disclosure. [Modes for carrying out the invention]

[0012] Embodiments of the present disclosure will be described in detail below, with appropriate reference to the drawings. However, unnecessary details may be omitted. For example, detailed explanations of already well-known matters and redundant explanations of substantially identical configurations may be omitted. This is to avoid the following explanation becoming unnecessarily verbose and to facilitate understanding by those skilled in the art. The accompanying drawings and the following explanation are provided to enable those skilled in the art to fully understand the present disclosure and are not intended to limit the subject matter of the claims.

[0013] (Embodiment 1) <Overview of the Gate System> Figure 1 is a diagram illustrating the overview of the gate system 9 according to Embodiment 1. Figure 2 is a diagram showing an example of the display of the matching result in the first display device 31 or the second display device 32 according to Embodiment 1.

[0014] The gate system 9 is a system that matches the facial image of person 1 and determines whether or not person 1 has the right to pass through the gate. The gate system 9 includes a first gate 11 and a second gate 12. The first gate 11 and the second gate 12 are installed at a predetermined distance apart in a passage 2 through which person 1 moves. In this embodiment, the direction from the second gate 12 toward the first gate 11 is defined as the first direction, and the direction from the first gate 11 toward the second gate 12 is defined as the second direction. In this embodiment, the area between the first gate 11 and the second gate 12 in the passage 2 is referred to as the person detection area 3. Note that the gate system 9 does not necessarily need to have a first gate 11 and a second gate 12 through which a person passes; it may also have a configuration as shown in Figure 31, which will be described later. In this case, the gate system 9 may set a virtual first line instead of the first gate 11, and a virtual second line instead of the second gate 12, and the area between the first line and the second line may be defined as the detection area 3. Furthermore, the direction from the first line to the second line may be referred to as the first direction, and the direction from the second line to the first line may be referred to as the second direction. Therefore, in the following explanation, the first gate 11 may be read as the first line, and the second gate 12 may be read as the second line.

[0015] Furthermore, in this embodiment, for the sake of explanation, the axis along the first or second direction will be described as the X-axis, the axis perpendicular to the X-axis and extending in the height direction will be described as the Z-axis, and the axis perpendicular to the X-axis and Z-axis will be described as the Y-axis.

[0016] The first gate 11 includes a first imaging device 21 capable of imaging at least a portion of the detection area 3, and a first display device 31 having a screen in the direction of the second gate 12. In Figure 1, the first gate 11 is shown to have two first imaging devices 21, but the first gate 11 may have one first imaging device 21, or three or more first imaging devices 21. Also, in Figure 1, the first gate 11 is shown to have one first display device 31, but the first gate 11 may have two or more first display devices 31.

[0017] The second gate 12 includes a second imaging device 22 capable of imaging at least a portion of the detection area 3, and a second display device 32 having a screen in the direction of the first gate 11. In Figure 1, the second gate 12 is shown to have two second imaging devices 22, but the second gate 12 may have one second imaging device 22, or three or more second imaging devices 22. Also, in Figure 1, the second gate 12 is shown to have one second display device 32, but the second gate 12 may have two or more second display devices 32.

[0018] In this embodiment, the first direction is described as the direction of entry into a facility, etc., where one has the right to enter and exit, and the second direction is described as the direction of exit from said facility, etc. However, the first direction may be the direction of exit from the facility, etc., and the second direction may be the direction of entry into the facility, etc.

[0019] The gate system 9 compares the face image of person 1 moving in a first direction, captured by the first imaging device 21 of the first gate 11, with each of the pre-registered matching face images, and displays the matching result on the first display device 31, as illustrated in Figure 2.

[0020] The gate system 9 compares the face image of person 1 moving in the second direction, captured by the second imaging device 22 of the second gate 12, with each of the pre-registered matching face images, and displays the matching result on the second display device 32, as illustrated in Figure 2.

[0021] The gate widths of the first gate 11 and the second gate 12, or the width of the detection area 3 in the Y direction, may be wide enough for multiple people 1 to pass through side by side. As a result, the gate system 9 according to this embodiment can allow many people 1 to enter and exit smoothly compared to a gate system with a gate width that allows only one person to pass through. For example, in the case of a gate system with a gate width for one person, if the matching of the person 1 in front fails, many people 1 following behind will be held up. However, in the case of a gate system 9 with a gate width that allows multiple people 1 to pass through side by side, as in this embodiment, even if the matching of the person 1 in front fails, many people 1 following behind can simply move around the person 1 that failed to be matched in front, and thus can pass through the gate smoothly without being held up. In other words, the gate system 9 according to this embodiment can suppress congestion when entering and exiting facilities, etc. The gate system 9 according to this embodiment will be described in detail below.

[0022] <Gate System Configuration> Figure 3 is a block diagram showing an example configuration of the gate system 9 according to Embodiment 1. Figure 4 is a diagram illustrating the face image and body image according to Embodiment 1.

[0023] The gate system 9 comprises a first gate 11 and a second gate 12. The first gate 11 comprises a first imaging device 21 and a first display device 31. The second gate 12 comprises a second imaging device 22 and a second display device 32. The first gate 11, the first imaging device 21, the first display device 31, the second gate 12, the second imaging device 22, and the second display device 32 may be collectively referred to as the gate device 10.

[0024] The gate system 9 further comprises a first edge device 41, a second edge device 42, a management device 50, and a face recognition device 60. The first imaging device 21 is connected to the first edge device 41 via an electrical cable (e.g., a USB cable or an Ethernet® cable). The first display device 31 is connected to the first edge device 41 via an electrical cable (e.g., an HDMI® cable). The second imaging device 22 is connected to the second edge device 42 via an electrical cable (e.g., a USB cable or an Ethernet cable). The second display device 32 is connected to the second edge device 42 via an electrical cable (e.g., an HDMI® cable). The first edge device 41 and the second edge device 42 are connected to the management device 50 via a communication network 4. The management device 50 is connected to the face recognition device 60 via the communication network 4. Examples of the communication network 4 include wired LAN, wireless LAN, mobile communication network (e.g., LTE, 4G, 5G), the Internet, and / or VPN.

[0025] The first imaging device 21 captures the detection area 3, generates an image, and transmits the image to the first edge device 41. The second imaging device 22 captures the detection area 3, generates an image, and transmits the image to the second edge device 42. The captured image may be a still image or a moving image.

[0026] As shown in Figures 4(a) and 4(b), the first edge device 41 detects the entire body of person 1 from the captured image received from the first imaging device 21. In this embodiment, the image of the entire body of person 1 detected from the captured image is referred to as the body image 200. The first edge device 41 transmits the detected body image 200 to the management device 50. The first edge device 41 also detects the face portion of person 1 from the captured image received from the first imaging device 21, as shown in Figure 4(a). In this embodiment, the image of the face portion of person 1 detected from the captured image is referred to as the face image 220. The first edge device 41 transmits the detected face image 220 to the management device 50.

[0027] As shown in Figures 4(a) and 4(b), the second edge device 42 detects the entire body of person 1 from the captured image received from the second imaging device 22. The second edge device 42 transmits the body image 200 detected from the captured image to the management device 50. Also, as shown in Figure 4(a), the second edge device 42 detects the face portion of person 1 from the captured image received from the second imaging device 22. The second edge device 42 transmits the face image 220 detected from the captured image to the management device 50. Details of the processing performed by the first edge device 41 and the second edge device 42 will be described later.

[0028] The management device 50 generates person information 100 (see Figure 6) corresponding to person 1 located in the detection area 3, based on the body image 200 or face image 220 received from the first edge device 41, or the body image 200 or face image 220 received from the second edge device 42. The management device 50 may generate person information 100 for each person 1 detected in the detection area 3. Details of the person information 100 will be described later (see Figure 6). The management device 50 transmits a face matching request including the face image 220 to the face matching device 60.

[0029] The facial recognition device 60 pre-stores a facial image for verification of person 1 who has the right to pass through the gate (e.g., enter or exit). For example, the facial recognition device 60 stores a facial image for verification of person 1 who has previously registered to enter the facility. The facial recognition device 60 may also manage the facial image for verification by linking it with the name, age, gender, and credit card information of person 1.

[0030] Furthermore, the face matching device 60 compares the face image 220 received from the management device 50 with each matching face image. If the face matching device 60 successfully matches with any of the registered matching face images, it generates face matching result information including the successful match and transmits it to the management device 50. At this time, the face matching device 60 may perform billing in real time using the credit card information associated with the successfully matched matching face image. If the face matching device 60 fails to match with any of the registered matching face images, it generates face matching result information including the failed match and transmits it to the management device 50. Note that the matching of the face image 220 with the matching face images may be achieved by known face image matching technology. Also, "matching" may be read as "authentication".

[0031] The management device 50 receives face matching result information from the face matching device 60. The management device 50 transmits person matching information, including face matching result information for the face image 220 of person 1 moving in the first direction, to the first edge device 41, and transmits person matching information, including face matching result information for the face image 220 of person 1 moving in the second direction, to the second edge device 42.

[0032] When the first edge device 41 receives person matching information from the management device 50, it displays the face matching result information included in the person matching information on the first display device 31, as shown in Figure 2. As a result, a person 1 located in the detection area 3 and moving in the first direction can see the screen of the first display device 31 located ahead of their movement direction and find out their matching result. In other words, the first edge device 41 allows a person 1 moving in the first direction to visually confirm their matching result.

[0033] When the second edge device 42 receives person matching information from the management device 50, it displays the face matching result information included in the person matching information on the second display device 32, as shown in Figure 2. As a result, person 1 located in the detection area 3 and moving in the second direction can see the screen of the second display device 32 located ahead of their direction of movement and find out their matching result. In other words, the second edge device 42 allows person 1 moving in the second direction to visually confirm their matching result. The processing performed by the first edge device 41, the second edge device 42, and the management device 50 will be described in detail below.

[0034] <Detection process in the first edge device> Figure 5 is a flowchart showing an example of the detection process performed by the first edge device 41 according to Embodiment 1. Note that for the detection process performed by the second edge device 42 according to Embodiment 1, in the following explanation referring to Figure 5, "first imaging device 21" should be read as "second imaging device 22".

[0035] The first edge device 41 receives the captured image from the first imaging device 21 (S101).

[0036] The first edge device 41 receives a distance image from the first imaging device 21 (S102). In this way, the first imaging device 21 may generate a distance image indicating the distance to a person 1, etc., located in the detection area 3. The second imaging device 22 does the same. The generation of the distance image may be achieved using any known technology, such as a stereo camera or a distance measuring sensor (e.g., LiDAR (Light Detection And Ranging)). In this embodiment, the processing of step S102 is performed after the processing of step S101, but this is not limited to this, and the order of the steps may be reversed.

[0037] As shown in Figure 4(a), the first edge device 41 detects the face image 220 of person 1 from the captured image (S103). The detection of the person's face image 220 can be achieved using any known technique, such as pattern matching or AI (Artificial Intelligence). The AI ​​may also be an image recognition technique using deep learning (for example, CNN (Convolutional Neural Network)).

[0038] The first edge device 41 determines in step S103 whether or not it was able to detect the face image 220 from the captured image (S104). If the face image 220 could not be detected (S104: NO), the first edge device 41 proceeds to step S108.

[0039] If a face image 220 is detected (S104: YES), the first edge device 41 uses the depth image to calculate the three-dimensional coordinates of the detected face image 220 (hereinafter referred to as face coordinates 221 (see Figure 4)) (S105). For example, as shown in Figure 4(a), the first edge device 41 calculates the center point of the face image 220 as the face coordinates 221.

[0040] The first edge device 41 generates face detection information (S106). For example, the first edge device 41 generates face detection information that includes an ID for identifying the first imaging device 21 which is the source of the captured image (hereinafter referred to as the imaging device ID), a face image 220, the size of the face image 220 (hereinafter referred to as the face image size), face coordinates 221, and the time the captured image was taken.

[0041] The first edge device 41 transmits the generated face detection information to the management device 50 (S107), and proceeds to the next step S108.

[0042] The first edge device 41 detects the body image 200 of person 1 from the captured image, as shown in Figure 4(a) or Figure 4(b) (S108). The detection of the body image 200 of person 1 may be achieved using any known technique, such as pattern matching or AI. The AI ​​may also be an image recognition technique using deep learning (e.g., CNN).

[0043] The first edge device 41 determines in step S108 whether or not it was able to detect the body image 200 from the captured image (S109). If the body image 200 could not be detected (S109: NO), the first edge device 41 returns to step S101.

[0044] If a body image 200 is detected (S109: YES), the first edge device 41 uses the depth image to calculate the three-dimensional coordinates of the detected body image 200 (hereinafter referred to as body coordinates 201 (see Figure 4)) (S110). For example, as shown in Figures 4(a) and 4(b), the first edge device 41 calculates the center point of the body image 200 as the body coordinates 201.

[0045] The first edge device 41 generates body detection information (S111). For example, the first edge device 41 generates body detection information that includes the imaging device ID, a body ID for identifying the body image 200, the body image 200, body coordinates 201, and the time the image was captured.

[0046] The first edge device 41 transmits the generated body detection information to the management device 50 (S112), and the process returns to step S101. In this embodiment, the process of detecting a face image from the captured image (S103-S107) is performed first, followed by the process of detecting a body image from the captured image (S108-S112). However, the process is not limited to this, and the order of the steps may be reversed.

[0047] In this way, by having the first edge device 41 or the second edge device 42 perform the process of detecting the face image 220 and / or body image 200 of person 1 from the captured image and the process of calculating the face coordinates 221 and / or body coordinates 201, the processing load can be distributed compared to when these processes are performed in the management device 50. If the distribution of processing load is not considered, the gate system 9 does not need to have the first edge device 41 and the second edge device 42. In this case, the first shooting device 21, the first display device 31, the second shooting device 22, and the second display device 32 may be directly connected to the management device 50. The management device 50 may then perform the process of detecting the face image 220 and / or body image 200 of person 1 from the captured image and the process of calculating the face coordinates 221 and / or body coordinates 201.

[0048] <Personal information> Figure 6 shows an example of the configuration of person information 100 according to Embodiment 1.

[0049] Person information 100 is generated for each person detected in the detection area 3. Person information 100 includes, as information or data, a tracking ID 101, the time of initial image capture 102, the time of image update 103, a body image 104, the time of body image update 105, body coordinates 106, a face image 107, the time of face image update 108, face coordinates 109, a face matching ID 110, matching status 111, and a direction of movement 112.

[0050] Tracking ID 101 is assigned an ID used to track the same person in detection area 3. Tracking ID 101 can also be described as an ID used to identify the person's information 100.

[0051] The initial image capture time 102 is set to the time when the body image 104 or face image 107 included in the person information 100 was first captured. The initial image capture time can also be said to be the time when the person information 100 was first generated.

[0052] The image update time 103 is set to the time when the body image 104 or face image 107 included in the person information 100 was last updated. The image update time 103 can also be said to be the time when the person information 100 was last updated.

[0053] Body image 104 is set to the body image 200 detected from the captured image of person 1 indicated by the person information 100. Body image 104 may be updated as needed with the body image 200 detected from the most recent captured image.

[0054] The body image update time 105 is set to the time when the body image 104 included in the person information 100 was last updated.

[0055] Body coordinates 106 are set to body coordinates 201 calculated based on body image 104. Body coordinates 106 may be expressed as three-dimensional coordinates (X coordinate, Y coordinate, Z coordinate) in the detection region 3.

[0056] The face image 107 is set to the face image 220 detected from the photograph of person 1 indicated by the person information 100. The face image 107 may be updated as needed with the face image 220 detected from the most recent photograph.

[0057] The face image update time 108 is set to the time when the face image 107 included in the person information 100 was last updated.

[0058] The face coordinate 109 is set to the face coordinate 221 calculated based on the face image 107. The face coordinate 109 may be expressed as a three-dimensional coordinate (X coordinate, Y coordinate, Z coordinate) in the detection region 3.

[0059] The face matching ID 110 is set to the ID assigned to the matching face image that was successfully matched. In other words, the face matching ID 110 can be said to be the ID that proves that person 1 corresponds to face image 107. If no matching face image was found that was successfully matched (i.e., face matching failed), the face matching ID 110 may be set to NULL.

[0060] The matching status 111 is set to indicate the status of the matching against the face image 107. For example, the matching status 111 can be set to one of the following: "Before matching request", "Matching in progress", "Matching successful", or "Matching failed".

[0061] The movement direction 112 is set to the movement direction of person 1 indicated by the person information 100. For example, the movement direction may be set to either the first direction or the second direction.

[0062] The management device 50 manages multiple pieces of person information 100 as a person information table (not shown). The person information table may be stored in the memory 1002 or storage 1003 (see Figure 32) of the management device 50, or it may be stored in a device other than the management device 50.

[0063] <Overall processing in the control device> Figure 7 is a flowchart showing an example of the overall processing performed by the management device 50 according to Embodiment 1.

[0064] The management device 50 determines whether or not it has received face detection information (see step S107 in Figure 5) from the first edge device 41 or the second edge device 42 (S201).

[0065] When the management device 50 receives face detection information (S201:YES), it performs face detection information processing (S202). Details of the face detection information processing will be described later (see Figure 8).

[0066] If the management device 50 has not received face detection information (S202: NO), it determines whether or not it has received body detection information (see step S112 in Figure 5) from the first edge device 41 or the second edge device 42 (S203).

[0067] When the management device 50 receives body detection information (S203:YES), it executes body detection information processing (S204). Details of the body detection information processing will be described later (see Figure 9).

[0068] If the management device 50 has not received body detection information (S203: NO), it determines whether or not it has received face matching result information (S205). Face matching result information is information returned by the face matching device 60 in response to a face matching request sent from the management device 50 to the face matching device 60 in Figure 8, which will be described later, and includes the result of face matching (for example, matching success or matching failure).

[0069] When the management device 50 receives face matching result information (S205:YES), it performs face matching result information processing (S206). Details of the face matching result information processing will be described later (see Figure 11).

[0070] If the control device 50 has not received face matching result information (S205: NO), it returns to step S201.

[0071] The management device 50 performs face detection information processing (S202) or body detection information processing (S204), then performs post-movement processing (S210), and returns the process to step S201. Details of the post-movement processing will be described later (see Figure 10). In this embodiment, steps S201, S203, and S205 are performed sequentially, but the system is not limited to this, and the processes in steps S201, S203, and S205 may be rearranged or performed simultaneously in parallel.

[0072] <Face detection information processing in the control device> Figure 8 is a flowchart showing an example of the face detection information processing (S202) shown in Figure 7.

[0073] The management device 50 detects person information 100 corresponding to the face detection information received from the first edge device 41 or the second edge device 42 from the person information table (S301). In this embodiment, the person information 100 corresponding to the face detection information is person information 100 that is presumed to belong to the same person as person 1 indicated by the face detection information. Details of the method for detecting person information 100 that is presumed to belong to the same person as person 1 indicated by the face detection information from the person information table will be described later (see Figure 15).

[0074] The management device 50 determines whether or not it was able to detect person information 100 from the person information table in step S301 (S302).

[0075] Next, we will explain the case where it is determined in step S302 that person information 100 could not be found in the person information table (S302: NO).

[0076] The management device 50 generates new person information 100 based on the face detection information (S303). At this time, the management device 50 sets the following content in the new person information 100. • Set a new tracking ID for tracking ID 101 in person information 100. • Set face image 107 of person information 100 to face image 220 included in face detection information. • Set face coordinates 109 of person information 100 to face coordinates 221 included in face detection information. - Set the initial image capture time 102 of person information 100 to the capture time included in the earliest face detection information (i.e., the earliest captured time). - Set the matching status 111 of person information 100 to "Before matching request".

[0077] The management device 50 detects the direction of movement of person 1 corresponding to the face detection information, and sets the detected direction of movement to the direction of movement 112 of the person information 100 generated in step S303 (S304). Details of the method for detecting the direction of movement of a person will be described later (see Figure 16).

[0078] The management device 50 registers the person information 100 generated and set in steps S303 and S304 into the person information table (S305).

[0079] The management device 50 transmits a face matching request, including the face image 107 and tracking ID 101, to the face matching device 60 and updates the matching status 111 of the person information 100 to "matching" (S306). Then, the control device 50 completes the process.

[0080] Next, we will explain the case where it is determined in step S302 that person information 100 has been detected from the person information table (S302:YES).

[0081] The management device 50 updates the content of the person information 100 detected in step S301 based on the face detection information (S310). For example, the management device 50 updates the detected person information 100 as follows: • Update face coordinates 109 of person information 100 to face coordinates 221 included in face detection information. • Update the image update time 103 and face image update time 108 of person information 100 to the shooting time included in the face detection information.

[0082] The management device 50 determines whether the matching status 111 of the detected person information 100 is "before matching request" or "matching failed" (S311). If the matching status 111 is not "before matching request" or "matching failed" (S311: NO), the management device 50 completes this process.

[0083] If the matching status 111 is "before matching request" or "matching failed" (S311: YES), the management device 50 sends a face matching request including the face image 107 and tracking ID 101 to the face matching device 60 and updates the matching status 111 of the person information 100 to "matching in progress" (S312). Then, the management device 50 completes this process.

[0084] <Body detection information processing in control devices> Figure 9 is a flowchart showing an example of the body detection information processing (S204) shown in Figure 7.

[0085] The management device 50 detects person information 100 corresponding to the body detection information received from the first edge device 41 or the second edge device 42 from the person information table (S401). In this embodiment, the person information 100 corresponding to the body detection information is person information 100 that is presumed to belong to the same person as person 1 indicated by the body detection information. Details of the method for detecting person information 100 that is presumed to belong to the same person as person 1 indicated by the body detection information from the person information table will be described later (see Figure 15).

[0086] The management device 50 determines whether or not it was able to detect person information 100 from the person information table in step S401 (S402).

[0087] Next, we will explain the case where it is determined in step S402 that person information 100 could not be found in the person information table (S402:NO).

[0088] The management device 50 generates new person information 100 based on the received body detection information (S403). At this time, the management device 50 sets the following content in the new person information 100. • Set a new tracking ID for tracking ID 101 in person information 100. • Set the body image 104 of person information 100 to the body image 200 included in the body detection information. - Set the body coordinates 106 of person information 100 to the body coordinates 202 included in the body detection information. - Set the initial image capture time 102 of person information 100 to the capture time included in the earliest body detection information (i.e., the earliest captured time).

[0089] The management device 50 detects the direction of movement of person 1 corresponding to the body detection information, and sets the detected direction of movement to the direction of movement 112 of the person information 100 generated in step S403 (S404). Details of the method for detecting the direction of movement of person 1 will be described later (see Figure 16).

[0090] The management device 50 registers the person information 100 generated and set in steps S403 and S404 into the person information table (S405). Then, the management device 50 completes this process.

[0091] Next, we will explain the case where it is determined in step S402 that person information 100 has been detected from the person information table (S402:YES).

[0092] The management device 50 updates the content of the person information 100 detected in step S401 based on the body detection information (S410). For example, the management device 50 updates the detected person information 100 as follows: • Update the body coordinates 106 of person information 100 to the body coordinates 202 included in the body detection information. • Update the image update time 103 and body image update time 105 of person information 100 to the body capture time included in the body detection information.

[0093] Then, the control device 50 completes the process.

[0094] <Post-transfer processing in the management device> Figure 10 is a flowchart showing an example of the post-processing (S210) shown in Figure 7.

[0095] The management device 50 generates person movement information based on the target person information (S501). In this embodiment, the target person information is the person information 100 detected from the person information table or newly generated person information 100 in the face detection information processing shown in Figure 8, or the person information 100 detected from the person information table or newly generated person information 100 in the body detection information processing shown in Figure 9. At this time, the management device 50 sets the following content in the person movement information. Tracking ID • Zone information • Face image (however, if a face image is unavailable, a body image) Matching status • Facial recognition ID

[0096] Here, zone information indicates whether person 1, corresponding to the target person information, is located in the matching zone 301 or the passage determination zone 302 within the detection area 3. Details of the matching zone 301 and the passage determination zone 302 will be described later (see Figure 13).

[0097] The management device 50 determines whether the direction of movement of the target person information is in the first direction or the second direction (S502).

[0098] If the direction of movement is the first direction (S502: first direction), the management device 50 transmits the person movement information generated in step S501 to the first edge device 41 (S503), and completes this process.

[0099] If the direction of movement is the second direction (S502: second direction), the management device 50 transmits the person movement information generated in step S501 to the second edge device 42 (S504), and completes this process.

[0100] <Processing of facial recognition result information in control devices> Figure 11 is a flowchart showing an example of the face matching result information processing (S206) shown in Figure 7.

[0101] The management device 50 sets the content of the face matching result information received from the face matching device 60 into the person information 100 in the person information table that corresponds to the face matching result information (S601). For example, the management device 50 sets the content of the face matching result information into the person information 100 as follows. • Set the face matching ID of the face matching result information to face matching ID 110 of person information 100. • Set the matching result (matching success or matching failure) of the facial matching result information to the matching status 111 of person information 100.

[0102] The management device 50 generates person matching information (S602) based on the person information 100 set in step S601. At this time, the management device 50 sets the following information as person identification information. Tracking ID • Zone information • Face image Matching status • Facial recognition ID

[0103] The management device 50 determines whether the movement direction 112 of the person information 100 set in step S601 is in the first direction or the second direction (S603).

[0104] If the movement direction 112 is the first direction (S602: first direction), the management device 50 transmits the person matching information generated in step S602 to the first edge device 41 (S604), and completes this process.

[0105] If the movement direction 112 is the second direction (S602: second direction), the management device 50 transmits the person matching information generated in step S602 to the second edge device 42 (S605), and completes this process.

[0106] <Display processing in the first edge device> Figure 12 is a flowchart showing an example of the display processing performed by the first edge device 41 according to Embodiment 1. Note that, in the following explanation referring to Figure 12, the display processing performed by the second edge device 42 according to Embodiment 1 can be described by substituting "first edge device 41" with "second edge device 42," "first display device 31" with "second display device 32," and "first direction" with "second direction."

[0107] The first edge device 41 determines whether it has received person movement information or person matching information from the management device 50 (S701). If the first edge device 41 has not received either person movement information or person matching information (S701: NO), it returns to step S701. If the first edge device 41 has received either person movement information or person matching information (S701: YES), it performs the following processing.

[0108] The first edge device 41 determines whether or not a face image or body image corresponding to a tracking ID included in the received person movement information or person matching information is displayed on the first display device 31 (S702).

[0109] If the face image or body image corresponding to the tracking ID is not displayed on the first display device 31 (S702: NO), the first edge device 41 newly displays the face image or body image and the matching status included in the person movement information or person matching information on the first display device 31 (S703). Then, the first edge device 41 returns to step S701.

[0110] If a face image or body image corresponding to the tracking ID is displayed on the first display device 31 (S702: YES), the first edge device 41 updates the displayed face image or body image and matching status to the face image or body image included in the person movement information or person matching information and displays it (S704). Then, the first edge device 41 returns to step S701.

[0111] As a result, the first display device 31 displays matching result images 70A, 70B, 70C, 70E, and 70F, which show the matching status (e.g., matching success or matching failure) of the face or body image of each person 1 moving in the first direction, as shown in Figure 2. Therefore, each person 1 moving in the first direction can see their matching result from the first display device 31 installed ahead in the direction of movement.

[0112] Similarly, as shown in Figure 2, the second display device 32 displays matching result images 70A, 70B, 70C, 70E, and 70F, which indicate the matching status (e.g., matching success or matching failure) of the face or body image of each person 1 moving in the second direction. Therefore, each person 1 moving in the second direction can see their matching result from the second display device 32 installed in the direction of their movement.

[0113] The display processing described above can be expressed as follows: The management device 50 detects the direction of movement of each person 1 moving between the first gate 11 and the second gate 12, displays the matching result of the face image 220 of person 1 moving in the first direction on the first display device 31 but not on the second display device 32, and displays the matching result of the face image 220 of person 1 moving in the second direction on the second display device 32 but not on the first display device 31.

[0114] As a result, the matching result for person 1 is displayed on the display device installed in the direction of person 1's movement, while the matching result for person 1 is not displayed on the display device installed in the opposite direction of person 1's movement. Therefore, the management device 50 can appropriately display the matching results to both person 1 moving in the first direction and person 1 moving in the second direction along the passage 2.

[0115] <Display Pattern> Figure 13 shows the matching zone 301 and the passage determination zone 302 in the detection area 3 according to Embodiment 1. In the following explanation, the first gate 11 and the first direction will be used as examples, but the same applies to the second gate 12 and the second direction.

[0116] The image captured by the first imaging device 21 at the first gate 11 is used for face matching of person 1 moving in the first direction. In the detection area 3, a predetermined area close to the second gate 12 is designated as the matching zone 301, and a predetermined area close to the first gate 11 is designated as the passage determination zone 302. The first edge device 41 may change the display pattern on the first display device 31 depending on whether person 1 is located in the matching zone 301 or the passage determination zone 302. Next, the display pattern will be described.

[0117] Figure 14 is a diagram illustrating an example of a display pattern in the first display device 31 according to Embodiment 1. For the display pattern in the second display device 32, in the following explanation referring to Figure 14, "first edge device 41" should be read as "second edge device 42" and "first display device 31" as "second display device 32".

[0118] If person 1 is located outside the detection area 3 (outside the zone), the first edge device 41 does not display the matching result image of person 1 on the first display device 31.

[0119] When person 1 is located in the matching zone 301, the first edge device 41 controls the display of the first display device 31 in the following patterns (A0) to (A3). (A0) If the first edge device 41 has a face image 220 of person 1 and the matching status 111 is "matching", it displays a matching result image on the first display device 31 that includes the face image 220 of person 1 and the message "matching". (A1) If the first edge device 41 finds that a face image 220 of person 1 exists and the matching status 111 is "matching successful", it displays a matching result image on the first display device 31 that includes the face image 220 of person 1 and the message "matching successful". At this time, the first edge device 41 may output a matching success sound from the speaker provided in the first display device 31. (A2) If the first edge device 41 finds that a face image 220 of person 1 exists and the matching status 111 is "matching failed", it displays a matching result image on the first display device 31 that includes the face image 220 of person 1 and the message "matching failed". At this time, the first edge device 41 may output a matching failure sound from the speaker provided in the first display device 31. (A3) If the first edge device 41 finds that there is no face image 220 of person 1 but there is a body image 200 of person 1, it displays a matching result image on the first display device 31 that includes the body image 200 of person 1 and the message "Warning". At this time, the first edge device 41 may output a warning sound from the speaker provided in the first display device 31.

[0120] If person 1 is located in the passage determination zone 302, the first edge device 41 controls the display of the first display device 31 as shown in the following patterns (A4) to (A7). (A4) If the first edge device 41 has a face image 220 of person 1 and the matching status 111 is "matching", it displays a matching result image on the first display device 31 that includes the face image 220 of person 1 and the message "matching". (A5) If the first edge device 41 finds that a face image 220 of person 1 exists and the matching status 111 is "matching successful", it displays a matching result image on the first display device 31 that includes the face image 220 of person 1 and the message "passed". At this time, the first edge device 41 may output a passing sound from the speaker provided in the first display device 31. (A6) If the first edge device 41 finds that a face image 220 of person 1 exists and the matching status 111 is "matching failed", it displays a matching result image on the first display device 31 that includes the face image 220 of person 1 and the message "illegal passage". At this time, the first edge device 41 may output an illegal passage sound from the speaker provided in the first display device 31. (A7) If the first edge device 41 does not have a face image 220 of person 1 but does have a body image 200, it displays a matching result image on the first display device 31 that includes the body image 200 of person 1 and the message "illegal passage". At this time, the first edge device 41 may output an illegal passage sound from the speaker provided in the first display device 31.

[0121] The display priority may be set as follows: The pass-through determination zone 302 has a higher display priority than the matching zone 301. The "Transiting" message has the highest display priority, followed by the "Matching Successful" message. • If there are 220 face images, the "Matching" message has the lowest display priority. • Images with 220 face images have a higher display priority than images with only 200 body images.

[0122] For example, the display priority of the patterns (A0) to (A7) described above may be set in the following order from lowest to highest: (A3)<(A7)<(A0)<(A4)<(A2)<(A6)<(A1)<(A5)

[0123] <Method for detecting information about the same person> Figures 15A and 15B illustrate a method for detecting person information 100, which is presumed to belong to the same person as person 1 indicated by the body detection information according to Embodiment 1, from a person information table. In Figure 15, the explanation assumes that person 1 is moving in a first direction. If person 1 is moving in a second direction, then in the explanation referring to Figure 15 below, replace "first imaging device 21" with "second imaging device 22" and "second imaging device 22" with "first imaging device 21".

[0124] The first imaging device 21 captures a body image 200 of person 1 moving in a first direction, showing the front view of person 1's body. Hereinafter, the X and Y coordinates of the body coordinates 201A calculated from the front view body image 200 will be referred to as the front view body X coordinate 201Ax and the Y coordinate 201Ay, respectively. In addition, the first imaging device 21 captures a face image 220 of person 1 moving in a first direction, showing the front view of person 1's face. Hereinafter, the X and Y coordinates of the face coordinates 221A calculated from the front view face image 220 will be referred to as the front view face X coordinate 221Ax and the Y coordinate 221Ay, respectively.

[0125] The second imaging device 22 captures a body image 200 of person 1 moving in the first direction, which shows the back of person 1's body. Hereinafter, the X and Y coordinates of the body coordinates 201B calculated from the back body image 200 will be referred to as the X coordinate 201Bx and Y coordinate 201By of the back of the body, respectively.

[0126] As shown in Figure 15A, the X-coordinate 201Ax on the front of the body and the X-coordinate 201Bx on the back of the body are separated by the thickness of Person 1's body in the front-to-back direction. Therefore, if the positional difference between the X-coordinate 201Ax on the front of the body and the X-coordinate 201Bx on the back of the body is smaller than a predetermined threshold W based on body thickness (hereinafter referred to as the body thickness threshold), then it can be estimated that the X-coordinate 201Ax on the front of the body and the X-coordinate 201Bx on the back of the body correspond to the X-coordinates of the same person. The body thickness threshold W may be slightly larger than the average body thickness. Similarly, if the positional difference between the X-coordinate 221Ax on the front of the face and the X-coordinate 201Bx on the back of the body is smaller than the body thickness threshold W, then it can be estimated that the X-coordinate 221Ax on the front of the face and the X-coordinate 201Bx on the back of the body correspond to the X-coordinates of the same person.

[0127] Furthermore, as shown in Figure 15B, if the change in the Y component between the Y coordinate 201Ay of body coordinate 201A included in the received body detection information (i.e., the current Y coordinate) and the Y coordinate 201Ay of body coordinate 201A included in the person information 100 (i.e., the past Y coordinate) is less than or equal to a predetermined threshold (hereinafter referred to as the body width threshold) W2, then it can be estimated that the current Y coordinate 201Ay and the past Y coordinate 201Ay belong to the same person. Note that face coordinate 221A may be used instead of body coordinate 201A.

[0128] In other words, if the management device 50 can detect a person information 100 from the person information table in which the positional difference between the Y coordinate of the body coordinate 201 (or face coordinate 221) included in the received body detection information and the Y coordinate of the body coordinate 201 (or face coordinate 221) included in the person information 100 is less than or equal to the body width threshold W2, and the positional difference between the X coordinate of the body coordinate 201 included in the received body detection information and the X coordinate of the body coordinate 201 or face coordinate 221 included in the person information 100 is less than the body thickness threshold W, then the management device 50 may estimate that the detected person information 100 belongs to the same person as person 1 indicated by the body detection information.

[0129] Similarly, if the management device 50 can detect a person information 100 from the person information table in which the positional difference between the Y coordinate of the body coordinate 201 (or face coordinate 221) included in the received body detection information and the Y coordinate of the body coordinate 201 (or face coordinate 221) included in the person information 100 is smaller than or equal to the body width threshold W2, and the positional difference between the X coordinate of the face coordinate 221 included in the received face detection information and the X coordinate of the body coordinate 201 or face coordinate 221 included in the person information 100 is smaller than the body thickness threshold W, then the management device 50 may estimate that the detected person information 100 belongs to the same person as person 1 indicated by the face detection information.

[0130] Furthermore, the conditions described above regarding the Y-coordinate position difference and the body width threshold W2, and the conditions regarding the X-coordinate position difference and the body thickness threshold W, are satisfied when the time difference between the face capture time included in the face detection information or the body capture time included in the body detection information and the face image update time 126 or body image update time 123 included in the person information 100 is sufficiently small (i.e., when the time difference is smaller than a predetermined time threshold T). This is because, since person 1 is moving, if the time difference is large, the person information 100 that satisfies the conditions described above regarding the Y-coordinate position difference and the body width threshold W2, and the X-coordinate position difference and the body thickness threshold W, is likely to belong to a different person who was in that position in the past.

[0131] In other words, in step S301 of Figure 8, the following processing may be performed. The management device 50 detects from the person information table person information 100 in which the time difference between the face capture time included in the face detection information and the face image update time 108 or body image update time 105 included in the person information 100 is less than a predetermined time threshold T, the position difference between the Y coordinate of face coordinate 221 included in the face detection information and the Y coordinate of face coordinate 109 included in the person information 100 is smaller than the body width threshold W2, and the position difference between the X coordinate of face coordinate 221 included in the face detection information and the X coordinate of face coordinate 109 or body coordinate 106 included in the person information 100 is smaller than the body thickness threshold W.

[0132] Furthermore, in step S401 of Figure 9, the following processing may be performed. The management device 50 detects from the person information table person information 100 in which the time difference between the body shooting time included in the body detection information and the face image update time 108 or body image update time 105 included in the person information 100 is less than a predetermined time threshold T, the position difference between the Y coordinate of the body coordinate 201 included in the body detection information and the Y coordinate of the body coordinate 106 included in the person information 100 is smaller than the body width threshold W2, and the position difference between the X coordinate of the body coordinate 201 included in the body detection information and the X coordinate of the face coordinate 109 or the X coordinate of the body coordinate 106 included in the person information 100 is smaller than the body thickness threshold W.

[0133] The above can be expressed as follows: The management device 50 estimates that the position of person 1 in the first image (e.g., body coordinate 201 or face coordinate 221) included in the first image (e.g., body image 200 or face image 220) captured by the first imaging device 21 and the position of person 1 in the second image (e.g., body image 200 or face image 220) captured by the second imaging device 22 are the same person if the difference is smaller than a predetermined threshold W based on the thickness of the person's body.

[0134] This allows the management device 50 to estimate, for example, whether the first image, which is a frontal view of person 1, and the second image, which is a rear view of person 1, are of the same person.

[0135] <If there is an accompanying person> Figure 16 is a diagram illustrating the case in which a person walking in front of a person walking ahead (hereinafter referred to as "forward person 1C") has a person walking immediately behind him (hereinafter referred to as "tailed person 1D"). In Figure 16, it is assumed that the forward person 1C and tailed person 1D are moving in a first direction. If the forward person 1C and tailed person 1D are moving in a second direction, then in the following explanation, "first imaging device 21" should be read as "second imaging device 22", "second imaging device 22" as "first imaging device 21", and "first edge device 41" as "second edge device 42".

[0136] The body image 200 captured by the first imaging device 21 shows the front view of the body of the person 1C in front. Therefore, the first edge device 41 generates body detection information (hereinafter referred to as the body detection information of person 1C in front) based on the body image 200 showing the front view of the body of person 1C in front, and transmits it to the management device 50.

[0137] The body image 200 captured by the second imaging device 22 shows the back of the body of the tailgating person 1D. Therefore, the second edge device 42 generates body detection information (hereinafter referred to as the body detection information of the tailgating person 1D) based on the body image 200 which shows the back of the body of the tailgating person 1D, and transmits it to the management device 50.

[0138] The positional difference between the X-coordinate 201Cx of body coordinate 201C included in the body detection information of person 1C in front and the X-coordinate 201Dx of body coordinate 201D included in the body detection information of accompanying person 1D is greater than the body thickness threshold W of one person. Therefore, the management device 50 can estimate that person 1C in front and accompanying person 1D are different people by the method of detecting the person information 100 of the same person as described above.

[0139] The process of estimating whether or not the individuals are traveling together is performed when the Y coordinate of the first person's body coordinate 201C and the Y coordinate of the second person's body coordinate 201D are less than the body width threshold W2. It does not need to be performed when the Y coordinate of the first person's body coordinate 201C and the Y coordinate of the second person's body coordinate 201D are equal to or greater than the body width threshold W2. This is because when the Y coordinate of the first person's body coordinate 201C and the Y coordinate of the second person's body coordinate 201D are equal to or greater than the body width threshold W2, the first person and the second person are not walking together, and therefore there is no need to estimate whether or not they are traveling together.

[0140] Furthermore, since the face of the accompanying person 1D is hidden by the person 1C in front, the first imaging device 21 cannot photograph the face of the accompanying person 1D. Therefore, the first edge device 41 displays a matching result image on the first display device 31 for the accompanying person 1D, which includes the message "illegal passage" as shown in pattern (A3) of Figure 14, or the message "warning" as shown in pattern (A7) of Figure 14. When the first edge device 41 displays a matching result image that includes the message "illegal passage" or "warning," it may also display the image 200 of the rear of the accompanying person 1D, which was taken by the second imaging device 22, on the first display device 31 along with the message.

[0141] In other words, even if a tailing person 1D exists as shown in Figure 16, the gate system 9 can determine that the person in front 1C and the tailing person 1D are different people in step S401 of Figure 9, without mistakenly identifying them as the same person. The gate system 9 can then display a matching result image containing a message such as "illegal passage" or "warning" to the tailing person 1D who is attempting to enter (or exit) illegally by concealing their face with the person in front 1C.

[0142] <Method for detecting a person's direction of movement> Figure 17 is a diagram illustrating a method for detecting the direction of movement of person 1 according to Embodiment 1.

[0143] The control device 50 detects the direction of movement of person 1 by, for example, one of the following first method, second method, or third method.

[0144] <<Method 1>> The management device 50 detects the direction of movement 112 of the person information 100 based on the direction of movement of the face coordinates 109 or body coordinates 106 included in the person information 100 over time. For example, if the management device 50 detects that the X coordinate of the face coordinates 109 or the X coordinate of the body coordinates 106 included in the person information 100 is moving in the positive direction of the X axis over time, it detects the direction of movement 112 of the person 1 as the first direction. If the management device 50 detects that the X coordinate of the face coordinates 109 or the X coordinate of the body coordinates 106 included in the person information 100 is moving in the negative direction of the X axis over time, it detects the direction of movement 112 of the person 1 as the second direction.

[0145] <<Second Method>> The control device 50 determines, based on the imaging device ID, whether the image of the person 1 that is first detected was taken by the first imaging device 21 or by the second imaging device 22. If the image of the person 1 that is first detected was taken by the first imaging device 21, the control device 50 detects the direction of movement 112 of the person 1 as the first direction. This is because the person 1 that enters the detection area 3 from the second gate 12 is first photographed by the first imaging device 21 located at the first gate 11. If the image of the person 1 that is first detected was taken by the second imaging device 22, the control device 50 detects the direction of movement 112 of the person 1 as the second direction. This is because the person 1 that enters the detection area 3 from the first gate 11 is first photographed by the second imaging device 22 located at the second gate 12.

[0146] <<Third Method>> As shown in Figure 17, the control device 50 detects the direction of movement 112 of a person 1 as the first direction if the face coordinates 109 or body coordinates 106 of the person 1 detected for the first time are included in the predetermined area 312 of the detection area 3 that is closer to the second gate 12. This is because a person 1 entering the detection area 3 from the second gate 12 will first enter the predetermined area 312 that is closer to the second gate 12. The control device 50 detects the direction of movement 112 of a person 1 as the second direction if the face coordinates 109 or body coordinates 106 of the person 1 detected for the first time are included in the predetermined area 311 of the detection area 3 that is closer to the first gate 11. This is because a person 1 entering the detection area 3 from the first gate 11 will first enter the predetermined area 311 that is closer to the first gate 11.

[0147] (Embodiment 2) The gate system 9 according to Embodiment 2 has a configuration common to the gate system 9 described in Embodiment 1. In Embodiment 2, components already described in Embodiment 1 are given common reference numerals, and their descriptions may be omitted.

[0148] Figure 18 shows a diagram illustrating the case in which a person 1E is moving backward in a first direction according to Embodiment 2.

[0149] If the gate system 9 only has a first gate 11 and no second gate 12, it cannot photograph the face of a person 1E moving backward in the first direction, and therefore the gate system 9 will determine that person 1E is passing through illegally (see pattern (A3) in Figure 14). In contrast, in the gate system 9 according to Embodiment 2, the second photographing device 22 provided in the second gate 12 can also photograph the face of a person 1E moving backward in the first direction. Therefore, in the gate system 9 according to Embodiment 2, even if the face of a person 1E moving backward in the first direction is photographed by the second photographing device 22, the gate system 9 displays the face image of person 1E, including the message "warning," on the second display device 32 provided in the second gate 12, and then displays the matching result image of person 1E on the first display device 31 provided in the first gate 11. As a result, the gate system 9 can perform face matching even for a person 1E moving backward in the first direction and display the matching result image on a display device at the direction of the person's movement. The details are explained below.

[0150] Figure 19 is a flowchart showing an example of face matching result information processing according to Embodiment 2. In the face matching result information processing (S206) shown in Figure 7, the management device 50 according to Embodiment 2 may perform the following processing, which will be explained with reference to Figure 19.

[0151] The management device 50 sets the content of the face matching result information received from the face matching device 60 into the person information 100 in the person information table that corresponds to the face matching result information (S801).

[0152] The management device 50 generates person matching information (S802) based on the person information 100 set in step S801.

[0153] The management device 50 determines whether the direction of movement of the person information 100 set in step S801 is the first direction or the second direction (S803).

[0154] Next, we will explain the process when it is determined to be the first direction in step S803 (S803: first direction).

[0155] The management device 50 transmits the person matching information generated in step S802 to the first edge device 41 (S804).

[0156] The management device 50 determines whether or not the facial image included in the person matching information was captured by the second imaging device 22 (S805).

[0157] If the facial image included in the person matching information was not captured by the second imaging device 22 (S805: NO), the management device 50 completes this process.

[0158] If the facial image included in the person matching information was captured by the second imaging device 22 (S805: YES), the management device 50 transmits backward warning information containing the same tracking ID as the tracking ID included in the person matching information to the second edge device 42 (S806), and completes this process. Upon receiving the backward warning information, the second edge device 42 displays a message on the second display device 32 warning person 1, who corresponds to the tracking ID, to move backward (hereinafter referred to as the backward warning message), as shown in the matching result image 70D of Figure 2. This allows person 1, who is moving backward in the first direction, to see the backward warning message.

[0159] Next, we will explain the process when it is determined to be the second direction in step S803 (S803: second direction).

[0160] The management device 50 transmits the person matching information generated in step S802 to the second edge device 42 (S810).

[0161] The management device 50 determines whether or not the facial image included in the person matching information was captured by the first imaging device 21 (S811).

[0162] If the facial image included in the person matching information was not captured by the first imaging device 21 (S811: NO), the management device 50 completes this process.

[0163] If the facial image included in the person matching information was captured by the first imaging device 21 (S811: YES), the management device 50 transmits backward warning information containing the same tracking ID as the tracking ID included in the person matching information to the first edge device 41 (S812), and completes this process. Upon receiving the backward warning information, the first edge device 41 displays a message on the first display device 31 warning the person corresponding to the tracking ID to avoid moving backward (backward warning message), as shown in the matching result image 70D in Figure 2. This allows person 1, who is moving backward in the second direction, to see the backward warning message.

[0164] The details of the above-described embodiment 2 can be expressed as follows: The management device 50 displays warning information (for example, a backward-facing warning message) on the second display device 32 if the face image 220 of person 1 moving in the first direction was captured by the second shooting device 22, and displays warning information (for example, a backward-facing warning message) on the first display device 31 if the face image 220 of person 1 moving in the second direction was captured by the first shooting device 21.

[0165] This allows the management device 50 to display warning information on the display device on the side visible to person 1E when person 1E is moving backward.

[0166] (Embodiment 3) The gate system 9 according to Embodiment 3 has a configuration common to the gate system 9 described in Embodiment 1. In Embodiment 3, components already described in Embodiment 1 are given common reference numerals, and their descriptions may be omitted.

[0167] Figure 20 shows an example of Embodiment 3 in which an adult 1F is holding a child 1G. Figure 20(a) shows the case where the face of child 1G, who is being held by adult 1F, is facing forward. Figure 20(b) shows the case where the face of child 1G, who is being held by adult 1F, is facing backward.

[0168] As shown in Figure 20, there are cases where an adult 1F carries a child 1G through the gate. In this case, the gate system 9 according to Embodiment 3 detects the faces of both the adult 1F and the child 1G. In Figure 20, the face coordinates calculated from the face image 220F of the adult 1F are referred to as the adult face coordinates 221F, and the face coordinates calculated from the face image 220G of the child 1G are referred to as the child's face coordinates 221G.

[0169] The positional difference between the Y coordinate 221Fy of the adult's face coordinate 221F and the Y coordinate 221Gy of the child's face coordinate 221G being held by the adult 1F is smaller than the body width threshold W2 described in Embodiment 1, and the positional difference between the X coordinate 221Fx of the adult's face coordinate 221F and the X coordinate 221Gx of the child's face coordinate 221G being held by the adult 1F may be smaller than the body thickness threshold W described in Embodiment 1. Therefore, in the method for detecting person information 100 of the same person described in Embodiment 1, the management device 50 may mistakenly estimate that the adult's face image 220F and the child's face image 220G belong to the same person. Accordingly, in the management device 50 according to Embodiment 3, if the positional difference of the XYZ coordinates of the two face coordinates 221F and 221G is greater than or equal to a predetermined threshold, these two face coordinates 221F and 221G belong to two different people who are in a holding relationship. This estimation process is called the "holding estimation process."

[0170] Figure 21 shows an example of the configuration of person information 100 according to Embodiment 3.

[0171] The person information 100 shown in Figure 21 includes, in addition to the items included in the person information 100 shown in Figure 6, a hugging flag 120 and information about a person in a hugging relationship (hereinafter referred to as hugging person information 121). The hugging flag 120 indicates whether or not the person information 100 includes hugging person information 121. If the person information 100 includes hugging person information 121, the hugging flag 120 is set to ON. If the person information 100 does not include hugging person information 121, the hugging flag 120 is set to OFF. Note that the expressions ON and OFF are examples, and for example, the hugging flag 120 may be set to "1" instead of ON, or to "0" instead of OFF.

[0172] The information on the person being held (121) includes, as items, a body image (122), a body image update time (123), body coordinates (124), a face image (125), a face image update time (126), face coordinates (127), a face matching ID (128), and a matching status (129) for the person being held.

[0173] Figure 22 is a flowchart showing an example of face detection information processing according to Embodiment 3. The management device 50 according to Embodiment 3 may perform the following processing, which will be explained with reference to Figure 22, in the face detection information processing (S202) shown in Figure 7.

[0174] The management device 50 retrieves person information 100 corresponding to the face detection information received from the first edge device 41 or the second edge device 42 from the person information table (S901).

[0175] The management device 50 determines whether or not it was able to detect person information 100 from the person information table in step S901 (S902).

[0176] If it is determined in step S901 that person information 100 could not be found in the person information table (S902: NO), the management device 50 executes the processes in steps S303 to S306 shown in Figure 8 and completes this process.

[0177] If the management device 50 determines in step S901 that it has detected person information 100 from the person information table (S902: YES), it determines whether the face detection information is for a person being held (S903). For example, the management device 50 makes this determination by performing the above-described holding estimation process on the face coordinates 221 of the received face detection information and the face coordinates 109 of the person information 100 detected in step S901.

[0178] If the face detection information is not for the person being held (S903:NO), the management device 50 executes the processes shown in steps S310 to S312 in Figure 8 and completes the process.

[0179] If the face detection information is for the person being held (S903:YES), the management device 50 performs the following processing.

[0180] Based on the face detection information, the management device 50 updates the person being held information 121 in the person information table, which is included in the person information 100 detected in step S901 (S904).

[0181] The management device 50 determines whether the matching status 129 of the person being held information 121 is "before matching request" or "matching failed" (S905).

[0182] If the matching status 129 of the person being held information 121 is "matching request not yet made" or "matching failed" (S905:YES), the management device 50 sends a face matching request including the face image 125 and tracking ID 101 to the face matching device 60 and updates the matching status 129 of the person being held information 121 to "Matching" (S906). Then the management device 50 completes this process.

[0183] If the matching status 129 of the person being held information 121 is not "matching request not yet made" or "matching failed" (S905:NO), the management device 50 completes this process.

[0184] Figure 23 is a flowchart showing a first example of face matching result information processing according to Embodiment 3. In the face matching result information processing (S206) shown in Figure 7, the management device 50 according to Embodiment 3 may perform the following processing, which will be explained with reference to Figure 23.

[0185] The management device 50 sets the content of the face matching result information received from the face matching device 60 into the person information 100 in the person information table that corresponds to the face matching result information (S1001).

[0186] The management device 50 determines whether the hugging flag 120 of the person information 100 is ON or OFF (S1002).

[0187] If the hugging flag 120 for the person information 100 is OFF (S1002: NO), the management device 50 executes the processes S602 to S605 shown in Figure 11 and completes this process.

[0188] If the hugging flag 120 for the person information 100 is ON (S1002: YES), the management device 50 determines whether the matching status 111 and 129 for all persons included in the person information 100, including the hugging person information 121, is a successful match (S1003).

[0189] If the matching status 111 and 129 for all individuals included in the person information 100 is successful (S1003: YES), the management device 50 generates person matching information that includes at least the tracking ID 101, the successful matching status, and the facial images 107 and 125 of all individuals included in the person information 100 (S1004), and proceeds to step S1006.

[0190] If the matching status 111, 129 of at least one person included in the person information 100 is a matching failure (S1003: NO), the management device 50 generates person matching information that includes at least the tracking ID 101, the matching failure, and the facial images 107, 125 of all persons included in the person information 100 (S1005), and proceeds to step S1006.

[0191] In other words, the process shown in Figure 23 treats adult 1F and the child being held 1G as one group. If face matching of both adult 1F and the child being held 1G is successful (i.e., face matching of all members of the group), the group is considered to have been matched successfully. If face matching of at least one person in adult 1F and the child being held 1G fails (i.e., face matching of at least one person in the group), the group is considered to have been matched unsuccessfully.

[0192] The management device 50 determines whether the movement direction 112 of the person information 100 is in the first direction or the second direction (S1006).

[0193] If the movement direction 112 of the person information 100 is the first direction (S1006: first direction), the management device 50 transmits the person matching information generated in step S1004 or S1005 to the first edge device 41 (S1007) and completes the process. In this case, as shown in the matching result image 70F of Figure 2, the first edge device 41 displays the face image 107 of adult 1F and the face image 125 of the child being held 1G together in a single-person display area of ​​the first display device 31, and also displays the matching result (i.e., matching success or matching failure) included in the person matching information.

[0194] If the movement direction 112 included in the person information 100 is the second direction (S1006: second direction), the management device 50 transmits the person matching information generated in step S1004 or S1005 to the second edge device 42 (S1008) and completes the process. In this case, as shown in the matching result image 70F of Figure 2, the second edge device 42 displays the face image 107 of adult 1F and the face image 125 of the child being held 1G together in a single-person display area of ​​the second display device 32, and also displays the matching result included in the person matching information (i.e., matching success or matching failure).

[0195] The content of the first example described above can be expressed as follows: The management device 50 detects the position of each person 1 moving between the first gate 11 and the second gate 12, and if multiple different face images 220 are detected within a predetermined range (for example, within the range of the body thickness threshold W) based on the position of person 1 moving in the first direction, and all of the multiple different face images 220 are successfully matched, the management device 50 displays the matching result including all the successfully matched face images 220 on the first display device 31. If multiple different face images 220 are detected within a predetermined range (for example, within the range of the body thickness threshold W) based on the position of person 1 moving in the second direction, and all of the multiple different face images 220 are successfully matched, the management device 50 displays the matching result including all the successfully matched face images 220 on the second display device 32.

[0196] As a result, for example, the facial image 107 of adult 1F and the facial image 125 of child 1G being held by adult 1F are detected, and even if the facial image 125 of child 1G was taken by a camera opposite to the direction of movement, the management device 50 can display both the facial image 107 of adult 1F and the facial image 125 of child 1G on the display device installed ahead in the direction of movement.

[0197] Figure 24 is a flowchart showing a second example of face matching result information processing according to Embodiment 3. The management device 50 according to Embodiment 3 may perform the following processing, which will be explained with reference to Figure 24, in the face matching result information processing (S206) shown in Figure 7. Furthermore, the management device 50 may adopt either the first example shown in Figure 23 or the second example shown in Figure 24.

[0198] The control device 50 performs the same processing as steps S1001 to S1002 shown in Figure 23 (S1101 to S1102).

[0199] If the hugging flag 120 for the person information 100 is OFF (S1102: NO), the management device 50 executes the processes of steps S602 to S605 shown in Figure 11 and completes the process.

[0200] If the hugging flag 120 of the person information 100 is ON (S1102: YES), the management device 50 determines whether the matching status 111, 129 of at least one person included in the person information 100, including the hugging person information 121, is a successful match (S1103).

[0201] If the matching status 111, 129 of at least one person included in the person information 100 is successful (S1103: YES), the management device 50 generates person matching information that includes at least the tracking ID 101, the successful matching status, and the successful matching facial images 107, 125 (S1104), and proceeds to step S1106.

[0202] If the matching status 111 and 129 for all individuals included in the person information 100 is a matching failure (S1103: NO), the management device 50 generates person matching information that includes at least the tracking ID 101, the matching failure, and the facial images 107 and 125 of all individuals included in the person information 100 (S1105), and proceeds to step S1106.

[0203] In other words, in the process shown in Figure 24, adult 1F and the child being held 1G are treated as one group. If face matching of at least one person in adult 1F and the child being held 1G is successful (i.e., face matching of at least one person in the group), the group is considered to have been matched successfully. If face matching of all members of adult 1F and the child being held 1G (i.e., face matching of all members in the group) fails, the group is considered to have been matched unsuccessfully.

[0204] The management device 50 determines whether the movement direction 112 of the person information 100 is in the first direction or the second direction (S1106).

[0205] If the movement direction 112 of the person information 100 is the first direction (S1106: first direction), the management device 50 transmits the person matching information generated in step S1104 or S1105 to the first edge device 41 (S1107) and completes the process. In this case, if the person matching information indicates a successful match, the first edge device 41 may display a matching result image on the first display device 31 that includes only the successfully matched face image and a matching success message. If the person matching information indicates a matching failure, the first edge device 41 may display a matching result image on the first display device 31 that includes the face image 107 of adult 1F, the face image 125 of the child being held 1G, and a matching failure message.

[0206] If the direction of movement of the person information 100 is the second direction (S1106: second direction), the management device 50 transmits the person matching information generated in step S1104 or S1105 to the second edge device 42 (S1108) and completes the process. In this case, if the person matching information indicates a successful match, the second edge device 42 may display a matching result image on the second display device 32 that includes only the successfully matched face image and a matching success message. If the person matching information indicates a matching failure, the second edge device 42 may display a matching result image on the second display device 32 that includes the face image 107 of adult 1F and the face image 125 of the child being held 1G, along with a matching failure message.

[0207] The content of the second example described above can be expressed as follows: The management device 50 detects the position of each person 1 moving between the first gate 11 and the second gate 12, detects multiple different face images 220 within a predetermined range (for example, within the range of the body thickness threshold W) based on the position of person 1 moving in the first direction, and if at least one of the multiple different face images 220 is successfully matched, the management device 50 displays the matching result including the at least one successfully matched face image 220 on the first display device 31. If multiple different face images 220 are detected within a predetermined range (for example, within the range of the body thickness threshold W) based on the position of person 1 moving in the second direction, and at least one of the multiple different face images 220 is successfully matched, the management device 50 displays the matching result including the at least one successfully matched face image 220 on the second display device 32.

[0208] As a result, for example, the facial image 107 of adult 1F and the facial image 125 of child 1G being held by adult 1F are detected. Even if one of the facial images fails to match, the management device 50 can display the facial image of the one that was successfully matched on the display device installed ahead in the direction of movement.

[0209] (Embodiment 4) The gate system 9 according to Embodiment 4 has a configuration common to the gate system 9 described in Embodiment 1. In Embodiment 4, components already described in Embodiment 1 are given common reference numerals, and their descriptions may be omitted.

[0210] Figure 25 shows an example in which a person's face is printed on the clothing worn by person 1H according to Embodiment 4.

[0211] As shown in Figure 25, there are cases where a person's face is printed on the clothing worn by person 1H. Hereinafter, the face printed on the clothing will be referred to as the clothing face image 220I. In this case, during the face detection process, both the person's face image 220H and the clothing face image 220I may be detected. In this embodiment, the face coordinates calculated from the person's face image 220H will be referred to as the person's face coordinates 221H, and the face coordinates calculated from the clothing face image 220I will be referred to as the clothing face coordinates 221I.

[0212] The positional difference between the Y coordinate 221H of the person's face coordinate 221H and the Y coordinate 221Iy of the clothing's face coordinate 221I is smaller than the body width threshold W2 described in Embodiment 1, and the positional difference between the X coordinate 221Hx of the person's face coordinate 221H and the X coordinate 221Ix of the clothing's face coordinate 221I may be smaller than the body thickness threshold W described in Embodiment 1. Therefore, there is a possibility that the person's face image 220H and the clothing's face image 220I may be mistakenly assumed to belong to the same person. Accordingly, the management device 50 according to Embodiment 4 determines that one of the two face images 220H and 220I is the person's face image if the positional difference between the Z coordinates of the two face coordinates 221H and 221I is greater than or equal to a predetermined threshold. This determination process is called the multiple face determination process.

[0213] As shown in Figure 25, typically, the person's face image 220H is located higher (in the positive Z-axis direction) than the clothing's face image 220I. Therefore, when the management device 50 detects two face images 220H and 220I, it may determine the face image located higher (in the positive Z-axis direction) to be the person's face image 220H in the multiple face determination process. In this embodiment, for the sake of clarity, an example is described in which one of the two face images 220H and 220I is the clothing's face image 220I, but two or more of the multiple face images may be clothing's face images.

[0214] Figure 26 shows an example of the configuration of person information 100 according to Embodiment 4.

[0215] The person information 100 shown in Figure 26 includes, in addition to the items included in the person information 100 shown in Figure 6, a multiple face flag 140 and information about multiple faces (hereinafter referred to as multiple face information 141). The multiple face flag 140 indicates whether or not the person information 100 includes the multiple face information 141. If the person information 100 includes the multiple face information 141, the multiple face flag 140 may be set to ON. If the person information 100 does not include the multiple face information 141, the multiple face flag 140 may be set to OFF. Note that the expressions ON and OFF are examples, and for example, the multiple face flag 140 may be set to "1" instead of ON, or to "0" instead of OFF.

[0216] Multiple face information 141 includes, as items, a face image 142, face image update time 143, face coordinates 144, face matching ID 145, and matching status 146 for the face or clothing of a person in a multiple face relationship.

[0217] Figure 27 is a flowchart showing an example of face detection information processing according to Embodiment 4. The management device 50 according to Embodiment 4 may perform the following processing, which will be explained with reference to Figure 27, in the face detection information processing (S202) shown in Figure 7.

[0218] The management device 50 retrieves person information 100 corresponding to the face detection information received from the first edge device 41 or the second edge device 42 from the person information table (S1201).

[0219] The management device 50 determines whether or not it was able to detect person information 100 from the person information table in step S1201 (S1202).

[0220] If it is determined in step S1201 that person information 100 could not be found in the person information table (S1202: NO), the management device 50 executes the processes in steps S303 to S306 shown in Figure 8 and completes this process.

[0221] If it is determined in step S1201 that person information 100 has been detected from the person information table (S1202: YES), the management device 50 determines whether the face detection information is for multiple faces (S1203). For example, the management device 50 performs the above-described multiple face determination process on the face coordinates 221 of the received face detection information and the face coordinates 109 of the person information 100 detected in step S1201 to make this determination.

[0222] If the face detection information is not for multiple faces (S1203: NO), the management device 50 executes the processes shown in steps S310 to S312 in Figure 8 and completes the process.

[0223] If the face detection information is related to multiple faces (S1203:YES), the management device 50 performs the following processing.

[0224] Based on the face detection information, the management device 50 updates the multiple face information 141 included in the person information 100 detected in step S1201 in the person information table (S1204).

[0225] The management device 50 determines whether the matching status 129 of the multiple face information 141 is "before matching request" or "matching failed" (S1205).

[0226] If the matching status 129 of the multiple face information 141 is "matching request not yet made" or "matching failed" (S1205: YES), the management device 50 sends a face matching request including the face image 125 and tracking ID 101 to the face matching device 60 and updates the matching status 129 of the person information 100 to "matching in progress" (S1206). Then, the management device 50 completes this process.

[0227] If the matching status 129 of the multiple facial information 141 is not "matching request not yet made" or "matching failed" (S1205: NO), the management device 50 completes this process.

[0228] Figure 28 is a flowchart showing an example of face matching result information processing according to Embodiment 4. The management device 50 according to Embodiment 4 may perform the following processing, which will be explained with reference to Figure 28, in the face matching result information processing (S206) shown in Figure 7.

[0229] The management device 50 sets the content of the face matching result information received from the face matching device 60 into the person information 100 in the person information table that corresponds to the face matching result information (S1301).

[0230] The management device 50 determines whether the multiple face flag 140 of the person information 100 is ON or OFF (S1302).

[0231] If the multiple face flag 140 for the person information 100 is OFF (S1302: NO), the management device 50 executes the processes in steps S602 to S605 shown in Figure 11 and completes the process.

[0232] If the multiple face flag 140 of the person information 100 is ON (S1302: YES), the management device 50 determines whether the matching status 111 or 146 of the face image with the highest face coordinates 109, 144 (hereinafter referred to as the high-position face image) among the two or more face images 107, 142 included in the person information 100, including the multiple face information 141, is a successful match (S1303).

[0233] If the matching status 111 or 146 of the high-position facial image included in the person information 100 is successful (S1303: YES), the management device 50 generates person matching information that includes at least the tracking ID 101, the successful matching status, and the high-position facial image included in the person information 100 (S1304), and proceeds to step S1306.

[0234] If the matching status 111 or 146 of the high-position facial image included in the person information 100 is a matching failure (S1303: NO), the management device 50 generates person matching information that includes at least the tracking ID 101, the matching failure, and the high-position facial image included in the person information 100 (S1305), and proceeds to step S1306.

[0235] The management device 50 determines whether the direction of movement 112 of the person information 100 is in the first direction or the second direction (S1306).

[0236] If the movement direction 112 of the person information 100 is the first direction (S1306: first direction), the management device 50 transmits the person matching information generated in step S1304 or S1305 to the first edge device 41 (S1307) and completes the process. In this case, as shown in the matching result image 70C of Figure 3, the first edge device 41 displays the face image 220H of person 1H (i.e., the high-position face image) instead of the face image 220I of the clothing in the display area of ​​the first display device 31, and also displays the matching result (i.e., matching success or matching failure) included in the person matching information.

[0237] If the movement direction 112 of the person information 100 is the second direction (S1306: second direction), the management device 50 transmits the person matching information generated in step S1304 or S1305 to the second edge device 42 (S1306), and completes the process. In this case, as shown in the matching result image 70C of Figure 3, the second edge device 42 displays the face image 220H of person 1H (i.e., the high-position face image) instead of the face image 220I of the clothing in the display area of ​​the second display device 32, and also displays the matching result (i.e., matching success or matching failure) included in the person matching information.

[0238] The contents of the above-described embodiment 4 can be expressed as follows: The management device 50 detects the position of each person 1 moving between the first gate 11 and the second gate 12. If multiple different face images 220 are detected within a predetermined range (for example, within the range of the body thickness threshold W) based on the position of person 1 moving in the first direction, the device displays the matching result of the face image 220 at the highest position on the first display device 31. If multiple different face images 220 are detected within a predetermined range (for example, within the range of the body thickness threshold W) based on the position of person 1 moving in the second direction, the device displays the matching result of the face image 220 at the highest position on the second display device 32.

[0239] As a result, even if, for example, the facial image 220H of person 1H and the facial image 220I of the clothing printed on the clothing of person 1H are detected, the management device 50 can correctly display the matching result of the facial image 220H of person 1H on the display device installed in the direction of movement of person 1H.

[0240] The contents of Embodiments 1 to 4 described above may be combined in any way. For example, Embodiment 1 may be combined with at least one of the contents of Embodiment 2, Embodiment 3, and Embodiment 4. Also, since the contents of Embodiment 3 and Embodiment 4 are similar, they may be implemented as a common process.

[0241] (A variation of the gate system) The configuration of the gate system 9 is not limited to the configuration shown in Figure 1. Several variations of the gate system 9 configuration are shown below.

[0242] Figure 29 shows a first modified example of the gate system 9 according to this disclosure.

[0243] As shown in Figure 29, the gate system 9 may be configured to include a first imaging device 21 and a first display device 31 suspended from the ceiling of a passageway with a ceiling, and a second imaging device 22 and a second display device 32. Note that Figure 29 omits the description of the second imaging device 22 and the second display device 32, as well as the description of the person 1 moving in the second direction.

[0244] Figure 30 shows a second modified example of the gate system 9 according to this disclosure.

[0245] As shown in FIG. 30, the gate system 9 may be an integrated device having a tunnel-shaped configuration that allows the person 1 to pass through, and comprising a first imaging device 21, a first display device 31, a second imaging device 22, and a second display device 32 on the top surface of the tunnel. In other words, the above-mentioned first gate 11 and second gate 12 may be configured integrally. Note that in FIG. 30, the illustration of the second imaging device 22 and the second display device 32, and the illustration of the person 1 moving in the second direction are omitted. The tunnel-type gate system 9 may be installed fixedly, or may be moved and installed at any location.

[0246] FIG. 31 is a diagram showing a third modification of the gate system 9 according to the present disclosure.

[0247] As shown in FIG. 31, the gate system 9 does not necessarily need to have a gate shape through which the person 1 passes, and may have a configuration including a movable first set including the first imaging device 21 and the first display device 31, and a movable second set including the second imaging device 22 and the second display device 32. Then, the first set may be installed at a position where the face of the person 1 can be imaged without obstructing the movement of the person 1 in the first direction, and the second set may be installed at a position where the face of the person 1 can be imaged without obstructing the movement of the person 1 in the second direction.

[0248] (Hardware Configuration) FIG. 32 is a diagram showing an example of the hardware configuration of a computer according to the present disclosure.

[0249] The first edge device 41, the second edge device 42, the management device 50, and the face matching device 60 described above may be configured to include the computer 1000 shown in FIG. 32. Further, the processing performed by the first edge device 41, the second edge device 42, the management device 50, and the face matching device 60 described above may be realized by the computer 1000 shown in FIG. 32 executing a computer program.

[0250] The computer 1000 includes a processor 1001, a memory 1002, a storage 1003, an input device 1004, an output device 1005, a communication device 1006, a GPU (Graphics Processing Unit) 1007, a reading device 1008, and a bus 1009. The processor 1001, the memory 1002, the storage 1003, the input device 1004, the output device 1005, the communication device 1006, the GPU 1007, and the reading device 1008 are connected to the bus 1009, and can transmit and receive data bidirectionally via the bus 1009.

[0251] The processor 1001 executes computer programs stored in the memory 1002 to implement the processing performed by each of the devices described above. Therefore, the processing performed by each of the devices described above can be reinterpreted as processing centered on the processor 1001. Examples of the processor 1001 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a controller, an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field-Programmable Gate Array).

[0252] The memory 1002 stores computer programs and data handled by the computer 1000. The memory 1002 may include a ROM (Read-Only Memory) and a RAM (Random Access Memory). Further, the memory 1002 may be configured of either a volatile storage medium or a non-volatile storage medium.

[0253] The storage 1003 is configured of a non-volatile storage medium, and stores computer programs and data handled by the computer 1000. Examples of the storage 1003 include an HDD (Hard Disk Drive), an SSD (Solid State Drive), a flash memory, and the like.

[0254] The input device 1004 receives data to be input to the processor 1001. Examples of the input device 1004 include a keyboard, mouse, touchpad, or microphone.

[0255] The output device 1005 outputs data generated by the processor 1001. Examples of the output device 1005 include a display or a speaker.

[0256] The communication device 1006 transmits and receives data with other devices via the communication network 4. The communication device 1006 may include a transmitting unit for transmitting data and a receiving unit for receiving data. The communication device 1006 may support either wired communication or wireless communication.

[0257] GPU1007 is a device that processes images at high speed. Furthermore, GPU1007 may be used for AI processing (e.g., machine learning or deep learning).

[0258] The reading device 1008 is a device that reads data from an external recording medium such as a DVD-ROM (Digital Versatile Disk Read Only Memory) or a USB (Universal Serial Bus) memory.

[0259] (Summary of this disclosure) The contents of this disclosure described above can be expressed as follows:

[0260] <Item 1> The gate system 9 includes an imaging device (e.g., first imaging device 21) capable of capturing images of the faces of a person moving in a first direction and a person moving in a second direction opposite to the first direction within a predetermined area (e.g., detection area 3) of the passage 2, a display device (e.g., display device 31) having a screen that allows the person moving in the first direction to be seen, and a processor (e.g., processor 1001 of the management device 50). The processor detects the direction of movement of each person, the imaging device captures images of the faces of the people moving in the passage 2, and the display device 31 displays the matching results of the faces of the people moving in the first direction but does not display the matching results of the faces of the people moving in the second direction. In this way, by detecting the direction of movement of a person, the display device will show the matching results of the face images of people detected to be moving in the first direction, and will not show the matching results of the face images of people detected to be moving in the second direction. Thus, a gate system 9 can be realized that can appropriately show matching results to people moving in the first direction.

[0261] <Item 2> In the gate system 9 described in item 1, the predetermined area is at least a part of the area between the virtual first line and the second line in the passage 2, the first direction is the direction from the first line to the second line, and the second direction may be the direction from the second line to the first line. This allows the matching results to be appropriately displayed to person 1 as they move between the first and second lines.

[0262] <Item 3> In the gate system 9 described in item 2, the widths of the first and second lines are such that multiple persons 1 can pass through side by side, and the display device may be capable of simultaneously displaying the matching results of multiple persons 1. This allows for simultaneous matching of multiple individuals, thus reducing congestion during the matching process.

[0263] <Item 4> The gate system 9 described in item 2 or 3 may further include a control device 50 including a processor. The control device 50 may detect the direction of movement 112 of each person 1 moving between the first line and the second line, and display the matching result of the face image 220 of the person 1 moving in the first direction on a display device. As a result, the display device will show the matching results of the facial images of people detected to be moving in the first direction, but will not show the matching results of the facial images of people detected to be moving in the second direction, thus allowing the matching results to be shown appropriately to people moving in the first direction.

[0264] <Item 5> In the gate system 9 described in any one of items 1 to 4, the control device 50 may display a body image of the first person along with warning information on the first display device 31 if there is a first person moving between the first line and the second line for which a face image cannot be obtained from the imaging device, and the first person is moving in the first direction. This allows a display device that can identify individuals whose facial images cannot be acquired to show an image and information visually indicating that the facial image of that individual could not be acquired.

[0265] <Item 6> In the gate system 9 described in any one of items 1 to 5, the control device 50 detects the position of each person 1 moving between the first line and the second line, and if multiple different face images 220 are detected within a predetermined range based on the position of person 1 moving in the first direction, and all of the multiple different face images 220 are successfully matched, the control device 50 may display the matching result, including all the successfully matched face images 220, on the display device. As a result, for example, if an adult's face image 220E and a child's face image 220F being held by the adult 1F are detected, and if the child's face image 220F was captured by the camera in the direction of movement 112, the management device 50 can display both the adult's face image 220E and the child's face image 220F on the display device installed ahead in the direction of movement.

[0266] <Item 7> In the gate system 9 according to any one of Items 1 to 5, the management device 50 detects the position of each person 1 moving between a first line and the second line, and when a plurality of different face images 220 are detected within a predetermined range based on the position of the person 1 moving in a first direction, and at least one face image 220 among the plurality of different face images 220 is successfully matched, a matching result including the at least one successfully matched face image 220 may be caused to be displayed on a display device. Thereby, for example, when an adult's face image 220E and a child's face image 220F held by the adult 1F are detected, even if one of the face images 220 fails to match, the management device 50 can cause the successfully matched face image 220 to be displayed on the display device installed ahead in the moving direction.

[0267] <Item 8> In the gate system 9 according to any one of Items 1 to 7, the management device 50 detects the position of each person 1 moving between a first line and a second line, and when a plurality of different face images 220 are detected within a predetermined range based on the position of the person 1 moving in a first direction, the matching result of the face image 220 located at the highest position may be caused to be displayed on a display device. Thereby, for example, even when a person's face image 220G and a face image 220H printed on the clothes of the person 1 are detected, the management device 50 can correctly cause the matching result of the person's face image 220G to be displayed on the display device installed in the moving direction of the person 1.

[0268] <Item 9> The gate device 10 includes an imaging device (e.g., first imaging device 21) capable of capturing images of the faces of a person moving in a first direction and a person moving in a second direction opposite to the first direction within a predetermined area (e.g., detection area 3) of the passage 2, and a display device (e.g., display device 31) having a screen on which a person moving in the first direction can be seen. In the control method of the gate device 10, the gate device 10 detects the direction of movement of each person, the imaging device captures images of the faces of the people moving in the passage 2, and the display device displays the matching results of the faces of the people moving in the first direction, but does not display the matching results of the faces of the people moving in the second direction. In this way, by detecting the direction of movement of a person, the display device will show the matching results of the face images of a person detected to be moving in the first direction, and will not show the matching results of the face images of a person detected to be moving in the second direction. Therefore, a control method for the gate device 10 that can appropriately show the matching results to a person moving in the first direction can be realized.

[0269] <Item 10> The gate device 10 includes a shooting device (e.g., first shooting device 21) capable of capturing images of the faces of a person moving in a first direction and a person moving in a second direction opposite to the first direction within a predetermined area (e.g., detection area 3) of the passage 2, and a display device (e.g., first shooting device 31) having a screen on which a person moving in the first direction can be seen. The control program for the gate device 10 causes the gate device 10 to detect the direction of movement of each person, to cause the shooting device to capture an image of the face of a person moving in the passage 2, to display the matching result of the face image of a person moving in the first direction on the display device, and not to display the matching result of the face image of a person moving in the second direction. In this way, by detecting the direction of movement of a person, the display device will show the matching results of the face images of people detected to be moving in the first direction, and will not show the matching results of the face images of people detected to be moving in the second direction. Therefore, a control program for the gate device 10 can be realized that appropriately shows the matching results to people moving in the first direction.

[0270] While embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these are also understood to fall within the technical scope of this disclosure. Furthermore, the components of the embodiments described above can be combined in any way without departing from the spirit of the invention. [Industrial applicability]

[0271] The technology disclosed herein is useful for systems, devices, methods, and programs for matching individuals. [Explanation of Symbols]

[0272] 1, 1E, 1H People 1C Front person 1D: Accompanying person 1F Adults 1G Child 2 aisles 3. Detection area 4. Communication Network 9 Gate System 10 Gate device 11. Gate 1 12. Gate 2 21 First imaging device 22. Second imaging device 31 1st display device 32 2nd display device 41. First Edge Device 42. Second edge device 50 Management device 60 Face Recognition Device Matching result images for 70A, 70B, 70C, 70D, 70E, and 70F. 100 Personal information 101 Tracking ID 102 Time of first image capture 103 Image update time 104 Body Images 105 Body image update time 106 Body coordinates 107 Face Images 108 Face image update time 109 Face coordinates 110 Face Recognition ID 111 Verification Status 112 Direction of movement 120 Hug flag 121 Person information 122 Body Images 123 Body image update time 124 Body coordinates 125 facial images 126 Face image update time 127 Face coordinates 128 Face Recognition ID 129 Matching Status 140 Multiple Face Flags 141 Multiple facial information 142 facial images 143 Face image update time 144 Face coordinates 145 Face Recognition ID 146 Matching Status 200 body images 201, 201A, 201B, 201C, 201D Body coordinates 220, 220E, 220F, 220G, 220H, 220I Face Images 221, 221A, 221F, 221G, 221H, 221I Face coordinates 301 Matching Zone 302 Passing Judgment Zone 311 The designated area closer to Gate 1 312 The designated area closer to Gate 2 1000 computers 1001 Processor 1002 memory 1003 Storage 1004 Input device 1005 Output device 1006 Communication device 1007 GPU 1008 Reader 1009 Bus

Claims

1. A photographing device capable of photographing a person moving in a first direction within a predetermined area of ​​a passageway and a person moving in a second direction opposite to the first direction, A display device having a screen that can be viewed simultaneously by multiple people moving in the first direction, A gate system comprising a control device, The predetermined region is at least a part of the region between a virtual first line set at an upstream position in the first direction in the passage and a virtual second line set at a downstream position in the first direction. A person moving in the first direction passes through the first line, the predetermined area, and the second line in that order. A person moving in the second direction passes through the second line, the predetermined area, and the first line in that order. The aforementioned photographic device generates a photographic image of a person moving along the passageway, The aforementioned control device is The captured image generated by the aforementioned imaging device is acquired, Based on the captured image, the direction of movement and position of each person moving between the first line and the second line are detected. Based on the captured image, if multiple different face images are detected within a predetermined range based on the position of the person moving in the first direction, and all of the multiple different face images are successfully matched, the matching result including all the successfully matched face images is displayed on the display device. The matching result of the face image of the person moving in the second direction is not displayed on the display device. Gate system.

2. A photographing device capable of photographing a person moving in a first direction within a predetermined area of ​​a passageway and a person moving in a second direction opposite to the first direction, A display device having a screen that can be viewed simultaneously by multiple people moving in the first direction, A gate system comprising a control device, The predetermined region is at least a part of the region between a virtual first line set at an upstream position in the first direction in the passage and a virtual second line set at a downstream position in the first direction. A person moving in the first direction passes through the first line, the predetermined area, and the second line in that order. A person moving in the second direction passes through the second line, the predetermined area, and the first line in that order. The aforementioned photographic device generates a photographic image of a person moving along the passageway, The aforementioned control device is The captured image generated by the aforementioned imaging device is acquired, Based on the captured image, the direction of movement and position of each person moving between the first line and the second line are detected. Based on the captured image, if a plurality of different face images are detected within a predetermined range based on the position of the person moving in the first direction, and at least one of the plurality of different face images is successfully matched, the matching result including the at least one successfully matched face image is displayed on the display device. The matching result of the face image of the person moving in the second direction is not displayed on the display device. Gate system.

3. The widths of the first and second lines are such that multiple people can pass through side by side. The display device is capable of simultaneously displaying the matching results of multiple individuals. The gate system according to claim 1 or 2.

4. The aforementioned control device is In cases where a face image cannot be obtained from the captured image generated by the aforementioned imaging device, and there is a first person moving between the first line and the second line, If the first person is moving in the first direction, the body image of the first person is displayed on the display device along with warning information. The gate system according to claim 1 or 2.

5. The aforementioned control device is Based on the captured image, the position of each person moving between the first line and the second line is detected. If multiple different face images are detected within a predetermined range based on the position of a person moving in the first direction, the matching result of the face image at the highest position is displayed on the display device. The gate system according to claim 1 or 2.

6. A photographing device capable of photographing a person moving in a first direction within a predetermined area of ​​a passageway and a person moving in a second direction opposite to the first direction, A display device having a screen that can be viewed simultaneously by multiple people moving in the first direction, A control method for a gate device comprising, The predetermined region is at least a part of the region between a virtual first line set at an upstream position in the first direction in the passage and a virtual second line set at a downstream position in the first direction. A person moving in the first direction passes through the first line, the predetermined area, and the second line in that order. A person moving in the second direction passes through the second line, the predetermined area, and the first line in that order. The gate device is The aforementioned camera captures a person moving through the passage and acquires the resulting image. Based on the captured image, the direction of movement and position of each person moving between the first line and the second line are detected. Based on the captured image, if multiple different face images are detected within a predetermined range based on the position of the person moving in the first direction, and all of the multiple different face images are successfully matched, the matching result including all the successfully matched face images is displayed on the display device. The matching result of the face image of the person moving in the second direction is not displayed on the display device. A method for controlling a gate device.

7. A photographing device capable of photographing a person moving in a first direction within a predetermined area of ​​a passageway and a person moving in a second direction opposite to the first direction, A display device having a screen that can be viewed simultaneously by multiple people moving in the first direction, A control method for a gate device comprising, The predetermined region is at least a part of the region between a virtual first line set at an upstream position in the first direction in the passage and a virtual second line set at a downstream position in the first direction. A person moving in the first direction passes through the first line, the predetermined area, and the second line in that order. A person moving in the second direction passes through the second line, the predetermined area, and the first line in that order. The gate device is The aforementioned camera captures a person moving through the passage and acquires the resulting image. Based on the captured image, the direction of movement and position of each person moving between the first line and the second line are detected. Based on the captured image, if a plurality of different face images are detected within a predetermined range based on the position of the person moving in the first direction, and at least one of the plurality of different face images is successfully matched, the matching result including the at least one successfully matched face image is displayed on the display device. The matching result of the face image of the person moving in the second direction is not displayed on the display device. A method for controlling a gate device.

8. A photographing device capable of photographing a person moving in a first direction within a predetermined area of ​​a passageway and a person moving in a second direction opposite to the first direction, A display device having a screen that can be viewed simultaneously by multiple people moving in the first direction, A control program for a gate device comprising, The predetermined region is at least a part of the region between a virtual first line set at an upstream position in the first direction in the passage and a virtual second line set at a downstream position in the first direction. A person moving in the first direction passes through the first line, the predetermined area, and the second line in that order. A person moving in the second direction passes through the second line, the predetermined area, and the first line in that order. The gate device, The aforementioned camera captures a person moving through the passage and acquires the resulting image. Based on the captured image, the direction of movement and position of each person moving between the first line and the second line are detected. Based on the captured image, if multiple different face images are detected within a predetermined range based on the position of the person moving in the first direction, and all of the multiple different face images are successfully matched, the matching result including all the successfully matched face images is displayed on the display device. The matching result of the face image of the person moving in the second direction is not displayed on the display device. A control program for a gate device that performs a specific action.

9. A photographing device capable of photographing a person moving in a first direction within a predetermined area of ​​a passageway and a person moving in a second direction opposite to the first direction, A display device having a screen that can be viewed simultaneously by multiple people moving in the first direction, A control program for a gate device comprising, The predetermined region is at least a part of the region between a virtual first line set at an upstream position in the first direction in the passage and a virtual second line set at a downstream position in the first direction. A person moving in the first direction passes through the first line, the predetermined area, and the second line in that order. A person moving in the second direction passes through the second line, the predetermined area, and the first line in that order. The gate device, The aforementioned camera captures a person moving through the passage and acquires the resulting image. Based on the captured image, the direction of movement and position of each person moving between the first line and the second line are detected. Based on the captured image, if a plurality of different face images are detected within a predetermined range based on the position of the person moving in the first direction, and at least one of the plurality of different face images is successfully matched, the matching result including the at least one successfully matched face image is displayed on the display device. The matching result of the face image of the person moving in the second direction is not displayed on the display device. A control program for a gate device that performs a specific action.

Images