Person verification system, information processing device, person verification method, and person verification program
The person verification system addresses tailgating by separating areas for verification and non-verification, using facial image matching to ensure only authorized individuals pass, thus preventing unauthorized entry and reducing congestion.
Patent Information
- Application Number
- PCT/JP2024/008867
- Authority / Receiving Office
- WO · WO
- Patent Type
- Applications
- Current Assignee / Owner
- Priority Date
- 2023-05-24
- Filing Date
- 2024-03-07
- Publication Date
- 2024-11-28
AI Technical Summary
Existing room entry control systems face challenges in preventing tailgating while ensuring authorized individuals can enter smoothly, often leading to congestion and incorrectly flagging legitimate users as unauthorized.
A person verification system that sets separate areas for tailgating determination, using facial image matching to allow verified individuals to pass while preventing unverified individuals from moving through these areas, allowing authorized individuals to bypass verification when accompanied by unregistered persons.
This approach effectively prevents unauthorized entry while minimizing congestion by allowing verified individuals to pass with unregistered companions, improving the flow of authorized personnel while maintaining security.
Smart Images

Figure JP2024008867_28112024_PF_FP_ABST
Abstract
Description
Person matching system, information processing device, person matching method, and person matching program
[0001] The present disclosure relates to a person matching system, an information processing device, a person matching method, and a person matching program.
[0002] Patent Literature 1 discloses an entry management system that includes a door opening / closing control unit that controls the opening and closing of a managed door, an imaging unit that acquires a monitoring image of a monitored space near the managed door, a person authentication unit that authenticates a person present in the monitored space from the image acquired by the imaging unit, a permission determination unit that determines whether the person authenticated by the person authentication unit is a person who will be permitted to enter by opening or closing the managed door, and a specific action authentication unit that authenticates a specific action that is continuously performed for a predetermined period of time by a person who will be permitted to enter by the permission determination unit, and the door opening / closing control unit opens the managed door if the specific action is authenticated by the specific action authentication unit, and permits tailgating entry of the person who will be permitted to enter the room accompanied by a person who is not permitted to enter, and does not open the managed door if the specific action is not authenticated by the specific action authentication unit.
[0003] Japanese Patent Application Laid-Open No. 2020-88462
[0004] However, the entry management system disclosed in Patent Document 1 opens the managed door when the specific action of the person who is allowed to enter is authenticated, so at the moment the specific action of the person who is allowed to enter is authenticated, there is a risk that a person (hereinafter referred to as an unauthorized passerby) other than the person who the person who is allowed to enter is accompanying (hereinafter referred to as an accompanying person) may enter through the managed door.
[0005] Furthermore, in the entry management system disclosed in Patent Document 1, a method can be considered to avoid the possibility of mistakenly flagging an unauthorized person as permitted to enter by having people other than the person accompanying the person wait until the person who is permitted to enter has completed their entry while the person who is permitted to enter performs the specific action described above. However, with the above method, other people who are permitted to enter and who are not unauthorized people are also forced to wait to enter, resulting in a backlog in front of the managed door.
[0006] An object of the present disclosure is to provide a technology for setting an area on an aisle where tailgating determination is performed separately from an area where tailgating determination is not performed, in a system that allows passage of a tailgating person.
[0007] The person matching system according to the present disclosure includes a photographing device that photographs a plurality of people moving through a predetermined passageway, and an information processing device that matches the people using facial images of the people photographed by the photographing device, wherein the information processing device sets a first area and a second area in the passageway, and in the first area, performs matching using the facial images on people present in the first area, allowing people who are successfully matched to move through the passageway and not allowing people who are unsuccessful to move through the passageway, and in the second area, performs matching using the facial images on people present in the second area, allowing a first person who is successfully matched to move through the passageway and allowing a second person who is unsuccessful to match to move through the passageway while the first person is present in the second area.
[0008] The information processing device according to the present disclosure is an information processing device that matches a plurality of people moving down a specified passageway using facial images of the people captured by a photographing device that photographs the people, and is equipped with a processor and a memory, and the processor works in cooperation with the memory to set a first area and a second area in the passageway, and in the first area, performs a match using the facial images on people present in the first area, allowing people who are successfully matched to move down the passageway and not allowing people who are unsuccessful to move down the passageway, and in the second area, performs a match using the facial images on people present in the second area, allowing a first person who is successfully matched to move down the passageway and allowing a second person who is unsuccessful to move down the passageway while the first person is present in the second area.
[0009] A person matching method according to the present disclosure is a person matching method for matching a plurality of people moving along a predetermined passageway using facial images of the people photographed by a photographing device that photographs the people, the method including: setting a first area and a second area in the passageway; in the first area, matching is performed using the facial images of people present in the first area, and people who are successfully matched are permitted to move along the passageway, and people who are unsuccessful in the matching are not permitted to move along the passageway; and in the second area, matching is performed using the facial images of people present in the second area, and a first person who is successfully matched is permitted to move along the passageway, and a second person who is unsuccessful in the matching is permitted to move along the passageway while the first person is present in the second area.
[0010] A person matching program according to the present disclosure causes a computer to execute the person matching method described above.
[0011] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium.
[0012] According to the present disclosure, in a system that permits passage of tailgating persons, an area in an aisle where tailgating determination is performed can be set separately from an area where tailgating determination is not performed. As a result, when a person whose face matches successfully (e.g., an employee) moves down the aisle accompanied by a person whose face matches unsuccessfully (e.g., a visitor), the person passes through the area where tailgating determination is performed, thereby maintaining a comfortable progress situation in the aisle while preventing the system from granting passage to persons who are not tailgating persons and enabling detection of unauthorized passersby.
[0013] FIG. 7 is a diagram for explaining an overview of a gate system according to the first embodiment. FIG. 8 is a diagram showing an example of display of a matching result on the first display device or the second display device according to the first embodiment. FIG. 9 is a block diagram showing an example of the configuration of a gate system according to the first embodiment. FIG. 10 is a diagram for explaining a face image and a body image according to the first embodiment. a flow chart showing an example of result information processing (S206); a flow chart showing an example of display processing performed by the first edge device according to embodiment 1; a diagram showing face matching zones and passage determination zones in the detection area according to embodiment 1; a diagram for explaining an example of a display pattern in the first display device according to embodiment 1; a first diagram for explaining a method for detecting, from the person information table, personal information that is presumed to be the same person as the person indicated by the body detection information according to embodiment 1; a second diagram for explaining a method for detecting, from the person information table, personal information that is presumed to be the same person as the person indicated by the body detection information according to embodiment 1; a diagram for explaining a method for detecting the movement direction of a person according to embodiment 1;FIG. 1 shows an example of setting a face matching zone, a passage judgment zone, and a tailgating judgment zone. A flowchart showing an example of processing performed by the management device when setting a face matching zone, a tailgating judgment zone, and a passage judgment zone in a detection area according to embodiment 2. A flowchart showing an example of face matching processing according to embodiment 2. A flowchart showing a first example of face matching and tailgating judgment processing according to embodiment 2. A flowchart showing an example of passage judgment processing according to embodiment 2. A flowchart showing a second example of face matching and tailgating judgment processing according to embodiment 2. A diagram showing an example of setting a tailgating judgment zone around a face-registered person according to embodiment 2. A flowchart showing a third example of face matching and tailgating judgment processing according to embodiment 2. A face-registered person and a face-unregistered person according to embodiment 2 perform cooperative operations. FIG. 26 is a diagram illustrating a first method for determining whether or not a face-unregistered person is a tailgating person by cooperatively operating a face-registered person and a face-unregistered person according to embodiment 2; FIG. 27 is a diagram illustrating a second method for determining whether or not a face-unregistered person is a tailgating person by cooperatively operating a face-registered person and a face-unregistered person according to embodiment 2; FIG. 28 is a diagram illustrating an example of a display on the first display device when the processing shown in FIG. 25 or FIG. 26 is performed;
[0014] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.
[0015] (First Embodiment) <Outline of Gate System> Fig. 1 is a diagram for explaining an outline of a gate system 9 according to the first embodiment. Fig. 2 is a diagram showing an example of display of a matching result on the first display device 31 or the second display device 32 according to the first embodiment.
[0016] The gate system 9 is a system that collates a facial image of a person 1 and determines whether the person 1 has the right to pass through the gate. The gate system 9 includes a first gate 11 and a second gate 12. The first gate 11 and the second gate 12 are installed a predetermined distance apart in a passage 2 through which the person 1 travels. In this embodiment, the direction from the second gate 12 toward the first gate 11 is defined as a first direction, and the direction from the first gate 11 toward the second gate 12 is defined as a second direction. In this embodiment, the area between the first gate 11 and the second gate 12 in the passage 2 is referred to as a person detection area 3. Note that the gate system 9 does not necessarily have to have the first gate 11 and the second gate 12 under which a person passes, as shown in FIG. 1 . In this case, the gate system 9 may set a virtual first line instead of the first gate 11 and a virtual second line instead of the second gate 12, and the area between the first line and the second line may be defined as the detection area 3. The direction from the first line to the second line may be defined as the first direction, and the direction from the second line to the first line may be defined as the second direction. Therefore, in the following description, the first gate 11 may be read as the first line, and the second gate 12 may be read as the second line.
[0017] In this embodiment, for convenience of explanation, the axis along the first or second direction is referred to as the Y-axis, the axis perpendicular to the Y-axis and extending in the height direction is referred to as the Z-axis, and the axis perpendicular to the Y-axis and the Z-axis is referred to as the X-axis. The direction of the X-axis may also be referred to as the width direction of the gate.
[0018] The first gate 11 includes a first imaging device 21 capable of capturing an image of at least a portion of the detection area 3, and a first display device 31 having a screen facing the second gate 12. In FIG. 1 , the first gate 11 includes three first imaging devices 21, but the first gate 11 may include one or two first imaging devices 21, or four or more first imaging devices 21. Also, in FIG. 1 , the first gate 11 includes one first display device 31, but the first gate 11 may include two or more first display devices 31.
[0019] The second gate 12 is equipped with a second imaging device 22 capable of capturing an image of at least a portion of the detection area 3, and a second display device 32 having a screen facing the first gate 11. In Fig. 1, the second gate 12 is equipped with three second imaging devices 22, but the second gate 12 may be equipped with one or two second imaging devices 22, or may be equipped with four or more second imaging devices 22. Furthermore, in Fig. 1, the second gate 12 is equipped with one second display device 32, but the second gate 12 may be equipped with two or more second display devices 32.
[0020] In this embodiment, the first direction is the direction of entry into a facility or the like where the person has the right to enter or exit, and the second direction is the direction of exit from the facility or the like. However, the first direction may also be the direction of exit from a facility or the like, and the second direction may also be the direction of entry into the facility or the like.
[0021] The gate system 9 compares the facial image of the person 1 moving in the first direction captured by the first photographing device 21 at the first gate 11 with each pre-registered facial image for comparison, and displays the comparison result on the first display device 31, as illustrated in Figure 2.
[0022] The gate system 9 compares the facial image of the person 1 moving in the second direction captured by the second photographing device 22 at the second gate 12 with each pre-registered facial image for comparison, and displays the comparison result on the second display device 32, as illustrated in Figure 2.
[0023] The gate width of the first gate 11 and the second gate 12, or the width in the X direction of the detection area 3, may be wide enough for multiple people 1 to pass through side by side. As a result, the gate system 9 according to this embodiment allows more people 1 to enter and exit smoothly compared to a gate system with a gate width wide enough for only one person to pass through. For example, in a gate system with a gate width wide enough for one person, if the verification of a person 1 ahead fails, the following people 1 will be held up. However, in a gate system 9 according to this embodiment with a gate width wide enough for multiple people 1 to pass through side by side, even if the verification of a person 1 ahead fails, the following people 1 can simply move around the person 1 ahead who failed to be verified, and thus can pass through the gate smoothly without being held up. In other words, the gate system 9 according to this embodiment can prevent congestion when entering or exiting a facility, etc. The gate system 9 according to this embodiment will be described in detail below.
[0024] <Configuration of Gate System> Fig. 3 is a block diagram showing an example of the configuration of the gate system 9 according to embodiment 1. Fig. 4 is a diagram for explaining face images and body images according to embodiment 1.
[0025] The gate system 9 includes a first gate 11 and a second gate 12. The first gate 11 includes a first imaging device 21 and a first display device 31. The second gate 12 includes a second imaging device 22 and a second display device 32. The first gate 11, the first imaging device 21, the first display device 31, the second gate 12, the second imaging device 22, and the second display device 32 may be collectively referred to as a gate device 10.
[0026] The gate system 9 further includes a first edge device 41, a second edge device 42, a management device 50, and a face matching device 60. The first image capturing device 21 is connected to the first edge device 41 via an electrical cable (e.g., a USB cable or an Ethernet (registered trademark) cable). The first display device 31 is connected to the first edge device 41 via an electrical cable (e.g., an HDMI (registered trademark) cable). The second image capturing device 22 is connected to the second edge device 42 via an electrical cable (e.g., a USB cable or an Ethernet cable). The second display device 32 is connected to the second edge device 42 via an electrical cable (e.g., an HDMI cable). The first edge device 41 and the second edge device 42 are connected to the management device 50 via a communication network 4. The management device 50 is connected to the face matching device 60 via the communication network 4. Examples of the communication network 4 include a wired LAN, a wireless LAN, a mobile communication network (e.g., LTE, 4G, 5G), the Internet, and / or a VPN.
[0027] The first imaging device 21 captures an image of the detection area 3 to generate a captured image, and transmits the captured image to the first edge device 41. The second imaging device 22 captures an image of the detection area 3 to generate a captured image, and transmits the captured image to the second edge device 42. The captured image may be a still image or a moving image.
[0028] As shown in FIGS. 4( a) and 4(b), the first edge device 41 detects the entire body of the person 1 from the photographed image received from the first image capturing device 21. In this embodiment, the image of the entire body of the person 1 detected from the photographed image is referred to as a body image 200. The first edge device 41 transmits the detected body image 200 to the management device 50. Furthermore, as shown in FIG. 4(a), the first edge device 41 detects the face portion of the person 1 from the photographed image received from the first image capturing device 21. In this embodiment, the image of the face portion of the person 1 detected from the photographed image is referred to as a face image 220. The first edge device 41 transmits the detected face image 220 to the management device 50.
[0029] As shown in Figures 4(a) and 4(b), the second edge device 42 detects the entire body of the person 1 from the captured image received from the second image capturing device 22. The second edge device 42 transmits a body image 200 detected from the captured image to the management device 50. Also, as shown in Figure 4(a), the second edge device 42 detects the face portion of the person 1 from the captured image received from the second image capturing device 22. The second edge device 42 transmits a face image 220 detected from the captured image to the management device 50. Details of the processing performed by the first edge device 41 and the second edge device 42 will be described later.
[0030] The management device 50 generates person information 100 (see FIG. 6 ) corresponding to the person 1 located in the detection area 3 based on the body image 200 or face image 220 received from the first edge device 41, or the body image 200 or face image 220 received from the second edge device 42. The management device 50 may generate person information 100 for each person 1 detected in the detection area 3. Details of the person information 100 will be described later (see FIG. 6 ). The management device 50 transmits a face matching request including the face image 220 to the face matching device 60.
[0031] The face matching device 60 stores in advance a face image for matching of a person 1 who has the right to pass through a gate (for example, to enter or exit). For example, the face matching device 60 stores a face image for matching of a person 1 who has already registered to enter the facility. The face matching device 60 may manage the face image for matching by linking it to the name, age, sex, credit card information, etc. of the person 1.
[0032] Furthermore, the face matching device 60 matches the face image 220 received from the management device 50 with each of the face images for matching. If the face matching device 60 is successful in matching with any of the registered face images for matching, it generates face matching result information including a successful matching result and transmits it to the management device 50. At this time, the face matching device 60 may perform billing in real time using credit card information linked to the successfully matched face image for matching. If the face matching device 60 fails to match with any of the registered face images for matching, it generates face matching result information including a failed matching result and transmits it to the management device 50. Note that the matching between the face image 220 and the face images for matching may be achieved by a known face image matching technology. Also, "matching" may be read as "authentication."
[0033] The management device 50 receives face matching result information from the face matching device 60. The management device 50 transmits person matching information including face matching result information for the face image 220 of the person 1 moving in the first direction to the first edge device 41, and transmits person matching information including face matching result information for the face image 220 of the person 1 moving in the second direction to the second edge device 42.
[0034] When the first edge device 41 receives person matching information from the management device 50, it displays the face matching result information included in the person matching information on the first display device 31, as shown in Fig. 2. This allows a person 1 located in the detection area 3 and moving in the first direction to view the screen of the first display device 31 located in the direction of movement and know his / her own matching result. In other words, the first edge device 41 allows a person 1 moving in the first direction to visually recognize the matching result of the person 1.
[0035] When the second edge device 42 receives person matching information from the management device 50, it displays the face matching result information included in the person matching information on the second display device 32, as shown in FIG. 2. This allows person 1 located in the detection area 3 and moving in the second direction to view the screen of the second display device 32 located at the end of the moving direction and know his / her matching result. In other words, the second edge device 42 allows person 1 moving in the second direction to visually confirm the matching result for that person 1. Below, the processes performed by the first edge device 41, the second edge device 42, and the management device 50 will be described in detail.
[0036] <Detection Process in First Edge Device> Fig. 5 is a flowchart showing an example of the detection process performed by the first edge device 41 according to embodiment 1. Note that, for the detection process performed by the second edge device 42 according to embodiment 1, in the following description with reference to Fig. 5, the "first image capturing device 21" can be read as the "second image capturing device 22."
[0037] The first edge device 41 receives a captured image from the first image capturing device 21 (S101).
[0038] The first edge device 41 receives a distance image from the first image capturing device 21 (S102). In this way, the first image capturing device 21 may generate a distance image indicating the distance to the person 1 or the like present in the detection area 3. The same applies to the second image capturing device 22. The distance image may be generated using any known technology, such as a stereo camera or a distance measuring sensor (e.g., LiDAR (Light Detection And Ranging)). Note that in this embodiment, the processing of step S102 is performed after the processing of step S101, but this is not limiting and the procedures may be reversed.
[0039] 4A, the first edge device 41 detects a facial image 220 of the person 1 from the captured image (S103). The detection of the facial image 220 of the person may be realized using any known technology, such as pattern matching or AI (Artificial Intelligence). The AI may also be an image recognition technology using deep learning (e.g., CNN (Convolutional Neural Network)).
[0040] The first edge device 41 determines whether or not the face image 220 was detected from the captured image in step S103 (S104). If the face image 220 was not detected (S104: NO), the first edge device 41 proceeds to step S108.
[0041] If the face image 220 is detected (S104: YES), the first edge device 41 uses the distance image to calculate three-dimensional coordinates (hereinafter referred to as face coordinates 221 (see FIG. 4)) of the detected face image 220 (S105). For example, as shown in FIG. 4(a), the first edge device 41 calculates the center point of the face image 220 as the face coordinates 221.
[0042] The first edge device 41 generates face detection information (S106), for example, including an ID (hereinafter referred to as a photographing device ID) for identifying the first photographing device 21 that is the sender of the photographed image, the face image 220, the size of the face image 220 (hereinafter referred to as a face image size), face coordinates 221, and the photographing time of the photographed image.
[0043] The first edge device 41 transmits the generated face detection information to the management device 50 (S107), and the process proceeds to the next step S108.
[0044] As shown in Fig. 4(a) or 4(b), the first edge device 41 detects a body image 200 of the person 1 from the captured image (S108). The detection of the body image 200 of the person 1 may be realized using any known technology, such as pattern matching or AI. The AI may also be an image recognition technology using deep learning (e.g., CNN).
[0045] The first edge device 41 determines whether or not the body image 200 was detected from the captured image in step S108 (S109). If the body image 200 was not detected (S109: NO), the first edge device 41 returns the process to step S101.
[0046] If the body image 200 is detected (S109: YES), the first edge device 41 uses the distance image to calculate three-dimensional coordinates (hereinafter referred to as body coordinates 201 (see FIG. 4)) of the detected body image 200 (S110). For example, as shown in FIGS. 4(a) and 4(b), the first edge device 41 calculates the center point of the body image 200 as the body coordinates 201.
[0047] The first edge device 41 generates body detection information (S111). For example, the first edge device 41 generates body detection information including an image capturing device ID, a body ID for identifying the body image 200, the body image 200, body coordinates 201, and the image capturing time of the captured image.
[0048] The first edge device 41 transmits the generated body detection information to the management device 50 (S112), and the process returns to step S101. Note that in this embodiment, the process of detecting a face image from the captured image (S103 to S107) is performed first, followed by the process of detecting a body image from the captured image (S108 to S112), but this is not limiting, and the steps may be reversed.
[0049] In this way, by performing the process of detecting the facial image 220 and / or the body image 200 of the person 1 from the captured image and the process of calculating the facial coordinates 221 and / or the body coordinates 201 at the first edge device 41 or the second edge device 42, the processing load can be distributed compared to when these processes are performed at the management device 50. Note that if distribution of the processing load is not a consideration, the gate system 9 does not need to include the first edge device 41 and the second edge device 42. In this case, the first image capturing device 21, the first display device 31, the second image capturing device 22, and the second display device 32 may be directly connected to the management device 50. Then, the management device 50 may perform the process of detecting the facial image 220 and / or the body image 200 of the person 1 from the captured image and the process of calculating the facial coordinates 221 and / or the body coordinates 201.
[0050] <Personal Information> FIG. 6 shows an example of the configuration of personal information 100 according to the first embodiment.
[0051] The person information 100 is generated for each person 1 detected in the detection area 3. The person information 100 includes, as information or data, a tracking ID 101, a first image capture time 102, an image update time 103, a body image 104, a body image update time 105, body coordinates 106, a face image 107, a face image update time 108, face coordinates 109, a face matching ID 110, a matching status 111, and a movement direction 112.
[0052] The tracking ID 101 is set to an ID used to track the same person in the detection area 3. The tracking ID 101 can also be said to be an ID for identifying the person information 100.
[0053] The first image capturing time 102 is set to the time when the body image 104 or face image 107 included in the personal information 100 was first captured. The first image capturing time can also be said to be the time when the personal information 100 was first generated.
[0054] The image update time 103 is set to the time when the body image 104 or the face image 107 included in the personal information 100 was last updated. The image update time 103 can also be said to be the time when the personal information 100 was last updated.
[0055] The body image 104 is set to a body image 200 detected from a photographed image of the person 1 indicated by the personal information 100. The body image 104 may be updated as needed to the body image 200 detected from the latest photographed image.
[0056] The body image update time 105 is set to the time when the body image 104 included in the personal information 100 was last updated.
[0057] The body coordinates 201 calculated based on the body image 104 are set as the body coordinates 106. The body coordinates 106 may be expressed as three-dimensional coordinates (X coordinate, Y coordinate, Z coordinate) in the detection area 3.
[0058] The face image 107 is set to a face image 220 detected from a photographed image of the person 1 indicated by the personal information 100. The face image 107 may be updated as needed to the face image 220 detected from the latest photographed image.
[0059] The face image update time 108 is set to the time when the face image 107 included in the person information 100 was last updated.
[0060] The face coordinates 109 are set to face coordinates 221 calculated based on the face image 107. The face coordinates 109 may be expressed as three-dimensional coordinates (X coordinate, Y coordinate, Z coordinate) in the detection area 3.
[0061] An ID assigned to a successfully matched face image for matching is set in face matching ID 110. In other words, face matching ID 110 can also be said to be an ID that certifies person 1 corresponding to face image 107. If no successfully matched face image for matching exists (i.e., if face matching fails), NULL may be set in face matching ID 110.
[0062] The matching status 111 is set to the matching status for the face image 107. For example, the matching status 111 is set to one of "before matching request", "matching in progress", "matching successful", and "matching failed".
[0063] The movement direction 112 is set to the movement direction of the person 1 indicated by the person information 100. For example, the movement direction is set to either the first direction or the second direction.
[0064] The management device 50 manages a plurality of pieces of personal information 100 as a personal information table (not shown). The personal information table may be stored in a memory 1002 or a storage 1003 (see FIG. 32 ) provided in the management device 50, or may be stored in a device different from the management device 50.
[0065] <Overall Processing in Management Device> FIG. 7 is a flowchart showing an example of overall processing performed by the management device 50 according to the first embodiment.
[0066] The management device 50 determines whether or not face detection information (see step S107 in FIG. 5) has been received from the first edge device 41 or the second edge device 42 (S201).
[0067] When the management device 50 receives face detection information (S201: YES), it executes face detection information processing (S202). Note that the face detection information processing will be described in detail later (see FIG. 8).
[0068] If the management device 50 has not received face detection information (S202: NO), it determines whether it has received body detection information (see step S112 in Figure 5) from the first edge device 41 or the second edge device 42 (S203).
[0069] When the management device 50 receives the body detection information (S203: YES), the management device 50 executes body detection information processing (S204). Details of the body detection information processing will be described later (see FIG. 9).
[0070] If the management device 50 has not received the body detection information (S203: NO), it determines whether or not it has received face matching result information (S205). The face matching result information is information returned from the face matching device 60 in response to a face matching request transmitted from the management device 50 to the face matching device 60 in Fig. 8, which will be described later, and includes the result of face matching (for example, matching success or failure).
[0071] When the management device 50 receives the face matching result information (S205: YES), it executes face matching result information processing (S206). Note that the face matching result information processing will be described in detail later (see FIG. 11).
[0072] If the management device 50 has not received the face matching result information (S205: NO), the process returns to step S201.
[0073] After executing the face detection information processing (S202) or the body detection information processing (S204), the management device 50 executes post-movement processing (S210) and returns the processing to step S201. Details of the post-movement processing will be described later (see FIG. 10 ). Note that in this embodiment, the processing of steps S201, S203, and S205 is performed in stages, but this is not limiting, and the processing of steps S201, S203, and S205 may be interchanged or may be performed simultaneously in parallel.
[0074] <Face Detection Information Processing in Management Apparatus> FIG. 8 is a flowchart showing an example of the face detection information processing (S202) shown in FIG.
[0075] The management device 50 detects, from the person information table, the person information 100 corresponding to the face detection information received from the first edge device 41 or the second edge device 42 (S301). In this embodiment, the person information 100 corresponding to the face detection information is the person information 100 that is presumed to be the same person as person 1 indicated by the face detection information. Note that the method for detecting, from the person information table, the person information 100 that is presumed to be the same person as person 1 indicated by the face detection information will be described in detail later (see FIGS. 15A and 15B ).
[0076] The management device 50 determines whether or not the personal information 100 has been detected from the personal information table in step S301 (S302).
[0077] Next, a case where it is determined in step S302 that the personal information 100 cannot be detected from the personal information table (S302: NO) will be described.
[0078] The management device 50 generates new personal information 100 based on the face detection information (S303). At this time, the management device 50 sets the following contents in the new personal information 100: - A new tracking ID is set as the tracking ID 101 of the personal information 100. - The face image 220 included in the face detection information is set as the face image 107 of the personal information 100. - The face coordinates 221 included in the face detection information are set as the face coordinates 109 of the personal information 100. - The first image capture time 102 of the personal information 100 is set to the capture time included in the earliest face detection information (i.e., the earliest acquired capture time). - The matching status 111 of the personal information 100 is set to "before matching request." The management device 50 detects the movement direction of the person 1 corresponding to the face detection information, and sets the detected movement direction as the movement direction 112 of the personal information 100 generated in step S303 (S304). Note that the method for detecting the movement direction of a person will be described in detail later (see FIG. 16 ).
[0079] The management device 50 newly registers the personal information 100 generated and set in steps S303 and S304 in the personal information table (S305).
[0080] The management device 50 transmits a face matching request including the face image 107 and the tracking ID 101 to the face matching device 60, and updates the matching status 111 of the person information 100 to "matching in progress" (S306). Then, the management device 50 completes this process.
[0081] Next, a case where it is determined in step S302 that the personal information 100 has been detected from the personal information table (S302: YES) will be described.
[0082] The management device 50 updates the content of the person information 100 detected in step S301 based on the face detection information (S310). For example, the management device 50 updates the detected person information 100 as follows: Updates the face coordinates 109 of the person information 100 to the face coordinates 221 included in the face detection information Updates the image update time 103 and face image update time 108 of the person information 100 to the shooting time included in the face detection information The management device 50 determines whether the matching status 111 of the detected person information 100 is "before matching request" or "matching failed" (S311). If the matching status 111 is not "before matching request" or "matching failed" (S311: NO), the management device 50 completes this process.
[0083] If the matching status 111 is "before matching request" or "matching failed" (S311: YES), the management device 50 transmits a face matching request including the face image 107 and the tracking ID 101 to the face matching device 60, and updates the matching status 111 of the person information 100 to "matching in progress" (S312). Then, the management device 50 completes this process.
[0084] <Body Detection Information Processing in Control Device> FIG. 9 is a flowchart showing an example of the body detection information processing (S204) shown in FIG.
[0085] The management device 50 detects, from the person information table, personal information 100 corresponding to the body detection information received from the first edge device 41 or the second edge device 42 (S401). In this embodiment, the personal information 100 corresponding to the body detection information is personal information 100 that is presumed to be the same person as person 1 indicated by the body detection information. Details of a method for detecting, from the person information table, personal information 100 that is presumed to be the same person as person 1 indicated by the body detection information will be described later (see FIGS. 15A and 15B ).
[0086] The management device 50 determines whether or not the personal information 100 has been detected from the personal information table in step S401 (S402).
[0087] Next, a case where it is determined in step S402 that the personal information 100 cannot be detected from the personal information table (S402: NO) will be described.
[0088] The management device 50 generates new personal information 100 based on the received body detection information (S403). At this time, the management device 50 sets the following contents in the new personal information 100: - A new tracking ID is set as the tracking ID 101 of the personal information 100; - The body image 200 included in the body detection information is set as the body image 104 of the personal information 100; - The body coordinates 202 included in the body detection information are set as the body coordinates 106 of the personal information 100; - The first image capture time 102 of the personal information 100 is set to the capture time included in the earliest body detection information (i.e., the earliest acquired capture time). The management device 50 detects the movement direction of the person 1 corresponding to the body detection information and sets the detected movement direction as the movement direction 112 of the personal information 100 generated in step S403 (S404). Note that the method for detecting the movement direction of the person 1 will be described in detail later (see FIG. 16 ).
[0089] The management device 50 newly registers the personal information 100 generated and set in steps S403 and S404 in the personal information table (S405), and the management device 50 then completes this process.
[0090] Next, a case where it is determined in step S402 that the personal information 100 has been detected from the personal information table (S402: YES) will be described.
[0091] The management device 50 updates the content of the person information 100 detected in step S401 based on the body detection information (S410). For example, the management device 50 updates the detected person information 100 as follows: Updates the body coordinates 106 of the person information 100 to the body coordinates 202 included in the body detection information Updates the image update time 103 and body image update time 105 of the person information 100 to the body image capture time included in the body detection information Then, the management device 50 completes this process.
[0092] <Post-Movement Processing in Management Apparatus> FIG. 10 is a flowchart showing an example of the post-move processing (S210) shown in FIG.
[0093] The management device 50 generates person movement information based on the target person information (S501). In this embodiment, the target person information is the person information 100 detected from the person information table or the newly generated person information 100 in the face detection information processing shown in FIG. 8, or the person information 100 detected from the person information table or the newly generated person information 100 in the body detection information processing shown in FIG. 9. At this time, the management device 50 sets the following information in the person movement information: Tracking ID Zone information Face image (or, if there is no face image, a body image) Matching status Face matching ID Here, the zone information is information indicating whether the person 1 corresponding to the target person information is located in the face matching zone 301 or the passage determination zone 302 in the detection area 3. The face matching zone 301 and the passage determination zone 302 will be described in detail later (see FIG. 13).
[0094] The management device 50 determines whether the movement direction of the target person information is the first direction or the second direction (S502).
[0095] If the movement direction is the first direction (S502: first direction), the management device 50 transmits the person movement information generated in step S501 to the first edge device 41 (S503), and completes this process.
[0096] If the movement direction is the second direction (S502: second direction), the management device 50 transmits the person movement information generated in step S501 to the second edge device 42 (S504), and completes this process.
[0097] <Face Matching Result Information Processing in Control Device> FIG. 11 is a flowchart showing an example of the face matching result information processing (S206) shown in FIG.
[0098] The management device 50 sets the content of the face matching result information received from the face matching device 60 to the person information 100 in the person information table that corresponds to the face matching result information (S601). For example, the management device 50 sets the content of the face matching result information to the person information 100 as follows: - Sets the face matching ID of the face matching result information to the face matching ID 110 of the person information 100 - Sets the matching result of the face matching result information (matching success or matching failure) to the matching status 111 of the person information 100 The management device 50 generates person matching information based on the person information 100 set in step S601 (S602).
[0099] At this time, the management device 50 sets the following information in the person matching information: Tracking ID Zone information Facial image Matching status Facial matching ID The management device 50 determines whether the movement direction 112 of the person information 100 set in step S601 is the first direction or the second direction (S603).
[0100] If the movement direction 112 is the first direction (S602: first direction), the management device 50 transmits the person matching information generated in step S602 to the first edge device 41 (S604), and completes this process.
[0101] If the movement direction 112 is the second direction (S602: second direction), the management device 50 transmits the person matching information generated in step S602 to the second edge device 42 (S605), and completes this process.
[0102] <Display Processing in First Edge Device> Fig. 12 is a flowchart showing an example of display processing performed by the first edge device 41 according to embodiment 1. Note that, for the display processing performed by the second edge device 42 according to embodiment 1, in the following description with reference to Fig. 12, "first edge device 41" may be read as "second edge device 42," "first display device 31" may be read as "second display device 32," and "first direction" may be read as "second direction."
[0103] The first edge device 41 determines whether or not it has received person movement information or person matching information from the management device 50 (S701). If the first edge device 41 has not received either person movement information or person matching information (S701: NO), it returns the process to step S701. If the first edge device 41 has received either person movement information or person matching information (S701: YES), it performs the following process.
[0104] The first edge device 41 determines whether a face image or a body image corresponding to the tracking ID included in the received person movement information or person matching information is displayed on the first display device 31 (S702).
[0105] If the face image or body image corresponding to the tracking ID is not displayed on the first display device 31 (S702: NO), the first edge device 41 newly displays the face image or body image and the matching status included in the person movement information or person matching information on the first display device 31 (S703). Then, the first edge device 41 returns the process to step S701.
[0106] If a face image or body image corresponding to the tracking ID is displayed on the first display device 31 (S702: YES), the first edge device 41 updates the displayed face image or body image and matching status to the face image or body image and matching status included in the person movement information or person matching information and displays them (S704).Then, the first edge device 41 returns the process to step S701.
[0107] 2, a matching result image showing the face image or body image of each person 1 moving in the first direction and the matching status (e.g., matching success or failure, etc.) is displayed on the first display device 31. Therefore, each person 1 moving in the first direction can visually confirm their own matching result from the first display device 31 installed at the destination in the direction of movement.
[0108] 2, the second display device 32 displays a matching result image showing the face image or body image of each person 1 moving in the second direction and the matching status (e.g., matching success or failure, etc.) Therefore, each person 1 moving in the second direction can visually confirm their own matching result from the second display device 32 installed at the destination in the direction of movement.
[0109] The details of the above-mentioned display process can be expressed as follows: That is, the management device 50 detects the movement direction of each person 1 moving between the first gate 11 and the second gate 12, and displays the matching result of the facial image 220 of the person 1 moving in the first direction on the first display device 31 but not on the second display device 32, and displays the matching result of the facial image 220 of the person 1 moving in the second direction on the second display device 32 but not on the first display device 31.
[0110] As a result, the matching result for person 1 is displayed on the display device installed in the direction of movement of person 1, and is not displayed on the display device installed opposite the direction of movement of person 1. Therefore, the management device 50 can appropriately show the matching result to each of person 1 moving in the first direction along the passage 2 and person 1 moving in the second direction.
[0111] 13 is a diagram showing face matching zone 301 and passage determination zone 302 in detection area 3 according to embodiment 1. Note that, although the following description will be given using first gate 11 and first direction as an example, the same applies to second gate 12 and second direction.
[0112] The image captured by the first photographing device 21 at the first gate 11 is used for face matching of the person 1 moving in the first direction. In the detection area 3, a predetermined area close to the second gate 12 is set as a face matching zone 301, and a predetermined area close to the first gate 11 is set as a passage determination zone 302. The first edge device 41 may change the display pattern on the first display device 31 depending on whether the person 1 is located in the face matching zone 301 or the passage determination zone 302. Next, the display pattern will be described.
[0113] 14 is a diagram for explaining an example of a display pattern in the first display device 31 according to embodiment 1. Note that, with regard to the display pattern in the second display device 32, in the following description with reference to FIG. 14, the "first edge device 41" may be read as the "second edge device 42," and the "first display device 31" may be read as the "second display device 32."
[0114] If the person 1 is located outside the detection area 3 (outside the zone), the first edge device 41 does not display the matching result image of the person 1 on the first display device 31 .
[0115] When person 1 is located in face matching zone 301, first edge device 41 controls the display of first display device 31 according to the following patterns (A0) to (A3).
[0116] (A0) When a facial image 220 of person 1 exists and the matching status 111 is “matching in progress”, the first edge device 41 displays a matching result image including the facial image 220 of person 1 and the message “matching in progress” on the first display device 31.
[0117] (A1) When the facial image 220 of person 1 exists and the matching status 111 is “matching successful,” the first edge device 41 displays a matching result image including the facial image 220 of person 1 and a message “matching successful” on the first display device 31. At this time, the first edge device 41 may output a matching success sound from a speaker provided in the first display device 31.
[0118] (A2) When the facial image 220 of person 1 exists and the matching status 111 is “matching failed,” the first edge device 41 displays a matching result image including the facial image 220 of person 1 and a message stating “matching failed” on the first display device 31. At this time, the first edge device 41 may output a matching failure sound from a speaker provided in the first display device 31.
[0119] (A3) When the face image 220 of person 1 does not exist but the body image 200 exists, the first edge device 41 displays a matching result image including the body image 200 of person 1 and a “warning” message on the first display device 31. At this time, the first edge device 41 may output a warning sound from a speaker provided in the first display device 31.
[0120] When person 1 is located in passage determination zone 302, first edge device 41 controls the display of first display device 31 according to the following patterns (A4) to (A7).
[0121] (A4) When a facial image 220 of person 1 exists and the matching status 111 is “matching in progress,” the first edge device 41 displays a matching result image including the facial image 220 of person 1 and the message “matching in progress” on the first display device 31.
[0122] (A5) When the facial image 220 of person 1 exists and the matching status 111 is “matching successful,” the first edge device 41 displays a matching result image including the facial image 220 of person 1 and a message “passed” on the first display device 31. At this time, the first edge device 41 may output a passing sound from a speaker provided in the first display device 31.
[0123] (A6) When the face image 220 of person 1 exists and the matching status 111 is “matching failed,” the first edge device 41 displays a matching result image including the face image 220 of person 1 and a message saying “unauthorized passage” on the first display device 31. At this time, the first edge device 41 may output an unauthorized passage sound from a speaker provided in the first display device 31.
[0124] (A7) When the face image 220 of person 1 does not exist but the body image 200 exists, the first edge device 41 displays a matching result image including the body image 200 of person 1 and a message saying “illegal passage” on the first display device 31. At this time, the first edge device 41 may output an illegal passage sound from a speaker provided in the first display device 31.
[0125] The display priority may be set as follows: The passage determination zone 302 has a higher display priority than the face matching zone 301. The "passing" message has the highest display priority, followed by the "matching successful" message. If a face image 220 is present, the "matching in progress" message has the lowest display priority. The presence of a face image 220 has a higher display priority than the presence of only a body image 200.
[0126] For example, the display priorities of the above-described patterns (A0) to (A7) may be set as follows, from lowest to highest: (A3) < (A7) < (A0) < (A4) < (A2) < (A6) < (A1) < (A5) <Method for Detecting Personal Information of the Same Person> FIGS. 15A and 15B are diagrams for explaining a method for detecting, from the personal information table, personal information 100 that is presumed to belong to the same person as person 1 indicated by the body detection information in embodiment 1. Note that FIG. 15 will be described assuming that person 1 is moving in a first direction. In the case where person 1 is moving in a second direction, in the following description referring to FIG. 15 , the "first camera device 21" should be read as the "second camera device 22," and the "second camera device 22" should be read as the "first camera device 21."
[0127] A body image 200 of person 1 moving in a first direction captured by the first camera 21 shows the front view of the body of person 1. Hereinafter, the X and Y coordinates of body coordinates 201A calculated from the front view of body image 200 will be referred to as the X coordinate 201Ax and Y coordinate 201Ay of the front of the body, respectively. Furthermore, a face image 220 of person 1 moving in the first direction captured by the first camera 21 shows the front view of the face of person 1. Hereinafter, the X and Y coordinates of face coordinates 221A calculated from the front view of face image 220 will be referred to as the X coordinate 221Ax and Y coordinate 221Ay of the front of the face, respectively.
[0128] The body image 200 of the person 1 moving in the first direction captured by the second image capturing device 22 shows the back of the body of the person 1. Hereinafter, the X coordinate and the Y coordinate of the body coordinate 201B calculated from the back body image 200 will be referred to as the X coordinate 201Bx and the Y coordinate 201By of the back of the body, respectively.
[0129] As shown in FIG. 15A , the Y coordinate 201Ay of the front of the body and the Y coordinate 201By of the back of the body are separated by the thickness of the person 1 in the front-to-back direction. Therefore, if the positional difference between the Y coordinate 201Ay of the front of the body and the Y coordinate 201By of the back of the body is smaller than a predetermined threshold W based on the body thickness (hereinafter referred to as the body thickness threshold), it can be assumed that the Y coordinate 201Ay of the front of the body and the Y coordinate 201By of the back of the body correspond to the Y coordinates of the same person. The body thickness threshold W may be a value slightly larger than the average body thickness. Similarly, if the positional difference between the Y coordinate 221Ay of the front of the face and the Y coordinate 201By of the back of the body is smaller than the body thickness threshold W, it can be assumed that the Y coordinate 221Ay of the front of the face and the Y coordinate 201By of the back of the body correspond to the Y coordinates of the same person.
[0130] 15B , if the amount of change in the X component between X-coordinate 201Ax of body coordinate 201A included in the received body detection information (i.e., the current X-coordinate) and X-coordinate 201Ax of body coordinate 201A included in personal information 100 (i.e., the past X-coordinate) is equal to or less than a predetermined threshold value (hereinafter referred to as the body width threshold value) W2, it can be estimated that X-coordinate 201Ax of the current body coordinate and X-coordinate 201Ax of the past body coordinate are the X-coordinates of the same person. Note that face coordinate 221A may be used instead of body coordinate 201A.
[0131] In other words, if the management device 50 can detect from the personal information table personal information 100 such that the positional difference between the X coordinate of the body coordinate 201 (or face coordinate 221) included in the received body detection information and the X coordinate of the body coordinate 201 (or face coordinate 221) included in the personal information 100 is less than the body width threshold W2, and the positional difference between the Y coordinate of the body coordinate 201 included in the received body detection information and the Y coordinate of the body coordinate 201 or the Y coordinate of the face coordinate 221 included in the personal information 100 is less than the body thickness threshold W, the management device 50 may estimate that the detected personal information 100 is the same person as the person 1 indicated by the body detection information.
[0132] Similarly, if the management device 50 can detect from the personal information table personal information 100 in which the positional difference between the X coordinate of the body coordinate 201 (or face coordinate 221) included in the received body detection information and the X coordinate of the body coordinate 201 (or face coordinate 221) included in the personal information 100 is smaller than the body width threshold W2, and the positional difference between the Y coordinate of the face coordinate 221 included in the received face detection information and the Y coordinate of the body coordinate 201 or the Y coordinate of the face coordinate 221 included in the personal information 100 is smaller than the body thickness threshold W, the management device 50 may estimate that the detected personal information 100 is the same person as the person 1 indicated by the face detection information.
[0133] The above-described conditions regarding the X-coordinate positional difference and the body width threshold W2, and the Y-coordinate positional difference and the body thickness threshold W are met when the time difference between the face photographing time included in the face detection information or the body photographing time included in the body detection information and the face image update time 108 or the body image update time 105 included in the person information 100 is sufficiently small (i.e., when the time difference is smaller than a predetermined time threshold T). Because person 1 is moving, if the time difference is large, person information 100 that satisfies the above-described conditions regarding the X-coordinate positional difference and the body width threshold W2, and the Y-coordinate positional difference and the body thickness threshold W is likely to belong to a different person who was in that position in the past.
[0134] 8 may perform the following process: The management device 50 detects, from the personal information table, personal information 100 for which the time difference between the face capture time included in the face detection information and the face image update time 108 or the body image update time 105 included in the personal information 100 is less than a predetermined time threshold T, the positional difference between the X coordinate of the face coordinate 221 included in the face detection information and the X coordinate of the face coordinate 109 included in the personal information 100 is smaller than a body width threshold W2, and the positional difference between the Y coordinate of the face coordinate 221 included in the face detection information and the Y coordinate of the face coordinate 109 or the Y coordinate of the body coordinate 106 included in the personal information 100 is smaller than a body thickness threshold W.
[0135] 9 may perform the following process: The management device 50 detects, from the personal information table, personal information 100 in which the time difference between the body image capture time included in the body detection information and the face image update time 108 or the body image update time 105 included in the personal information 100 is less than a predetermined time threshold T, the positional difference between the X coordinate of the body coordinate 201 included in the body detection information and the X coordinate of the body coordinate 106 included in the personal information 100 is smaller than a body width threshold W2, and the positional difference between the X coordinate of the body coordinate 201 included in the body detection information and the Y coordinate of the face coordinate 109 or the Y coordinate of the body coordinate 106 included in the personal information 100 is smaller than a body thickness threshold W.
[0136] The above content can be expressed as follows: That is, if the difference between the position (e.g., body coordinates 201 or face coordinates 221) of person 1 included in a first image (e.g., body image 200 or face image 220) captured by first image capture device 21 and the position (e.g., body coordinates 201 or face coordinates 221) of person 1 included in a second image (e.g., body image 200 or face image 220) captured by second image capture device 22 is smaller than a predetermined threshold W based on the thickness of the person's body, management device 50 estimates that the position of person 1 included in the first image and the position of person 1 included in the second image represent the positions of the same person.
[0137] This allows the management device 50 to estimate, for example, whether a first image capturing the front of person 1 and a second image capturing the back of person 1 are of the same person.
[0138] <Method for Detecting Person's Moving Direction> FIG. 16 is a diagram for explaining a method for detecting the moving direction of person 1 according to the first embodiment.
[0139] The management device 50 detects the moving direction of the person 1 by, for example, the following first, second, or third method.
[0140] <<First Method>> The management device 50 detects the movement direction 112 of the person 1 based on the movement direction over time of the face coordinates 109 or the body coordinates 106 included in the person information 100. For example, when the management device 50 detects that the Y coordinate of the face coordinates 109 or the Y coordinate of the body coordinates 106 included in the person information 100 is moving in the negative direction of the Y axis over time, the management device 50 detects the movement direction 112 of the person 1 as a first direction. When the management device 50 detects that the Y coordinate of the face coordinates 109 or the Y coordinate of the body coordinates 106 included in the person information 100 is moving in the positive direction of the Y axis over time, the management device 50 detects the movement direction 112 of the person 1 as a second direction.
[0141] <<Second Method>> The management device 50 determines whether the image of the person 1 detected for the first time was captured by the first camera 21 or the second camera 22, based on the camera device ID. If the image of the person 1 detected for the first time was captured by the first camera 21, the management device 50 detects the movement direction 112 of the person 1 as the first direction. This is because the person 1 entering the detection area 3 from the second gate 12 is first photographed by the first camera 21 provided at the first gate 11. If the image of the person 1 detected for the first time was captured by the second camera 22, the management device 50 detects the movement direction 112 of the person 1 as the second direction. This is because the person 1 entering the detection area 3 from the first gate 11 is first photographed by the second camera 22 provided at the second gate 12.
[0142] 16 , when the face coordinates 109 or body coordinates 106 of a person 1 detected for the first time are included in a predetermined area 312 of the detection area 3 that is closer to the second gate 12, the management device 50 detects the movement direction 112 of the person 1 as a first direction. This is because the person 1 entering the detection area 3 from the second gate 12 first enters the predetermined area 312 closer to the second gate 12. When the face coordinates 109 or body coordinates 106 of a person 1 detected for the first time are included in a predetermined area 311 of the detection area 3 that is closer to the first gate 11, the management device 50 detects the movement direction 112 of the person 1 as a second direction. This is because the person 1 entering the detection area 3 from the first gate 11 first enters the predetermined area 311 closer to the first gate 11.
[0143] (Embodiment 2) In Embodiment 2, a gate system 9 will be described that allows passage even for a person who failed the facial matching described in Embodiment 1, if the person satisfies certain conditions. For example, when an employee passes through the gate system 9 with a visitor, the employee's facial image has been registered, but the visitor's facial image has not. Therefore, if the gate system 9 described in Embodiment 1 is used as is, all of the visitors will fail matching. To avoid this, a method of temporarily registering the facial images of all visitors by photographing their faces before they pass through the gate system 9 is conceivable. However, some visitors may be reluctant to have their faces photographed and their facial images registered, even if only temporarily. Furthermore, when there are a large number of visitors, photographing each visitor's face would take a long time. Therefore, the following describes a gate system 9 that allows passage for a person (e.g., a visitor) accompanied by a person (e.g., an employee) whose facial image has been registered and whose facial matching is successful, even without registering their facial image.
[0144] In the second embodiment, components that have already been described in the first embodiment may be denoted by common reference numerals and descriptions thereof may be omitted. Also, the contents of the first embodiment may be incorporated into the second embodiment.
[0145] <Setting and Processing of Tailgating Determination Zones> FIG. 17 is a diagram showing an example in which a face matching zone 301, a passage determination zone 302, and a tailgating determination zone 303 are set in the detection area 3 according to the second embodiment.
[0146] Management device 50 sets face matching zone 301, passage determination zone 302, and tailgating determination zone 303 in detection area 3. For example, as shown in Fig. 17 , management device 50 sets face matching zone 301 and tailgating determination zone 303 on the entrance side in the first direction, and sets passage determination zone 302 on the exit side in the first direction. Note that tailgating determination zone 303 may be set over the entire detection area 3. Also, face matching zone 301 may be read as the first area, and tailgating determination zone 303 may be read as the second area.
[0147] Next, the process performed by the management device 50 when the zones shown in FIG. 17 are set will be described with reference to FIG.
[0148] FIG. 18 is a flowchart showing an example of processing performed by management device 50 when face matching zone 301, tailgating determination zone 303, and passage determination zone 302 are set in detection area 3 according to the second embodiment.
[0149] The process shown in FIG. 18 is a process for one person to be tracked, and the management device 50 executes the process shown in FIG. 18 for each of a plurality of people to be tracked detected from the detection area 3.
[0150] The management device 50 detects a person who has entered the detection area 3 as a person to be tracked using the method described in the first embodiment (S1001).
[0151] The management device 50 determines whether the person to be tracked is located in the face matching zone 301, the tailgating determination zone 303, or the passage determination zone 302 (S1002).
[0152] When the management device 50 determines that the person being tracked is located in the face matching zone 301 (S1002: face matching zone), it executes face matching processing (S1003) and proceeds to step S1010. Details of the face matching processing will be described later (see FIG. 19).
[0153] If the management device 50 determines that the person being tracked is located in the tailgating determination zone 303 (S1002: tailgating determination zone), it executes face matching and tailgating determination processing (S1004), and proceeds to step S1010. Details of the face matching and tailgating determination processing will be described later (see FIG. 20).
[0154] When the management device 50 determines that the person being tracked is located in the passage determination zone 302 (S1002: passage determination zone), it executes a passage determination process (S1005) and proceeds to step S1010. The passage determination process will be described in detail later (see FIG. 21).
[0155] The management device 50 determines whether to end tracking of the person to be tracked (S1010). For example, the management device 50 ends tracking of the person to be tracked when the person to be tracked is no longer detected in the detection area 3. If the management device 50 continues tracking of the person to be tracked (S1010: NO), the process returns to step S1002, and if the management device 50 ends tracking of the person to be tracked (S1010: YES), the process ends.
[0156] <Flowchart of Face Collation Processing> Fig. 19 is a flowchart showing an example of face collation processing according to embodiment 2. The processing shown in Fig. 19 corresponds to details of the processing of step S1003 shown in Fig. 18 .
[0157] The management device 50 performs face matching of the tracking person by the method described in the first embodiment (S1101). At this time, the management device 50 may display a message indicating that matching is in progress on the first display device 31 from the start of face matching until the display of the face matching result.
[0158] The management device 50 determines whether or not the face matching is successful (S1102).
[0159] If the face matching is successful (S1102: YES), the management device 50 displays a message indicating the successful matching on the first display device 31 (S1103). Then, the management device 50 assigns a pass permission flag to the person being tracked (S1104), and completes this process.
[0160] If the face matching fails (S1102: NO), the management device 50 displays a matching failure message on the first display device 31 (S1105), and ends this process.
[0161] Note that the management device 50 may omit the process shown in Fig. 19 for a tracking person for whom face matching has been successful or unsuccessful once. After completion of this process, the process of step S1010 shown in Fig. 18 is executed.
[0162] <First Example: Flowchart of Face Matching and Tailgating Determination Processing> Fig. 20 is a flowchart showing a first example of face matching and tailgating determination processing according to Embodiment 2. The processing shown in Fig. 20 corresponds to details of the processing of step S1004 shown in Fig. 18 .
[0163] The management device 50 performs face matching of the person to be tracked using the method described in the first embodiment (S1201).
[0164] The management device 50 determines whether or not the face matching is successful (S1202).
[0165] If the face matching is successful (S1202: YES), the management device 50 displays a message indicating that the tailgating permission mode is in effect in the tailgating determination zone 303 on the first display device 31 (S1203). Then, the management device 50 assigns a passage permission flag to the person being tracked (S1204), and ends this process.
[0166] If the face matching fails (S1202: NO), the management device 50 displays a message indicating that tailgating permission determination is in progress on the first display device 31 (S1205).Then, the management device 50 determines whether or not a person whose face has already been successfully matched (i.e., face-registered person 1A) is present in the tailgating determination zone 303, apart from the tracking person whose face matching failed (S1206).
[0167] If there is no person whose face has already been successfully matched within the tailgating determination zone 303 (S1206: NO), the management device 50 displays a message indicating that tailgating permission determination is in progress on the first display device 31 (S1210) and terminates this process.
[0168] If a person whose face has already been successfully matched is present within the tailgating determination zone 303 (S1206: YES), the management device 50 determines that the person being tracked whose face has not been matched within the tailgating determination zone 303 (i.e., person 1B whose face has not been registered) is a tailgating person 1C (e.g., a visitor) brought by a person whose face has been successfully matched (e.g., an employee) (S1207).
[0169] The management device 50 displays a message on the first display device 31 to the effect that the passage of the tailgating person 1C is permitted (S1208).
[0170] The management device 50 assigns a passage permission flag to the tracking person whose face matching has failed (i.e., the tailgating person 1C) (S1209), and ends this process.
[0171] That is, in the tailgating determination zone 303, the management device 50 performs face matching on the person being tracked who is present in the tailgating determination zone 303, and allows the passage of a face-registered person 1A (e.g., an employee) whose face matching is successful, and while the face-registered person 1A is present in the tailgating determination zone 303, allows the passage of an unregistered person 1B (e.g., a visitor) who is present in the tailgating determination zone 303 and whose face matching is unsuccessful, as a companion 1C of the face-registered person 1A. On the other hand, if the face-registered person 1A is not present in the tailgating determination zone 303, the management device 50 does not allow the face-unregistered person 1B to pass. Note that the face-registered person 1A may be read as a first person, and the face-unregistered person 1B may be read as a second person.
[0172] As a result, tailgating person 1C, who is brought by face-registered person 1A whose face image has been registered and whose face matching is successful, can pass through passage 2 without undergoing face registration by passing through tailgating determination zone 303. Furthermore, face-unregistered person 1B, who is not brought by face-registered person 1A and whose face matching fails, cannot pass through face matching zone 301, let alone tailgating determination zone 303.
[0173] <Flowchart of Passage Determination Processing> Fig. 21 is a flowchart showing an example of the passage determination processing according to Embodiment 2. The processing shown in Fig. 21 corresponds to details of the processing of step S1005 shown in Fig. 18 .
[0174] The management device 50 determines whether or not a passage permission flag has been assigned to the person to be tracked (S1301).
[0175] If the tracking person has been assigned a passage permission flag (S1301: YES), the management device 50 displays a message indicating that passage is permitted on the first display device 31 (S1302), and then ends this process.
[0176] If the passage permission flag is not assigned to the person being tracked (S1301: NO), the management device 50 displays a message of unauthorized passage on the first display device 31 (S1303).
[0177] The management device 50 transmits the facial image of the person being tracked (including the accompanying person 1C) stored in memory 1002, who is passing without being assigned a passage permission flag, to, for example, the terminal of a staff member monitoring entry and exit (S1304), and completes this process.
[0178] As a result, a tracked person who has been given a passage permission flag in the face matching zone 301, and a tracked person who has been given a passage permission flag in the tailgating determination zone 303 (including tailgating person 1C) can pass through the passage determination zone 302.
[0179] <Second Example: Activating Tailgating Determination Zone Depending on Movement of Face-Registered Person> Fig. 22 is a flowchart showing a second example of the face matching and tailgating determination process according to embodiment 2. The process shown in Fig. 22 corresponds to details of the process of step S1004 shown in Fig. 18. Next, with reference to Fig. 22, a process for activating tailgating determination zone 303 when face-registered person 1A is present in tailgating determination zone 303 and has performed a first movement will be described.
[0180] The management device 50 performs face matching of the person to be tracked using the method described in the first embodiment (S1401).
[0181] The management device 50 determines whether or not the face matching is successful (S1402).
[0182] First, the case where face matching is successful (S1402: YES) will be described.
[0183] If the face matching is successful (S1402: YES), the management device 50 displays a message indicating successful matching on the first display device 31 (S1403).
[0184] The management device 50 assigns a passage permission flag to the person being tracked (S1404).
[0185] The management device 50 determines whether the person being tracked (i.e., face-registered person 1A) is performing a first action (S1405). For example, the management device 50 determines, as the first action, whether face-registered person 1A has been standing still for a certain period of time in the tailgating determination zone 303. Note that the first action is not limited to standing still for a certain period of time, and may be, for example, a motion in which face-registered person 1A is performing a predetermined pose (e.g., raising one's hand).
[0186] If the management device 50 determines that the tracking person (i.e., face-registered person 1A) is not performing the first action (S1405: NO), the management device 50 ends this process.
[0187] If the management device 50 determines that the person being tracked (i.e., face-registered person 1A) is performing the first action (S1405: YES), it sets the tailgating determination zone 303 to tailgating permission mode (i.e., enables the tailgating determination zone 303) (S1406).
[0188] The management device 50 displays a message indicating that the tailgating permission mode is in effect on the first display device 31 (S1407), and ends this process.
[0189] Next, a case where face matching is successful (S1402: NO) will be described.
[0190] If the face matching fails (S1402: NO), the management device 50 determines whether the tailgating determination zone 303 is in the tailgating permission mode (S1408).
[0191] If the tailgating determination zone 303 is not in tailgating permission mode (S1408: NO), the management device 50 displays a message on the first display device 31 indicating that matching has failed for the tracked person (i.e., person 1B whose face is not registered) (S1409), and terminates this processing.
[0192] If the tailgating determination zone 303 is in the tailgating permission mode (S1408: YES), the management device 50 displays a message for the tracking person (i.e., the face-unregistered person 1B) that tailgating is permitted on the first display device 31 (S1410). Then, the management device 50 assigns a passage permission flag to the tracking person (i.e., the face-unregistered person 1B) (S1411), and ends this process.
[0193] As a result, while face-registered person 1A is performing the first action in tailgating determination zone 303, management device 50 can place tailgating determination zone 303 in the tailgating permission mode and permit unregistered face person 1B present in tailgating determination zone 303 to pass as companion person 1C of face-registered person 1A. Furthermore, management device 50 does not place tailgating determination zone 303 in the tailgating permission mode if face-registered person 1A is not performing the first action in tailgating determination zone 303. As a result, face-registered person 1A can control whether or not to place tailgating determination zone 303 in the tailgating permission mode depending on whether or not he or she performs the first action in tailgating determination zone 303.
[0194] <Third Example: Setting a Tailgating Determination Zone Around a Face-Registered Person> Fig. 23 is a diagram showing an example of setting a tailgating determination zone 303 around a face-registered person 1A according to Embodiment 2. Fig. 24 is a flowchart showing a third example of the face matching and tailgating determination process according to Embodiment 2. The process shown in Fig. 24 corresponds to details of the process of step S1004 shown in Fig. 18 .
[0195] Next, with reference to FIGS. 23 and 24, a process for setting the tailgating determination zone 303 in a predetermined range including the face-registered person 1A will be described.
[0196] The management device 50 performs face matching of the person to be tracked using the method described in the first embodiment (S1501).
[0197] The management device 50 determines whether or not the face matching is successful (S1502).
[0198] First, the case where face matching is successful (S1502: YES) will be described.
[0199] If the face matching is successful (S1502: YES), the management device 50 displays a message indicating successful matching on the first display device 31 (S1503).
[0200] The management device 50 assigns a passage permission flag to the person being tracked (i.e., face-registered person 1A) (S1504).
[0201] The management device 50 determines whether the person to be tracked (i.e., the face-registered person 1A) is performing the first action (S1505).
[0202] If the management device 50 determines that the tracking person (i.e., face-registered person 1A) is not performing the first action (S1505: NO), the management device 50 ends this process.
[0203] When the management device 50 determines that the person to be tracked (i.e., face-registered person 1A) is performing the first action (S1505: YES), it sets a predetermined range including the position where the person to be tracked (i.e., face-registered person 1A) is present as the tailgating determination zone 303 (S1506). For example, as shown in Fig. 23 , the management device 50 sets a range with a predetermined radius centered on face-registered person 1A as the tailgating determination zone 303. Note that the shape of the tailgating determination zone 303 is not limited to a circle, and may be an oval, a rectangle, or the like.
[0204] The management device 50 displays a message indicating that the tailgating permission mode is in effect on the first display device 31 (S1507), and ends this process.
[0205] Next, a case where face matching fails (S1502: NO) will be described.
[0206] If face matching fails (S1502: NO), management device 50 determines (S1508) whether the person to be tracked (i.e., person 1B whose face is not registered) is present in tailgating determination zone 303. Note that if tailgating determination zone 303 is not set, management device 50 determines that the person to be tracked (i.e., person 1B whose face is not registered) is not present in tailgating determination zone 303.
[0207] If the person being tracked (i.e., person 1B whose face is not registered) is not present in the tailgating determination zone 303 (S1508: NO), the management device 50 displays a message on the first display device 31 indicating that matching for the person being tracked (i.e., person 1B whose face is not registered) has failed (S1509), and terminates this process.
[0208] If the person to be tracked (i.e., person 1B whose face is not registered) is present in tailgating determination zone 303 (S1508: YES), management device 50 displays a message for allowing tailgating for the person to be tracked (i.e., person 1B whose face is not registered) on first display device 31 (S1510). Then, management device 50 assigns a passage permission flag to the person to be tracked (i.e., person 1B whose face is not registered) (S1511), and ends this process.
[0209] As a result, while face-registered person 1A is performing the first action, management device 50 sets tailgating determination zone 303 in a predetermined range including face-registered person 1A, and can permit unregistered person 1B present in tailgating determination zone 303 to pass as a companion person 1C of face-registered person 1A. Furthermore, if face-registered person 1A is not performing the first action, management device 50 does not set tailgating determination zone 303. As a result, face-registered person 1A can control whether or not to set tailgating determination zone 303 depending on whether or not the first action is performed.
[0210] <Example 4: A face-registered person and a face-unregistered person cooperate to designate the face-unregistered person as a tailgating person> Figure 25 is a diagram illustrating a first method relating to embodiment 2, in which a face-registered person 1A and a face-unregistered person 1B cooperate to designate the face-unregistered person 1B as a tailgating person 1C.
[0211] As shown in FIG. 25, after successfully matching the face of a face-registered person 1A, if the distance L between the face-registered person 1A and the face-unregistered person 1B is equal to or less than a predetermined threshold, and both the face-registered person 1A and the face-unregistered person 1B perform a first action (for example, an action of standing still for a certain period of time) (i.e., the face-registered person 1A and the face-unregistered person 1B perform a cooperative action), the management device 50 will designate the face-unregistered person 1B as a person 1C accompanying the face-registered person 1A.
[0212] As a result, if face-registered person 1A and face-unregistered person 1B whose faces have been successfully authenticated do not cooperate with each other, face-unregistered person 1B will not be determined to be a tailgating person 1C. This reduces the possibility of erroneously determining face-unregistered person 1B, who is not a person brought by face-registered person 1A, as a tailgating person 1C.
[0213] FIG. 26 is a diagram illustrating a second method according to the second embodiment, in which a face-registered person 1A and a face-unregistered person 1B cooperate to turn the face-unregistered person 1B into a tailgating person 1C.
[0214] 26 , after successfully matching the face of registered person 1A, if the distance L between registered person 1A and unregistered person 1B within tailgating determination zone 303 is equal to or less than a predetermined threshold and both registered person 1A and unregistered person 1B perform a first action (for example, standing still for a certain period of time) (i.e., registered person 1A and unregistered person 1B perform a cooperative action), management device 50 determines unregistered person 1B to be a companion 1C of registered person 1A. Tailgating determination zone 303 may be set to the entire detection area 3 as shown in FIG. 26 , or may be set to a part of detection area 3 as shown in FIG. 17 .
[0215] As a result, even if the distance L between the face-registered person 1A and the face-unregistered person 1B is equal to or less than a predetermined threshold value and the face-registered person 1A and the face-unregistered person 1B whose faces have been successfully authenticated cooperate with each other, the face-unregistered person 1B will not be determined to be a tailgating person 1C if the face-registered person 1A or the face-unregistered person B is not present in the tailgating determination zone 303. This reduces the possibility of erroneously determining that the face-unregistered person 1B, who is not brought by the face-registered person 1A, is a tailgating person 1C.
[0216] FIG. 27 is a diagram showing an example of a display on the first display device 31 when the process shown in FIG. 25 or FIG. 26 is performed.
[0217] If the management device 50 succeeds in face matching for the face-registered person 1A, it displays a message on the first display device 31 indicating that tailgating permission mode is in effect for the face-registered person 1A, and displays a message indicating that tailgating permission determination is in progress for the face-unregistered person 1B, as shown in Figure 27 (a).
[0218] Then, when the distance L between the face-registered person 1A and the face-unregistered person 1B is equal to or less than a predetermined threshold and both the face-registered person 1A and the face-unregistered person 1B perform a first action (for example, an action of standing still for a certain period of time) (i.e., when the face-registered person 1A and the face-unregistered person 1B perform a cooperative action), the management device 50 displays a message to the face-unregistered person 1B on the first display device 31 that tailgating is permitted, as shown in Figure 27 (b).
[0219] This allows face-registered person 1A to recognize whether or not the vehicle is in the tailgating permission mode by looking at first display device 31. Also, face-unregistered person 1B can recognize by looking at first display device 31 whether or not the vehicle is in the tailgating permission determination mode and whether or not tailgating has been permitted.
[0220] Fig. 28 is a flowchart showing a fourth example of the face matching and tailgating determination process according to embodiment 2. The process shown in Fig. 28 corresponds to the details of the process of step S1004 shown in Fig. 18. Next, with reference to Figs. 25, 26, 27, and 28, a detailed description will be given of the face matching and tailgating determination process in the case where face-registered person 1A and face-unregistered person 1B cooperate to determine face-unregistered person 1B as tailgating person 1C.
[0221] The processes of S1601 to S1605 shown in Fig. 28 are the same as the processes of S1501 to S1505 shown in Fig. 24, and therefore will not be described here. After executing the process of step S1605, management device 50 determines whether or not the person to be tracked (i.e., person 1B whose face is not registered) is present in tailgating determination zone 303 (S1606). Note that, as in Fig. 25, if tailgating determination zone 303 does not exist, step S1606 may be omitted and the process of step S1607 may be executed.
[0222] If it is determined that the person to be tracked (i.e., person 1B whose face is not registered) is not present in tailgating zone 303 (S1606: NO), management device 50 ends this process.
[0223] If it is determined that the person being tracked (i.e., person 1B whose face is not registered) is present in the tailgating zone 303 (S1606: YES), the management device 50 displays a message indicating that tailgating is permitted on the first display device 31 (S1607) and terminates this processing.
[0224] Next, a case where face matching fails in step S1602 (S1602: NO) will be described.
[0225] If face matching fails (S1602: NO), management device 50 determines (S1608) whether or not the person being tracked (i.e., person 1B whose face is not registered) is present in tailgating determination zone 303. Note that, as shown in FIG. 25 , if tailgating determination zone 303 does not exist, step S1608 may be omitted and the process of step S1610 may be executed.
[0226] If it is determined that the person being tracked (i.e., person 1B with an unregistered face) is not present in the tailgating determination zone 303 (S1608: NO), the management device 50 displays a message on the first display device 31 indicating that matching for the person being tracked (i.e., person 1B with an unregistered face) has failed (S1609), and terminates this processing.
[0227] If it is determined that the person being tracked (i.e., person 1B whose face is not registered) is present in the tailgating determination zone 303 (S1608: YES), the management device 50 displays a message indicating that tailgating permission determination is in progress for the person being tracked (i.e., person 1B whose face is not registered) on the first display device 31 (S1610), as shown in FIG. 27(a).
[0228] The management device 50 determines whether the distance L between the person who has successfully completed face matching and is performing the first action (face-registered person 1A) and the person being tracked (face-unregistered person 1B) is less than or equal to a predetermined threshold (S1611).
[0229] If the distance L between the person (face-registered person 1A) who has successfully completed face matching and is performing the first action and the person being tracked (face-unregistered person 1B) is greater than a predetermined threshold (S1611: NO), the management device 50 displays a message on the first display device 31 indicating that matching has failed for the person being tracked (i.e., face-unregistered person 1B) (S1609), and terminates this processing.
[0230] If the distance L between the person (face-registered person 1A) who has successfully completed face matching and is performing the first action and the person being tracked (face-unregistered person 1B) is less than or equal to a predetermined threshold (S1611: YES), the management device 50 proceeds to the next step S1612.
[0231] The management device 50 determines whether the person being tracked (person 1B whose face is not registered) is performing an action in coordination with the first action (S1612).
[0232] If the management device 54 determines that the person being tracked (person 1B whose face is not registered) is not performing an action in coordination with the first action (S1612: NO), it displays a message on the first display device 31 indicating that matching has failed for the person being tracked (i.e., person 1B whose face is not registered) (S1609), and terminates this processing.
[0233] When the management device 54 determines that the tracking person (unregistered face person 1B) is performing an action in coordination with the first action (S1612: YES), it displays a message for allowing the tracking person (unregistered face person 1B) to travel with the tracking person on the first display device 31 (S1613), as shown in Fig. 27(b). Then, the management device 50 assigns a passage permission flag to the tracking person (unregistered face person 1B) (S1614), and ends this process.
[0234] This allows the management device 50 to perform the operations shown in FIG. 25, FIG. 26, or FIG.
[0235] 28 is based on the process shown in FIG. 24 , but the same process as that shown in FIG. 28 can be achieved even if the process shown in FIG. 22 or FIG. 24 is used as a base. For example, the management device 50 may perform a process in which steps S1408 to S1411 shown in FIG. 22 are replaced with steps S1608 to S1613 shown in FIG. 28 . Alternatively, the management device 50 may perform a process in which steps S1508 to S1511 shown in FIG. 24 are replaced with steps S1608 to S1613 shown in FIG. 28 . <Canceling the Tailgating Permission Mode> FIG. 29 is a diagram showing a first example of a display on the first display device 31 when the tailgating permission mode is canceled according to the second embodiment. FIG. 30 is a diagram showing a second example of a display on the first display device 31 when the tailgating permission mode is canceled according to the second embodiment.
[0236] 22 , while face-registered person 1A is performing the first action, management device 50 sets tailgating determination zone 303 to the tailgating permission mode, and during the tailgating permission mode, allows unregistered face person 1B present within tailgating determination zone 303 to pass as a companion person 1C of face-registered person 1A. In this case, as shown in FIG. 29( a) or 30(a), first display device 31 may display a message indicating that the tailgating permission mode is in effect for face-registered person 1A, and a message indicating that tailgating is permitted for tailgating person 1C.
[0237] Here, when the management device 50 detects that the face-registered person 1A has performed a second action different from the first action, the management device 50 may cancel the tailgating permission mode. For example, if the first action is standing still for a certain period of time, the second action may be starting to move from a standing still state. For example, if the first action is raising a hand for a certain period of time, the second action may be lowering the raised hand.
[0238] When the management device 50 cancels the tailgating permission mode, it may display a message to the face-registered person 1A indicating that the match was successful and that the tailgating permission mode has been cancelled, as shown in FIG. 29(b) or FIG. 30(a).
[0239] Even if the management device 50 cancels the tailgating permission mode, it may maintain the tailgating permission for the face-unregistered person 1B who was permitted to tailgate before the face-registered person 1A performed the second action, as shown in Figure 29 (b).
[0240] Alternatively, when the management device 50 cancels the tailgating permission mode, it may cancel the tailgating permission for the face-unregistered person 1B who was permitted to tailgate before the face-registered person 1A performed the second action, as shown in Figure 30 (b), and may display, for example, a warning message.
[0241] When performing the process shown in FIG. 24 or FIG. 28, management device 50 may cancel the setting of tailgating determination zone 303 instead of canceling the above-described tailgating permission mode.
[0242] <Inputting Whether or Not to Allow Tailgating into Terminal> FIG. 31 is a diagram showing an example of a screen for inputting whether or not to allow tailgating into a terminal according to the second embodiment.
[0243] When the management device 50 detects a tailgating person 1C using the above-described method, it may not immediately grant permission for the tailgating person 1C, but may instead transmit information about the detected tailgating person 1C to a terminal 400 carried by a face-registered person 1A (e.g., an employee) whose face has been successfully matched. The information about the tailgating person 1C may include a facial image of the tailgating person 1C captured by the first image capturing device 21 (or the second image capturing device 22). Examples of the terminal 400 include a tablet terminal, a smartphone, a notebook PC, etc.
[0244] Using the information received from the management device 50, the terminal 400 may display on the screen 401, as shown in FIG. 31, a facial image 410 of the accompanying person 1C, a button to designate the accompanying person 1C as a target for tailgating (hereinafter referred to as the tailgating target button 411), and a button to designate the accompanying person 1C as an ineligible target for tailgating (hereinafter referred to as the ineligible target button 412).
[0245] The face-registered person 1A looks at the facial image 410 of the accompanying person 1C displayed on the screen 401 of the terminal 400, and if the accompanying person 1C is to be a target for accompanying (i.e., if the person is to be allowed to pass), the face-registered person 1A touches the target for accompanying button 411, and if the accompanying person 1C is not to be a target for accompanying (i.e., if the person is not to be allowed to pass), the face-registered person 1A touches the non-target for accompanying button 412.
[0246] In addition, Figure 31 displays a facial image 410 of one accompanying person 1C and corresponding buttons 411, 412, but if there are multiple accompanying people 1C, the facial image 410 of each accompanying person 1C and corresponding buttons 411, 412 may be displayed.
[0247] The terminal 400 transmits to the management device 50 information about the tailgating person 1C and information indicating which button 411 or 412 was pressed in response to the information.
[0248] The management device 50 may determine whether or not to permit passage for each tailgating person 1C based on the information received from the terminal 400. For example, the management device 50 displays a message indicating permission to tailgate for a tailgating person 1C who has touched the tailgating target button 411 on the first display device 31, and displays a warning or a message indicating a verification failure for a tailgating person 1C who has touched the tailgating target non-target button 412.
[0249] This allows the face-registered person 1A (e.g., an employee) who has successfully passed face matching to explicitly specify, by operating the terminal 400, a person (e.g., a guest) whom the management device 50 has determined to be a tailgating person 1C and whom the management device 50 wishes to permit passage of, thereby preventing a person who is not brought by the face-registered person 1A from being erroneously granted a passage permit.
[0250] <Consideration of Passage Authorization in addition to Facial Registration> The management device 50 is not limited to the method of determining whether to permit passage through the gate system 9 based on whether a facial image is registered, but may also manage the security level for each gate system 9. In this case, the management device 50 manages the security level assigned to each face-registered person 1A and the security level assigned to each gate system 9. If the security level assigned to the gate system 9 through which the face-registered person 1A is about to pass is equal to or lower than the security level assigned to the face-registered person 1A, the management device 50 may determine that the face-registered person 1A has passage authorization and may permit the person to pass, but if the security level assigned to the gate system 9 is higher than the security level assigned to the face-registered person 1A, the management device 50 may determine that the face-registered person 1A does not have passage authorization and may not permit the person to pass. In this case, if the management device 50 allows the face-registered person 1A to pass, it also assigns a pass permission flag to the face-unregistered person 1B who is accompanied by the face-registered person 1A, and if it does not allow the face-registered person 1A to pass, it does not need to assign a pass permission flag to the face-unregistered person 1B who is accompanied by the face-registered person 1A.
[0251] As a result, the management device 50 can allow the accompanying person 1C to pass through gate systems 9 for which the face-registered person 1A has passage authority, and can not allow the accompanying person 1C to pass through gate systems 9 for which the face-registered person 1A does not have passage authority.
[0252] <When there are multiple face-registered persons> In the above, for ease of understanding, the case where there is one face-registered person 1A has been described, but there may be multiple face-registered persons 1A. In this case, the management device 50 may perform any one of the following processes (A1) to (A5).
[0253] (A1) In the example shown in Figure 17, if there is at least one person with a registered face 1A in the tailgating determination zone 303, the management device 50 may determine YES in step S1206 of Figure 20, place the tailgating determination zone 303 in tailgating permission mode, and determine the person with an unregistered face 1B present in the tailgating determination zone 303 as a tailgating person 1C.
[0254] (A2) In the example shown in Fig. 22 , if at least one of face-registered persons 1A present in tailgating determination zone 303 is performing the first action, management device 50 may determine YES in step S1405 of Fig. 22 , place tailgating determination zone 303 in tailgating permission mode, and designate unregistered face person 1B present in tailgating determination zone 303 as tailgating person 1C. Alternatively, in the example shown in Fig. 22 , if all of face-registered persons 1A present in tailgating determination zone 303 are performing the first action, management device 50 may determine YES in step S1405 of Fig. 22 , place tailgating determination zone 303 in tailgating permission mode, and designate unregistered face person 1B present in tailgating determination zone 303 as tailgating person 1C.
[0255] (A3) In the example shown in Figure 23, the management device 50 sets tailgating determination zones 303 around each of the multiple face-registered persons 1A (S1506), and if a face-unregistered person 1B is present in any of the set tailgating determination zones 303, the face-unregistered person 1B may be determined to be a tailgating person 1C.
[0256] (A4) In the example shown in FIG. 25, if the distance L between any one of a plurality of face-registered persons 1A and an unregistered face person 1B is equal to or less than a predetermined threshold, and both the face-registered person 1A and the unregistered face person 1B perform a first action, the management device 50 may treat the unregistered face person 1B as a person 1C accompanying the face-registered person 1A.
[0257] (A5) In the example shown in FIG. 26, if the distance L between any one of a plurality of face-registered persons 1A and an unregistered person 1B within the tailgating determination zone 303 is less than a predetermined threshold, and both the face-registered person 1A and the unregistered person 1B perform a first action, the management device 50 may determine that the unregistered person 1B is the accompanying person 1C of the face-registered person 1A.
[0258] <Registering a facial image of a person with an unregistered face who has been permitted to tailgate> The management device 50 may register a facial image of the face of the tailgating person 1C who has been permitted to pass in the memory 1002 or the storage 1003 (see FIG. 32 ). This allows the management device 50 to treat the tailgating person 1C who was initially permitted to pass as an unregistered person 1B as a registered person 1A when passing through the aisle 2 from the next time onwards. This allows the first tailgating person 1C to pass through the aisle 2 from the next time onwards, even if he or she is not accompanied by the registered person 1A.
[0259] When determining whether or not the face-unregistered person 1B is a tailgating person 1C, the management device 50 may photograph the face of the face-unregistered person 1B using the first photographing device 21 or the second photographing device 22. If the face-unregistered person 1B is determined to be a tailgating person 1C, the management device 50 may register the photographed face image. At this time, the management device 50 determines whether or not the photographed face image is of a quality usable for face matching. If the quality is usable for face matching, the management device 50 may register the photographed face image. If the quality is not usable for face matching, the management device 50 may not register the photographed face image. For example, if the photographed face image is a face image wearing a mask, a face image with closed eyes, or a face image not facing forward, the management device 50 may determine that the photographed face image is of a quality usable for face matching. For example, if the photographed face image is a face image not wearing a mask, with eyes closed, and a face facing forward, the management device 50 may determine that the photographed face image is of a quality usable for face matching.
[0260] Furthermore, when registering the facial image of person 1B whose face has not been registered, management device 50 may display the facial image to be registered and a message indicating that the facial image will be registered on first display device 31 (or second display device 32). This allows tailgating person 1C to know that his or her facial image has been registered.
[0261] Whether or not to permit registration of the face image of the accompanying person 1C may be instructed by the face-to-be-registered person 1A who accompanies the accompanying person 1C.
[0262] (Hardware Configuration) FIG. 32 is a diagram illustrating an example of a hardware configuration of a computer according to the present disclosure.
[0263] The above-described first edge device 41, second edge device 42, management device 50, and face matching device 60 may be configured to include a computer 1000 shown in Fig. 32. Furthermore, the processes performed by the above-described first edge device 41, second edge device 42, management device 50, and face matching device 60 may be realized by the computer 1000 shown in Fig. 32 executing a computer program.
[0264] The computer 1000 includes a processor 1001, a memory 1002, a storage 1003, an input device 1004, an output device 1005, a communication device 1006, a graphics processing unit (GPU) 1007, a reading device 1008, and a bus 1009. The processor 1001, the memory 1002, the storage 1003, the input device 1004, the output device 1005, the communication device 1006, the GPU 1007, and the reading device 1008 are connected to the bus 1009 and can send and receive data bidirectionally via the bus 1009.
[0265] The processor 1001 executes computer programs stored in the memory 1002 to realize the processes performed by the above-mentioned devices. Therefore, the processes performed by the above-mentioned devices can be interpreted as processes mainly performed by the processor 1001. Examples of the processor 1001 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a controller, an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field-Programmable Gate Array).
[0266] The memory 1002 stores computer programs and data used by the computer 1000. The memory 1002 may include a read-only memory (ROM) and a random access memory (RAM). The memory 1002 may be configured as either a volatile storage medium or a non-volatile storage medium.
[0267] The storage 1003 is configured as a non-volatile storage medium and stores computer programs and data used by the computer 1000. Examples of the storage 1003 include a hard disk drive (HDD), a solid state drive (SSD), and flash memory.
[0268] The input device 1004 accepts data to be input to the processor 1001. Examples of the input device 1004 include a keyboard, a mouse, a touchpad, and a microphone.
[0269] The output device 1005 outputs data generated by the processor 1001. For example, the output device 1005 outputs display data generated by the processor 1001 to the first display device 31 or the second display device 32. Alternatively, the output device 1005 may be a display and display the display data itself. Alternatively, the output device 1005 outputs audio data generated by the processor 1001 to a speaker.
[0270] The communication device 1006 transmits and receives data to and from other devices via the communication network 4. The communication device 1006 may include a transmitter that transmits data and a receiver that receives data. The communication device 1006 may support either wired communication or wireless communication.
[0271] The GPU 1007 is a device that processes image rendering at high speed. The GPU 1007 may be used for AI processing (e.g., machine learning or deep learning).
[0272] The reading device 1008 is a device that reads data from an external recording medium such as a DVD-ROM (Digital Versatile Disk Read Only Memory) or a USB (Universal Serial Bus) memory.
[0273] (Summary of Second Embodiment) The above description of the second embodiment discloses the following techniques.
[0274] <Technology 1> A person matching system (e.g., gate system 9) includes: photographing devices (e.g., first photographing device 21, second photographing device 22) that photograph a plurality of persons (1) moving through a predetermined passageway (2); and information processing devices (e.g., management device 50, face matching device 60) that match the persons using facial images (107) of the persons photographed by the photographing devices; the information processing device sets a first area (e.g., face matching zone 301) and a second area (e.g., tailgating determination zone 303) in the passageway; in the first area, it performs matching using the facial images of persons present in the first area, permitting persons who are successfully matched to move through the passageway and not permitting persons who are unsuccessful to move through the passageway; in the second area, it performs matching using the facial images of persons present in the second area, permitting a first person who is successfully matched to move through the passageway and permitting a second person who is unsuccessful to move through the passageway while the first person is present in the second area.
[0275] According to the above configuration, when a first person (e.g., an employee) whose face is successfully matched moves down an aisle with a second person (e.g., a visitor) whose face is unsuccessful, the second person is permitted to move along the aisle by passing through the second area. This allows the first person to move down the aisle with the second person without having to request that the second person register their face image.
[0276] <Technology 2> In the person matching system described in Technology 1, the second area is set in advance within a predetermined range within the passageway, and the information processing device sets the second area to a state in which the second person is permitted to move along the passageway while the first person is present in the second area and when the first person performs a predetermined first action.
[0277] According to the above configuration, when the first person is present in the second area and performs the first action, the second person can pass through the second area, thereby reducing the possibility that the information processing device will allow a person other than the second person accompanied by the first person to move.
[0278] <Technology 3> In the person matching system described in Technology 1 or 2, when the first person performs a predetermined first action in the second area after the matching is successful, the information processing device sets a predetermined range including the first person as the second area, and sets the second area to a state in which the second person is permitted to move through the passage.
[0279] According to the above configuration, a second area is set around the first person, and the second person present in the second area is permitted to move along the passageway, thereby reducing the possibility that the information processing device will permit a person other than the second person accompanied by the first person to move.
[0280] <Technology 4> In the person matching system described in any one of Technologies 1 to 3, the information processing device allows the second person to move through the passage when the distance between the first person and the second person in the second area is equal to or less than a predetermined threshold.
[0281] According to the above configuration, a second person whose distance from the first person is equal to or less than a predetermined threshold is permitted to move along the passageway, thereby reducing the possibility that the information processing device will permit a person other than the second person accompanied by the first person to move along the passageway.
[0282] <Technology 5> The person matching system described in any one of Technologies 1 to 4 further includes a display device (e.g., a first display device 31, a second display device 32), and when the first person performs the first action, the information processing device causes the display device to display that the second area is in a state in which movement of the second person through the passage is permitted.
[0283] According to the above configuration, the first person and / or the second person can know from the display on the display device whether or not the second area is in a state in which movement is permitted.
[0284] <Technology 6> The person verification system according to any one of technologies 1 to 4 further includes a display device, wherein when the first person performs a predetermined second action while the second area is in a state where movement through the passage is permitted for the second person, the information processing device sets the second area to a state where movement through the passage is not permitted for the second person, and causes the display device to display that the second area is in a state where movement through the passage is not permitted for the second person.
[0285] According to the above configuration, the first person can switch the state of the passage to one in which movement is not permitted by performing the second action, and the first person and / or the second person can know from the display on the display device whether the second area is in a state in which movement is permitted.
[0286] <Technology 7> The person matching system according to any one of technologies 1 to 3 further includes a display device, and when the first person performs a predetermined second action while the second area is in a state where the second person is permitted to move through the passage, the information processing device cancels the setting of the second area and causes the display device to display a message indicating that the second person is not permitted to move through the passage.
[0287] According to the above configuration, the first person can switch the state of the passage to one in which movement is not permitted by performing the second action, and the first person and / or the second person can know from the display on the display device whether the second area is in a state in which movement is permitted.
[0288] <Technology 8> In the person verification system described in Technology 6 or 7, when the first person performs a predetermined second action in a state in which the second area permits the second person to move through the passage, the information processing device continues to maintain the state of tailgating permission for the second person who was permitted to tailgate before the first person performed the second action.
[0289] According to the above configuration, even if a first person performs a predetermined second action, the second person who was permitted to tailgate before the first person performed the second action can continue to maintain the state of being permitted to tailgate.
[0290] <Technology 9> In the person verification system described in Technology 6 or 7, when the first person performs a predetermined second action while the second area is in a state where movement in the passage is permitted for the second person, the information processing device cancels the state of tailgating permission for the second person who was permitted to tailgate before the first person performed the second action, and causes the display device to display a message indicating that movement in the passage is not permitted.
[0291] According to the above configuration, when a first person performs a second action, the tailgating permission state can be canceled for a second person who was permitted to tailgate before the first person performed the second action.
[0292] <Technology 10> The person matching system according to any one of technologies 1 to 9 further includes a terminal (400), and when the information processing device detects the second person in the second area, the information processing device transmits a facial image of the second person captured by the imaging device to the terminal.
[0293] According to the above configuration, a facial image of the second person detected in the second area is transmitted to the terminal, thereby allowing a person carrying the terminal (e.g., the first person) to know that the second person has been detected.
[0294] <Technology 11> The person matching system described in Technology 10 further includes a display device, wherein the terminal displays a facial image of the second person received from the information processing device, accepts an input as to whether or not to allow the second person to move through the passage, and transmits a result of the input to the information processing device, and the information processing device displays information related to the result of the input received from the terminal on the display device.
[0295] According to the above configuration, a person carrying the terminal (e.g., a first person) can determine which of the detected second persons is permitted to move along the passageway, thereby preventing the information processing device from permitting the movement of a person not accompanied by the first person.
[0296] <Technology 12> In the person matching system described in any one of Technologies 1 to 11, the information processing device permits the second person to move through the passage when the first person is successfully matched using the facial image and has authority to move through the passage.
[0297] According to the above configuration, if the first person whose face has been successfully matched has the authority to move through the passage, the second person is also permitted to move through the passage. As a result, if the first person does not have the authority to move through the passage, the second person accompanied by the first person is not permitted to move through the passage.
[0298] <Technology 13> In the person matching system described in any one of Technologies 1 to 3, the information processing device stores, in a predetermined storage unit, facial images of a quality that can be used for matching using the facial images, from among the facial images of the second person who has been permitted to move through the passage, which have been photographed by the photographing device.
[0299] According to the above configuration, among the facial images of the second person, the facial images that can be used for face matching are stored in the storage unit, so that from the next time onwards, the second person can move along the passageway without being accompanied by the first person, by using the facial images stored in the storage unit for face matching.
[0300] <Technology 14> The person verification system according to Technology 13 further includes a display device, and when the information processing device stores the facial image of the second person in the storage unit, it causes the display device to display that fact.
[0301] According to the above configuration, the second person can know that the face image of the second person has been stored in the storage unit.
[0302] <Technology 15> An information processing device that matches a plurality of people moving along a predetermined passageway using facial images of the people photographed by a photographing device that photographs the people includes a processor (1001) and a memory (1002), and the processor works in cooperation with the memory to set a first area and a second area in the passageway, and in the first area, performs matching using the facial images on people present in the first area, allowing people who are successfully matched to move along the passageway and not allowing people who are unsuccessful to move along the passageway, and in the second area, performs matching using the facial images on people present in the second area, allowing a first person who is successfully matched to move along the passageway and allowing a second person who is unsuccessful to match to move along the passageway while the first person is present in the second area.
[0303] According to the above configuration, when a first person (e.g., an employee) whose face is successfully matched moves down an aisle with a second person (e.g., a visitor) whose face is unsuccessful, the second person is permitted to move along the aisle by passing through the second area. This allows the first person to move down the aisle with the second person without having to request that the second person register their face image.
[0304] <Technology 16> A person matching method for matching a plurality of people moving along a predetermined passageway using facial images of the people photographed by a photographing device, includes setting a first area and a second area in the passageway, performing matching using the facial images on people present in the first area in the first area, allowing people who are successfully matched to move along the passageway, and not allowing people who are unsuccessful to match to move along the passageway, and performing matching using the facial images on people present in the second area in the second area, allowing a first person who is successfully matched to move along the passageway, and allowing a second person who is unsuccessful to match to move along the passageway while the first person is present in the second area.
[0305] According to the above method, when a first person (e.g., an employee) whose face is successfully matched moves down an aisle with a second person (e.g., a visitor) whose face is unsuccessful, the second person is permitted to move along the aisle by passing through the second area, thereby allowing the first person to move along the aisle with the second person without having to request that the second person register their facial image.
[0306] <Technology 17> A person matching program causes a computer to execute the person matching method described in Technology 14 above.
[0307] According to the person matching program, when a first person (e.g., an employee) whose face matches a face matching success moves down an aisle with a second person (e.g., a visitor) whose face matching failure occurs, the second person is permitted to move along the aisle by passing through the second area, thereby enabling the first person to move along the aisle with the second person without having to request that the second person register a facial image.
[0308] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner without departing from the spirit of the invention.
[0309] The technology of the present disclosure is useful for realizing a system, device, method, computer program, etc. that takes into account a person who does not have the right of passage being accompanied by a person who has the right of passage.
[0310] 1 Person 1A Face-registered person 1B Face-unregistered person 1C Tailgating person 2 Passage 3 Detection area 4 Communication network 9 Gate system 10 Gate device 11 First gate 12 Second gate 21 First image capturing device 22 Second image capturing device 31 First display device 32 Second display device 41 First edge device 42 Second edge device 50 Management device 60 Face matching device 100 Person information 102 First image capturing time 103 Image update time 104 Body image 105 Body image update time 106 Body coordinates 107 Face image 108 Face image update time 109 Face coordinates 110 Face matching ID 111 Matching status 112 Movement direction 200 Body image 201, 201A, 201B Body coordinates 202 Body coordinates 220 Face image 221, 221A Face coordinates 301 Face matching zone 302 Passage determination zone 303 Tailgating determination zone 311 Predetermined area closer to first gate 312 Predetermined area closer to second gate 400 Terminal 401 Screen 411 Tailgating target button 412 Tailgating non-target button 1000 Computer 1001 Processor 1002 Memory 1003 Storage 1004 Input device 1005 Output device 1006 Communication device 1007 GPU 1008 Reading device 1009 Bus
Claims
1. A person matching system comprising: a photographing device which photographs a number of people moving through a specified passage; and an information processing device which matches the people using facial images of the people photographed by the photographing device, wherein the information processing device sets a first area and a second area in the passage; in the first area, it performs a match using the facial images of people present in the first area, allowing people who are successfully matched to move through the passage and not allowing people who are unsuccessful to move through the passage; and in the second area, it performs a match using the facial images of people present in the second area, allowing a first person who is successfully matched to move through the passage, and allowing a second person who is unsuccessful to match to move through the passage while the first person is present in the second area.
2. The person matching system of claim 1, wherein the second area is preset within a predetermined range within the passageway, and the information processing device sets the second area to a state in which the second person is permitted to move along the passageway while the first person is present in the second area and when the first person performs a predetermined first action.
3. The person matching system of claim 1, wherein the information processing device, when the first person performs a predetermined first action in the second area after the matching is successful, sets a predetermined range including the first person as the second area, and sets the second area to a state in which movement of the second person through the passage is permitted.
4. The person matching system according to any one of claims 1 to 3, wherein the information processing device permits the second person to move along the passage when a distance between the first person and the second person in the second area is equal to or less than a predetermined threshold.
5. The person matching system of claim 2 or 3, further comprising a display device, wherein the information processing device, when the first person performs the first action, causes the display device to display that the second area is in a state that allows the second person to move through the passage.
6. A person matching system as described in claim 2 or 3, further comprising a display device, wherein when the first person performs a predetermined second action while the second area is in a state in which movement along the passage is permitted for the second person, the information processing device sets the second area to a state in which movement along the passage is not permitted for the second person, and causes the display device to display that the second area is in a state in which movement along the passage is not permitted for the second person.
7. The person matching system of claim 3, further comprising a display device, wherein the information processing device cancels the setting of the second area and causes the display device to display a message indicating that the second person is not permitted to move along the passage when the first person performs a predetermined second action while the second area is in a state in which the second person is permitted to move along the passage.
8. The person matching system of claim 6, wherein when the first person performs a predetermined second action while the second area is in a state in which movement of the second person through the passage is permitted, the information processing device continues to maintain the state of tailgating permission for the second person who was permitted to tailgate before the first person performed the second action.
9. The person matching system of claim 6, wherein when the first person performs a predetermined second action while the second area is in a state in which movement in the passage is permitted for the second person, the information processing device cancels the state of tailgating permission for the second person who was permitted to tailgate before the first person performed the second action, and causes the display device to display a message indicating that movement in the passage is not permitted.
10. A person matching system as described in any one of claims 1 to 3, further comprising a terminal, wherein when the information processing device detects the second person in the second area, the information processing device transmits a facial image of the second person captured by the photographing device to the terminal.
11. The person matching system of claim 10, further comprising a display device, wherein the terminal displays a facial image of the second person received from the information processing device, accepts input as to whether or not to allow the second person to move through the passage, and transmits a result of the input to the information processing device, and the information processing device causes information relating to the result of the input received from the terminal to be displayed on the display device.
12. The person matching system according to any one of claims 1 to 3, wherein the information processing device permits the second person to move through the passage when the first person is successfully matched using the facial image and has authority to move through the passage.
13. The person matching system according to any one of claims 1 to 3, wherein the information processing device stores in a specified memory unit, among the facial images of the second person who has been permitted to move through the passageway and which have been photographed by the photographing device, facial images of a quality usable for matching using the facial images.
14. The person matching system according to claim 13, further comprising a display device, wherein when the information processing device stores the facial image of the second person in the memory unit, it causes the display device to display a message to that effect.
15. An information processing device that matches multiple people moving through a specified passageway using facial images of the people captured by a photographing device, the information processing device having a processor and memory, the processor working in cooperation with the memory to: set a first area and a second area in the passageway, in the first area, perform a match using the facial images of people present in the first area, allowing people who are successfully matched to move through the passageway and not allowing people who are unsuccessful to match to move through the passageway, and in the second area, perform a match using the facial images of people present in the second area, allowing a first person who is successfully matched to move through the passageway and allowing a second person who is unsuccessful to match to move through the passageway while the first person is present in the second area.
16. A person matching method for matching multiple people moving along a specified passageway using facial images of the people photographed by an imaging device, the method comprising: setting a first area and a second area in the passageway; in the first area, matching is performed on people present in the first area using the facial images, allowing people who are successfully matched to move along the passageway and not allowing people who are unsuccessful to match to move along the passageway; and in the second area, matching is performed on people present in the second area using the facial images, allowing a first person who is successfully matched to move along the passageway and allowing a second person who is unsuccessful to match to move along the passageway while the first person is present in the second area.
17. A person matching program for causing a computer to execute the person matching method according to claim 16.
Citation Information
Patent Citations
Information processing device
JP2020086780A
Entrance management system
JP2020088462A
Image processing apparatus, image processing system, image processing method, and program
JP2022053683A
Information processing device, information processing system, and passage management method
JP2022093489A
Frictionless building access control system with tailgate detection
US20190355193A1
