Person collation system, information processing device, person collation method, and person collation program
The system addresses tailgating issues by separating verification areas, enabling smooth passage for authorized individuals even if others fail, thus preventing unauthorized entry and reducing congestion.
Patent Information
- Application Number
- JP2023085542
- Authority / Receiving Office
- JP · JP
- Patent Type
- Applications
- Current Assignee / Owner
- Filing Date
- 2023-05-24
- Publication Date
- 2024-12-05
AI Technical Summary
Existing room entry control systems allow unauthorized individuals to enter by tailgating with authorized individuals, causing congestion and inefficiency as all individuals are forced to wait if one person's verification fails.
A person verification system that sets separate areas for tailgating determination, allowing successful verifications to proceed while unsuccessful verifications are managed separately, using facial image matching and display devices to guide individuals through the process.
This system prevents unauthorized entry while maintaining efficient flow by allowing authorized individuals to move freely, even if others' verification fails, reducing congestion and improving access control.
Smart Images

Figure 2024168668000001_ABST
Abstract
Description
[Technical Field]
[0001] The present disclosure relates to a person matching system, an information processing device, a person matching method, and a person matching program. [Background technology]
[0002] Patent Document 1 discloses an entry management system. The entry management system includes a door opening / closing control unit that controls the opening and closing of a managed door, an imaging unit that acquires a monitoring image of a monitored space near the managed door, a person authentication unit that authenticates a person present in the monitored space from the image acquired by the imaging unit, a permission determination unit that determines whether the person authenticated by the person authentication unit is a person who will be permitted to enter by opening or closing the managed door, and a specific action authentication unit that authenticates a specific action that is continuously performed for a predetermined time by a person who will be permitted to enter by the permission determination unit. If the specific action is authenticated by the specific action authentication unit, the door opening / closing control unit opens the managed door and permits tailgating entry, in which the person who will be permitted to enter enters the room accompanied by a person who is not permitted to enter, and if the specific action is not authenticated by the specific action authentication unit, the door does not open the managed door and does not permit tailgating entry, in which the person who will be permitted to enter enters the room accompanied by a person who is not permitted to enter. [Prior art documents] [Patent documents]
[0003] [Patent Document 1] Japanese Patent Publication No. 2020-88462 Summary of the Invention [Problem to be solved by the invention]
[0004] However, the entry management system disclosed in Patent Document 1 opens the managed door when the specific action of the person who is allowed to enter is authenticated, so at the moment the specific action of the person who is allowed to enter is authenticated, there is a risk that a person (hereinafter referred to as an unauthorized passerby) other than the person who the person who is allowed to enter is accompanying (hereinafter referred to as an accompanying person) may enter through the managed door.
[0005] Furthermore, in the entry management system disclosed in Patent Document 1, a method can be considered to avoid the possibility of mistakenly flagging an unauthorized person as permitted to enter by having people other than the person accompanying the person wait until the person who is permitted to enter has completed their entry while the person who is permitted to enter performs the specific action described above. However, with the above method, other people who are permitted to enter and who are not unauthorized people are also forced to wait to enter, causing congestion in front of the door to be managed.
[0006] An object of the present disclosure is to provide a technology for setting an area on an aisle where tailgating determination is performed separately from an area where tailgating determination is not performed, in a system that allows passage of a tailgating person. [Means for solving the problem]
[0007] The person matching system according to the present disclosure includes a photographing device that photographs a plurality of people moving through a predetermined passageway, and an information processing device that matches the people using facial images of the people photographed by the photographing device, wherein the information processing device sets a first area and a second area in the passageway, and in the first area, performs matching using the facial images on people present in the first area, allowing people who are successfully matched to move through the passageway and not allowing people who are unsuccessful to match to move through the passageway, and in the second area, performs matching using the facial images on people present in the second area, allowing a first person who is successfully matched to move through the passageway and allowing a second person who is unsuccessful to match to move through the passageway while the first person is present in the second area.
[0008] The information processing device according to the present disclosure is an information processing device that matches multiple people moving down a specified passageway using facial images of the people photographed by a photographing device that photographs the people, and is equipped with a processor and a memory, and the processor works in cooperation with the memory to set a first area and a second area in the passageway, and in the first area, performs a match using the facial images on people present in the first area, allowing people who are successfully matched to move down the passageway and not allowing people who are unsuccessful to move down the passageway, and in the second area, performs a match using the facial images on people present in the second area, allowing a first person who is successfully matched to move down the passageway and allowing a second person who is unsuccessful to move down the passageway while the first person is present in the second area.
[0009] A person matching method according to the present disclosure is a person matching method for matching a plurality of people moving along a predetermined passageway using facial images of the people photographed by a photographing device that photographs the people, the method including: setting a first area and a second area in the passageway; in the first area, matching is performed using the facial images of people present in the first area, and people who are successfully matched are permitted to move along the passageway, and people who are unsuccessful in the matching are not permitted to move along the passageway; and in the second area, matching is performed using the facial images of people present in the second area, and a first person who is successfully matched is permitted to move along the passageway, and a second person who is unsuccessful in the matching is permitted to move along the passageway while the first person is present in the second area.
[0010] A person matching program according to the present disclosure causes a computer to execute the person matching method described above.
[0011] These comprehensive or specific aspects may be realized as a system, an apparatus, a method, an integrated circuit, a computer program, or a recording medium, or may be realized as any combination of a system, an apparatus, a method, an integrated circuit, a computer program, and a recording medium. [Effects of the Invention]
[0012] According to the present disclosure, in a system that permits passage of tailgating persons, an area in an aisle where tailgating determination is performed can be set separately from an area where tailgating determination is not performed. As a result, when a person whose face matches successfully (e.g., an employee) moves down the aisle accompanied by a person whose face matches unsuccessfully (e.g., a visitor), the person passes through the area where tailgating determination is performed, thereby maintaining a comfortable progress situation in the aisle while preventing the system from granting passage to persons who are not tailgating persons and enabling detection of unauthorized passersby. [Brief explanation of the drawings]
[0013] [Figure 1] FIG. 1 is a diagram for explaining an overview of a gate system according to a first embodiment. [Figure 2] FIG. 10 is a diagram showing an example of displaying a matching result on the first display device or the second display device according to the first embodiment; [Figure 3] A block diagram showing a configuration example of a gate system according to a first embodiment. [Figure 4] FIG. 1 is a diagram for explaining a face image and a body image according to the first embodiment. [Figure 5] 1 is a flowchart showing an example of a detection process performed by a first edge device according to the first embodiment; [Figure 6] FIG. 1 shows an example of the configuration of person information according to the first embodiment. [Figure 7] 1 is a flowchart showing an example of overall processing performed by a management device according to a first embodiment; [Figure 8] Flowchart showing an example of face detection information processing (S202) shown in FIG. 7 [Figure 9] Flowchart showing an example of the body detection information processing (S204) shown in FIG. 7 [Figure 10] A flowchart showing an example of post-movement processing (S210) shown in FIG. 7. [Figure 11] Flowchart showing an example of face matching result information processing (S206) shown in FIG. 7 [Figure 12] 1 is a flowchart showing an example of a display process performed by a first edge device according to the first embodiment; [Figure 13]FIG. 1 is a diagram showing a face matching zone and a passage determination zone in a detection area according to the first embodiment; [Figure 14] FIG. 1 is a diagram for explaining an example of a display pattern in a first display device according to embodiment 1; [Figure 15A] FIG. 1 is a first diagram illustrating a method for detecting, from a personal information table, personal information that is presumed to belong to the same person as the person indicated by the body detection information according to the first embodiment; [Figure 15B] FIG. 2 is a second diagram for explaining a method for detecting, from the personal information table, personal information that is presumed to belong to the same person as the person indicated by the body detection information according to the first embodiment; [Figure 16] FIG. 1 is a diagram illustrating a method for detecting a person's moving direction according to the first embodiment. [Figure 17] FIG. 10 is a diagram showing an example in which a face matching zone, a passage determination zone, and a tailgating determination zone are set in a detection area according to the second embodiment; [Figure 18] 10 is a flowchart showing an example of processing performed by the management device when a face matching zone, a tailgating determination zone, and a passage determination zone are set in the detection area according to the second embodiment. [Figure 19] 10 is a flowchart showing an example of face matching processing according to the second embodiment. [Figure 20] Flowchart showing a first example of face matching and tailgating determination processing according to the second embodiment [Figure 21] Flowchart showing an example of a passage determination process according to the second embodiment [Figure 22] 10 is a flowchart showing a second example of face matching and tailgating determination processing according to the second embodiment. [Figure 23] FIG. 10 is a diagram showing an example of setting a tailgating determination zone around a face-registered person according to the second embodiment; [Figure 24] Flowchart showing a third example of face matching and tailgating determination processing according to the second embodiment [Figure 25] FIG. 10 is a diagram for explaining a first method for making a face-unregistered person a tailgating person by having a face-registered person and a face-unregistered person perform cooperative actions according to the second embodiment; [Figure 26]FIG. 10 is a diagram for explaining a second method for making a face-unregistered person a tailgating person by having a face-registered person and a face-unregistered person perform cooperative actions according to the second embodiment. [Figure 27] FIG. 27 is a diagram showing an example of a display on the first display device when the process shown in FIG. 25 or FIG. 26 is performed. [Figure 28] Flowchart showing a fourth example of face matching and tailgating determination processing according to the second embodiment [Figure 29] FIG. 10 is a diagram showing a first display example of the first display device when the tailgating permission mode according to the second embodiment is cancelled; [Figure 30] FIG. 10 is a diagram showing a second display example of the first display device when the tailgating permission mode according to the second embodiment is cancelled; [Figure 31] FIG. 10 is a diagram showing an example of a screen for inputting whether or not to permit tailgating on a terminal according to the second embodiment; [Figure 32] FIG. 1 is a diagram illustrating an example of a hardware configuration of a computer according to the present disclosure. DETAILED DESCRIPTION OF THE INVENTION
[0014] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings. However, more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially identical configurations may be omitted. This is to avoid unnecessary redundancy in the following description and to facilitate understanding by those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.
[0015] (Embodiment 1) <Gate system overview> Fig. 1 is a diagram illustrating an overview of a gate system 9 according to embodiment 1. Fig. 2 is a diagram illustrating an example of how a matching result is displayed on the first display device 31 or the second display device 32 according to embodiment 1.
[0016] The gate system 9 is a system that collates a facial image of a person 1 and determines whether the person 1 has the right to pass through the gate. The gate system 9 includes a first gate 11 and a second gate 12. The first gate 11 and the second gate 12 are installed a predetermined distance apart in a passage 2 through which the person 1 moves. In this embodiment, the direction from the second gate 12 toward the first gate 11 is defined as a first direction, and the direction from the first gate 11 toward the second gate 12 is defined as a second direction. In this embodiment, the area between the first gate 11 and the second gate 12 in the passage 2 is referred to as a person detection area 3. Note that the gate system 9 does not necessarily have to have the first gate 11 and the second gate 12 under which a person passes as shown in FIG. 1 , and may have a configuration as shown in FIG. 36 described below. In this case, the gate system 9 may set a virtual first line instead of the first gate 11 and a virtual second line instead of the second gate 12, and the area between the first line and the second line may be defined as the detection area 3. The direction from the first line to the second line may be referred to as the first direction, and the direction from the second line to the first line may be referred to as the second direction. Therefore, in the following description, the first gate 11 may be read as the first line, and the second gate 12 may be read as the second line.
[0017] For ease of explanation, the present embodiment will be described assuming that the axis along the first or second direction is the Y-axis, the axis perpendicular to the Y-axis and extending in the height direction is the Z-axis, and the axis perpendicular to the Y-axis and the Z-axis is the X-axis. The X-axis direction may also be referred to as the width direction of the gate.
[0018] The first gate 11 is equipped with a first imaging device 21 capable of capturing an image of at least a portion of the detection area 3, and a first display device 31 having a screen facing the second gate 12. In FIG. 1, the first gate 11 is shown equipped with three first imaging devices 21, but the first gate 11 may be equipped with one or two first imaging devices 21, or four or more first imaging devices 21. In addition, in FIG. 1, the first gate 11 is shown equipped with one first display device 31, but the first gate 11 may be equipped with two or more first display devices 31.
[0019] The second gate 12 is equipped with a second imaging device 22 capable of capturing an image of at least a portion of the detection area 3, and a second display device 32 having a screen facing the first gate 11. In FIG. 1, the second gate 12 is shown equipped with three second imaging devices 22, but the second gate 12 may be equipped with one or two second imaging devices 22, or four or more second imaging devices 22. In addition, in FIG. 1, the second gate 12 is shown equipped with one second display device 32, but the second gate 12 may be equipped with two or more second display devices 32.
[0020] In this embodiment, the first direction is the direction of entry into a facility or the like where the person has the right to enter or exit, and the second direction is the direction of exit from the facility or the like. However, the first direction may also be the direction of exit from a facility or the like, and the second direction may also be the direction of entry into the facility or the like.
[0021] The gate system 9 compares the facial image of the person 1 moving in the first direction captured by the first photographing device 21 at the first gate 11 with each pre-registered facial image for comparison, and displays the comparison result on the first display device 31, as shown in FIG. 2.
[0022] The gate system 9 compares the facial image of the person 1 moving in the second direction captured by the second photographing device 22 at the second gate 12 with each pre-registered facial image for comparison, and displays the comparison result on the second display device 32, as illustrated in FIG. 2.
[0023] The gate width of the first gate 11 and the second gate 12, or the width in the X direction of the detection area 3, may be a width that allows multiple people 1 to pass through side by side. As a result, the gate system 9 according to this embodiment allows more people 1 to enter and exit smoothly compared to a gate system with a gate width that allows only one person to pass through. For example, in a gate system with a gate width wide enough for one person, if the verification of a person 1 ahead fails, the following people 1 will be held up. However, in a gate system 9 according to this embodiment with a gate width that allows multiple people 1 to pass through side by side, even if the verification of a person 1 ahead fails, the following people 1 can simply move around the person 1 ahead who failed to be verified, and thus can pass through the gate smoothly without being held up. In other words, the gate system 9 according to this embodiment can prevent congestion when entering or exiting a facility, etc. The gate system 9 according to this embodiment will be described in detail below.
[0024] <Gate system configuration> Fig. 3 is a block diagram showing an example of the configuration of the gate system 9 according to the embodiment 1. Fig. 4 is a diagram for explaining face images and body images according to the embodiment 1.
[0025] The gate system 9 includes a first gate 11 and a second gate 12. The first gate 11 includes a first imaging device 21 and a first display device 31. The second gate 12 includes a second imaging device 22 and a second display device 32. The first gate 11, the first imaging device 21, the first display device 31, the second gate 12, the second imaging device 22, and the second display device 32 may be collectively referred to as a gate device 10.
[0026] The gate system 9 further includes a first edge device 41, a second edge device 42, a management device 50, and a face matching device 60. The first image capturing device 21 is connected to the first edge device 41 via an electrical cable (e.g., a USB cable or an Ethernet (registered trademark) cable). The first display device 31 is connected to the first edge device 41 via an electrical cable (e.g., an HDMI (registered trademark) cable). The second image capturing device 22 is connected to the second edge device 42 via an electrical cable (e.g., a USB cable or an Ethernet cable). The second display device 32 is connected to the second edge device 42 via an electrical cable (e.g., an HDMI cable). The first edge device 41 and the second edge device 42 are connected to the management device 50 via a communication network 4. The management device 50 is connected to the face matching device 60 via the communication network 4. Examples of the communication network 4 include a wired LAN, a wireless LAN, a mobile communication network (e.g., LTE, 4G, 5G), the Internet, and / or a VPN.
[0027] The first image capturing device 21 captures an image of the detection area 3 to generate a captured image, and transmits the captured image to the first edge device 41. The second image capturing device 22 captures an image of the detection area 3 to generate a captured image, and transmits the captured image to the second edge device 42. The captured image may be a still image or a moving image.
[0028] As shown in FIGS. 4(a) and 4(b), the first edge device 41 detects the entire body of the person 1 from the photographed image received from the first image capturing device 21. In this embodiment, the image of the entire body of the person 1 detected from the photographed image is referred to as a body image 200. The first edge device 41 transmits the detected body image 200 to the management device 50. Furthermore, as shown in FIG. 4(a), the first edge device 41 detects the face portion of the person 1 from the photographed image received from the first image capturing device 21. In this embodiment, the image of the face portion of the person 1 detected from the photographed image is referred to as a face image 220. The first edge device 41 transmits the detected face image 220 to the management device 50.
[0029] As shown in FIGS. 4(a) and 4(b), the second edge device 42 detects the entire body of the person 1 from the photographed image received from the second image capturing device 22. The second edge device 42 transmits a body image 200 detected from the photographed image to the management device 50. Also, as shown in FIG. 4(a), the second edge device 42 detects the face portion of the person 1 from the photographed image received from the second image capturing device 22. The second edge device 42 transmits a face image 220 detected from the photographed image to the management device 50. Details of the processes performed by the first edge device 41 and the second edge device 42 will be described later.
[0030] The management device 50 generates person information 100 (see FIG. 6) corresponding to the person 1 located in the detection area 3 based on the body image 200 or face image 220 received from the first edge device 41, or the body image 200 or face image 220 received from the second edge device 42. The management device 50 may generate person information 100 for each person 1 detected in the detection area 3. Details of the person information 100 will be described later (see FIG. 6). The management device 50 transmits a face matching request including the face image 220 to the face matching device 60.
[0031] The face matching device 60 stores in advance a face image for matching of a person 1 who has the right to pass through a gate (for example, to enter or exit). For example, the face matching device 60 stores a face image for matching of a person 1 who has already registered to enter the facility. The face matching device 60 may manage the face image for matching by linking it to the name, age, sex, credit card information, etc. of the person 1.
[0032] Furthermore, the face matching device 60 matches the face image 220 received from the management device 50 with each of the face images for matching. If the face matching device 60 is successful in matching with any of the registered face images for matching, it generates face matching result information including a successful matching result and transmits it to the management device 50. At this time, the face matching device 60 may perform billing in real time using credit card information linked to the successfully matched face image for matching. If the face matching device 60 fails to match with any of the registered face images for matching, it generates face matching result information including a failed matching result and transmits it to the management device 50. Note that the matching between the face image 220 and the face images for matching may be achieved by a known face image matching technology. Also, "matching" may be read as "authentication."
[0033] The management device 50 receives face matching result information from the face matching device 60. The management device 50 transmits person matching information including face matching result information for the face image 220 of the person 1 moving in the first direction to the first edge device 41, and transmits person matching information including face matching result information for the face image 220 of the person 1 moving in the second direction to the second edge device 42.
[0034] When the first edge device 41 receives person matching information from the management device 50, it displays the face matching result information included in the person matching information on the first display device 31, as shown in Fig. 2. This allows a person 1 located in the detection area 3 and moving in the first direction to know his / her own matching result by looking at the screen of the first display device 31 located in the direction of movement. In other words, the first edge device 41 allows a person 1 moving in the first direction to visually recognize the matching result of the person 1.
[0035] When the second edge device 42 receives person matching information from the management device 50, it displays the face matching result information included in the person matching information on the second display device 32, as shown in FIG. 2. This allows a person 1 located in the detection area 3 and moving in the second direction to view the screen of the second display device 32 located at the end of the moving direction to know his / her own matching result. In other words, the second edge device 42 allows a person 1 moving in the second direction to visually recognize the matching result of the person 1. Below, the processes performed by the first edge device 41, the second edge device 42, and the management device 50 will be described in detail.
[0036] <Detection process in first edge device> 5 is a flowchart showing an example of the detection process performed by the first edge device 41 according to embodiment 1. Note that, for the detection process performed by the second edge device 42 according to embodiment 1, in the following description with reference to FIG. 5, the "first image capturing device 21" can be read as the "second image capturing device 22."
[0037] The first edge device 41 receives a captured image from the first image capturing device 21 (S101).
[0038] The first edge device 41 receives a distance image from the first image capturing device 21 (S102). In this way, the first image capturing device 21 may generate a distance image indicating the distance to the person 1 or the like present in the detection area 3. The same applies to the second image capturing device 22. The distance image may be generated using any known technology, such as a stereo camera or a distance measuring sensor (e.g., LiDAR (Light Detection And Ranging)). Note that in this embodiment, the processing of step S102 is performed after the processing of step S101, but this is not limiting and the order may be reversed.
[0039] As shown in Fig. 4(a), the first edge device 41 detects a facial image 220 of the person 1 from the captured image (S103). The detection of the facial image 220 of the person may be realized using any known technology, such as pattern matching or AI (Artificial Intelligence). The AI may also be an image recognition technology using deep learning (e.g., CNN (Convolutional Neural Network)).
[0040] The first edge device 41 determines whether or not the face image 220 was detected from the captured image in step S103 (S104). If the face image 220 was not detected (S104: NO), the first edge device 41 proceeds to step S108.
[0041] If the face image 220 is detected (S104: YES), the first edge device 41 uses the distance image to calculate three-dimensional coordinates (hereinafter referred to as face coordinates 221 (see FIG. 4)) for the detected face image 220 (S105). For example, as shown in FIG. 4(a), the first edge device 41 calculates the center point of the face image 220 as the face coordinates 221.
[0042] The first edge device 41 generates face detection information (S106). For example, the first edge device 41 generates face detection information including an ID (hereinafter referred to as a photographing device ID) for identifying the first photographing device 21 that is the sender of the photographed image, a face image 220, a size of the face image 220 (hereinafter referred to as a face image size), face coordinates 221, and a photographing time of the photographed image.
[0043] The first edge device 41 transmits the generated face detection information to the management device 50 (S107), and the process proceeds to the next step S108.
[0044] As shown in FIG. 4(a) or 4(b), the first edge device 41 detects a body image 200 of the person 1 from the captured image (S108). The detection of the body image 200 of the person 1 may be realized using any known technology, such as pattern matching or AI. The AI may also be an image recognition technology using deep learning (e.g., CNN).
[0045] The first edge device 41 determines whether or not the body image 200 was detected from the captured image in step S108 (S109). If the body image 200 was not detected (S109: NO), the first edge device 41 returns the process to step S101.
[0046] If the body image 200 is detected (S109: YES), the first edge device 41 uses the distance image to calculate three-dimensional coordinates (hereinafter referred to as body coordinates 201 (see FIG. 4)) of the detected body image 200 (S110). For example, the first edge device 41 calculates the center point of the body image 200 as the body coordinates 201, as shown in FIGS. 4(a) and 4(b).
[0047] The first edge device 41 generates body detection information (S111). For example, the first edge device 41 generates body detection information including an imaging device ID, a body ID for identifying the body image 200, the body image 200, body coordinates 201, and the imaging time of the captured image.
[0048] The first edge device 41 transmits the generated body detection information to the management device 50 (S112), and the process returns to step S101. Note that in this embodiment, the process of detecting a face image from the captured image (S103 to S107) is performed before the process of detecting a body image from the captured image (S108 to S112), but this is not limiting and the steps may be reversed.
[0049] In this way, by performing the process of detecting the face image 220 and / or the body image 200 of the person 1 from the captured image and the process of calculating the face coordinates 221 and / or the body coordinates 201 in the first edge device 41 or the second edge device 42, the processing load can be distributed compared to when these processes are performed in the management device 50. Note that if distribution of the processing load is not a consideration, the gate system 9 does not need to include the first edge device 41 and the second edge device 42. In this case, the first image capturing device 21, the first display device 31, the second image capturing device 22, and the second display device 32 may be directly connected to the management device 50. Then, the management device 50 may perform the process of detecting the face image 220 and / or the body image 200 of the person 1 from the captured image and the process of calculating the face coordinates 221 and / or the body coordinates 201.
[0050] <Person information> FIG. 6 shows an example of the configuration of the person information 100 according to the first embodiment.
[0051] Person information 100 is generated for each person 1 detected in the detection area 3. The person information 100 includes, as information or data, a tracking ID 101, a first image capture time 102, an image update time 103, a body image 104, a body image update time 105, body coordinates 106, a face image 107, a face image update time 108, face coordinates 109, a face matching ID 110, a matching status 111, and a movement direction 112.
[0052] The tracking ID 101 is set to an ID used to track the same person in the detection area 3. The tracking ID 101 can also be said to be an ID for identifying the person information 100.
[0053] The first image capturing time 102 is set to the time when the body image 104 or face image 107 included in the personal information 100 was first captured. The first image capturing time can also be said to be the time when the personal information 100 was first generated.
[0054] The image update time 103 is set to the time when the body image 104 or face image 107 included in the personal information 100 was last updated. The image update time 103 can also be said to be the time when the personal information 100 was last updated.
[0055] The body image 104 is set to a body image 200 detected from a photographed image of the person 1 indicated by the personal information 100. The body image 104 may be updated as needed to the body image 200 detected from the latest photographed image.
[0056] The body image update time 105 is set to the time when the body image 104 included in the personal information 100 was last updated.
[0057] The body coordinates 201 calculated based on the body image 104 are set as the body coordinates 106. The body coordinates 106 may be expressed as three-dimensional coordinates in the detection area 3 (X coordinate, Y coordinate, Z coordinate).
[0058] A face image 220 detected from a photographed image of person 1 indicated by the personal information 100 is set as the face image 107. The face image 107 may be updated as needed to the face image 220 detected from the latest photographed image.
[0059] The face image update time 108 is set to the time when the face image 107 included in the person information 100 was last updated.
[0060] The face coordinates 109 are set to face coordinates 221 calculated based on the face image 107. The face coordinates 109 may be expressed as three-dimensional coordinates (X coordinate, Y coordinate, Z coordinate) in the detection area 3.
[0061] An ID assigned to a successfully matched face image for matching is set in face matching ID 110. In other words, face matching ID 110 can also be said to be an ID that certifies person 1 corresponding to face image 107. If there is no successfully matched face image for matching (i.e., if face matching fails), NULL may be set in face matching ID 110.
[0062] The matching status 111 is set to the matching status for the face image 107. For example, the matching status 111 is set to one of "before matching request", "matching in progress", "matching successful", and "matching failed".
[0063] The moving direction 112 is set to the moving direction of the person 1 indicated by the person information 100. For example, the moving direction is set to either the first direction or the second direction.
[0064] The management device 50 manages a plurality of pieces of personal information 100 as a personal information table (not shown). The personal information table may be stored in a memory 1002 or a storage 1003 (see FIG. 37 ) included in the management device 50, or may be stored in a device different from the management device 50.
[0065] <Overall processing in the management device> FIG. 7 is a flowchart showing an example of the overall process performed by the management device 50 according to the first embodiment.
[0066] The management device 50 determines whether or not face detection information (see step S107 in FIG. 5) has been received from the first edge device 41 or the second edge device 42 (S201).
[0067] When the management device 50 receives face detection information (S201: YES), the management device 50 executes face detection information processing (S202). Details of the face detection information processing will be described later (see FIG. 8).
[0068] If the management device 50 has not received face detection information (S202: NO), it determines whether or not it has received body detection information (see step S112 in FIG. 5) from the first edge device 41 or the second edge device 42 (S203).
[0069] When the management device 50 receives the body detection information (S203: YES), the management device 50 executes body detection information processing (S204). Details of the body detection information processing will be described later (see FIG. 9).
[0070] If the management device 50 has not received the body detection information (S203: NO), it determines whether or not it has received face matching result information (S205). The face matching result information is information returned from the face matching device 60 in response to a face matching request transmitted from the management device 50 to the face matching device 60 in Fig. 8, which will be described later, and includes the result of face matching (for example, matching success or matching failure).
[0071] When the management device 50 receives the face matching result information (S205: YES), it executes face matching result information processing (S206). The face matching result information processing will be described in detail later (see FIG. 11).
[0072] If the management device 50 has not received the face matching result information (S205: NO), the processing returns to step S201.
[0073] After executing face detection information processing (S202) or body detection information processing (S204), the management device 50 executes post-movement processing (S210) and returns the processing to step S201. Details of the post-movement processing will be described later (see FIG. 10). Note that in this embodiment, the processing of steps S201, S203, and S205 is performed in stages, but this is not limiting, and the processing of steps S201, S203, and S205 may be interchanged or may be performed simultaneously in parallel.
[0074] <Face detection information processing in the management device> FIG. 8 is a flowchart showing an example of the face detection information processing (S202) shown in FIG.
[0075] The management device 50 detects, from the person information table, the person information 100 corresponding to the face detection information received from the first edge device 41 or the second edge device 42 (S301). In this embodiment, the person information 100 corresponding to the face detection information is person information 100 that is presumed to be the same person as person 1 indicated by the face detection information. Note that a method for detecting, from the person information table, the person information 100 that is presumed to be the same person as person 1 indicated by the face detection information will be described in detail later (see FIGS. 15A and 15B).
[0076] The management device 50 determines whether or not the personal information 100 has been detected from the personal information table in step S301 (S302).
[0077] Next, a case where it is determined in step S302 that the personal information 100 cannot be detected from the personal information table (S302: NO) will be described.
[0078] The management device 50 generates new personal information 100 based on the face detection information (S303). At this time, the management device 50 sets the following content in the new personal information 100. Set a new tracking ID to tracking ID 101 of person information 100 The face image 220 included in the face detection information is set to the face image 107 of the person information 100. Set the face coordinates 221 included in the face detection information to the face coordinates 109 of the person information 100 The first image shooting time 102 of the person information 100 is set to the shooting time included in the earliest face detection information (i.e., the earliest acquired shooting time). Set "Before matching request" to the matching status 111 of the person information 100
[0079] The management device 50 detects the moving direction of the person 1 corresponding to the face detection information, and sets the detected moving direction as the moving direction 112 of the person information 100 generated in step S303 (S304). Note that the method for detecting the moving direction of a person will be described in detail later (see FIG. 16).
[0080] The management device 50 newly registers the personal information 100 generated and set in steps S303 and S304 in the personal information table (S305).
[0081] The management device 50 transmits a face matching request including the face image 107 and the tracking ID 101 to the face matching device 60, and updates the matching status 111 of the person information 100 to "matching in progress" (S306). Then, the management device 50 completes this process.
[0082] Next, a case where it is determined in step S302 that the personal information 100 has been detected from the personal information table (S302: YES) will be described.
[0083] The management device 50 updates the content of the person information 100 detected in step S301 based on the face detection information (S310). For example, the management device 50 updates the detected person information 100 as follows. Update the face coordinates 109 of the person information 100 to the face coordinates 221 included in the face detection information The image update time 103 and face image update time 108 of the person information 100 are updated to the shooting time included in the face detection information.
[0084] The management device 50 determines whether the matching status 111 of the detected person information 100 is "before matching request" or "matching failed" (S311). If the matching status 111 is not "before matching request" or "matching failed" (S311: NO), the management device 50 completes this process.
[0085] If the matching status 111 is "before matching request" or "matching failed" (S311: YES), the management device 50 transmits a face matching request including the face image 107 and the tracking ID 101 to the face matching device 60, and updates the matching status 111 of the person information 100 to "matching in progress" (S312). Then, the management device 50 completes this process.
[0086] <Body detection information processing in control devices> FIG. 9 is a flowchart showing an example of the body detection information processing (S204) shown in FIG.
[0087] The management device 50 detects, from the person information table, the person information 100 corresponding to the body detection information received from the first edge device 41 or the second edge device 42 (S401). In this embodiment, the person information 100 corresponding to the body detection information is the person information 100 that is presumed to be the same person as the person 1 indicated by the body detection information. Details of a method for detecting, from the person information table, the person information 100 that is presumed to be the same person as the person 1 indicated by the body detection information will be described later (see FIGS. 15A and 15B).
[0088] The management device 50 determines whether or not the personal information 100 has been detected from the personal information table in step S401 (S402).
[0089] Next, a case where it is determined in step S402 that the personal information 100 cannot be detected from the personal information table (S402: NO) will be described.
[0090] The management device 50 generates new personal information 100 based on the received body detection information (S403). At this time, the management device 50 sets the following content in the new personal information 100. Set a new tracking ID to tracking ID 101 of person information 100 The body image 200 included in the body detection information is set to the body image 104 of the person information 100. The body coordinates 202 included in the body detection information are set to the body coordinates 106 of the person information 100. The first image capture time 102 of the person information 100 is set to the capture time included in the earliest body detection information (i.e., the earliest acquired capture time).
[0091] The management device 50 detects the moving direction of the person 1 corresponding to the body detection information, and sets the detected moving direction as the moving direction 112 of the person information 100 generated in step S403 (S404). Note that the method for detecting the moving direction of the person 1 will be described in detail later (see FIG. 16).
[0092] The management device 50 newly registers the personal information 100 generated and set in steps S403 and S404 in the personal information table (S405), and then the management device 50 completes this process.
[0093] Next, a case where it is determined in step S402 that the personal information 100 has been detected from the personal information table (S402: YES) will be described.
[0094] The management device 50 updates the content of the person information 100 detected in step S401 based on the body detection information (S410). For example, the management device 50 updates the detected person information 100 as follows. The body coordinates 106 of the person information 100 are updated to the body coordinates 202 included in the body detection information. The image update time 103 and the body image update time 105 of the person information 100 are updated to the body image capture time included in the body detection information.
[0095] Then, the management device 50 completes this process.
[0096] <Post-movement processing in management device> FIG. 10 is a flowchart showing an example of the post-movement process (S210) shown in FIG.
[0097] The management device 50 generates person movement information based on the target person information (S501). In this embodiment, the target person information is person information 100 detected from the person information table or newly generated person information 100 in the face detection information processing shown in Fig. 8, or person information 100 detected from the person information table or newly generated person information 100 in the body detection information processing shown in Fig. 9. At this time, the management device 50 sets the following content in the person movement information. Tracking ID Zone Information -Facial image (or body image if no facial image is available) · Matching status Face Matching ID
[0098] Here, the zone information is information indicating whether the person 1 corresponding to the target person information is located in the face matching zone 301 or the passage determination zone 302 in the detection area 3. The face matching zone 301 and the passage determination zone 302 will be described in detail later (see FIG. 13).
[0099] The management device 50 determines whether the movement direction of the target person information is the first direction or the second direction (S502).
[0100] If the movement direction is the first direction (S502: first direction), the management device 50 transmits the person movement information generated in step S501 to the first edge device 41 (S503), and completes this process.
[0101] If the movement direction is the second direction (S502: second direction), the management device 50 transmits the person movement information generated in step S501 to the second edge device 42 (S504), and completes this process.
[0102] <Face matching result information processing in control device> FIG. 11 is a flowchart showing an example of the face matching result information processing (S206) shown in FIG.
[0103] The management device 50 sets the content of the face matching result information received from the face matching device 60 in the personal information 100 corresponding to the face matching result information in the personal information table (S601). For example, the management device 50 sets the contents of the face matching result information in the person information 100 as follows. Set the face matching ID of the face matching result information to face matching ID 110 of person information 100 The matching result (matching success or failure) of the face matching result information is set to the matching status 111 of the person information 100
[0104] The management device 50 generates person verification information based on the person information 100 set in step S601 (S602). At this time, the management device 50 sets the following content in the person verification information. Tracking ID Zone Information Facial image · Matching status Face Matching ID
[0105] The management device 50 determines whether the movement direction 112 of the person information 100 set in step S601 is the first direction or the second direction (S603).
[0106] If the movement direction 112 is the first direction (S602: first direction), the management device 50 transmits the person verification information generated in step S602 to the first edge device 41 (S604), and completes this process.
[0107] If the movement direction 112 is the second direction (S602: second direction), the management device 50 transmits the person verification information generated in step S602 to the second edge device 42 (S605), and completes this process.
[0108] <Display processing in first edge device> 12 is a flowchart showing an example of display processing performed by the first edge device 41 according to embodiment 1. Note that, for the display processing performed by the second edge device 42 according to embodiment 1, in the following description with reference to FIG. 12, "first edge device 41" should be read as "second edge device 42," "first display device 31" should be read as "second display device 32," and "first direction" should be read as "second direction."
[0109] The first edge device 41 determines whether or not it has received person movement information or person matching information from the management device 50 (S701). If the first edge device 41 has not received either person movement information or person matching information (S701: NO), it returns the process to step S701. If the first edge device 41 has received either person movement information or person matching information (S701: YES), it performs the following process.
[0110] The first edge device 41 determines whether a face image or a body image corresponding to the tracking ID included in the received person movement information or person matching information is displayed on the first display device 31 (S702).
[0111] If the face image or body image corresponding to the tracking ID is not displayed on the first display device 31 (S702: NO), the first edge device 41 newly displays the face image or body image and matching status included in the person movement information or person matching information on the first display device 31 (S703). Then, the first edge device 41 returns the process to step S701.
[0112] If a face image or body image corresponding to the tracking ID is displayed on the first display device 31 (S702: YES), the first edge device 41 updates the displayed face image or body image and matching status to the face image or body image and matching status included in the person movement information or person matching information and displays them (S704).Then, the first edge device 41 returns the process to step S701.
[0113] As a result, a matching result image showing the face image or body image of each person 1 moving in the first direction and the matching status (for example, matching success or failure, etc.) is displayed on the first display device 31, as shown in Fig. 2. Therefore, each person 1 moving in the first direction can visually confirm their own matching result from the first display device 31 installed at the destination in the direction of movement.
[0114] Similarly, as shown in Fig. 2, the second display device 32 displays a matching result image showing the face image or body image of each person 1 moving in the second direction and the matching status (for example, matching success or failure, etc.). Therefore, each person 1 moving in the second direction can visually confirm their own matching result from the second display device 32 installed at the destination in the direction of movement.
[0115] The above-mentioned display processing can be expressed as follows: That is, the management device 50 detects the movement direction of each person 1 moving between the first gate 11 and the second gate 12, and displays the matching result of the facial image 220 of the person 1 moving in the first direction on the first display device 31 but not on the second display device 32, and displays the matching result of the facial image 220 of the person 1 moving in the second direction on the second display device 32 but not on the first display device 31.
[0116] As a result, the matching result for person 1 is displayed on the display device installed in the direction of movement of person 1, and is not displayed on the display device installed opposite the direction of movement of person 1. Therefore, the management device 50 can appropriately show the matching result to each of person 1 moving in the first direction along the passage 2 and person 1 moving in the second direction.
[0117] <Display pattern> 13 is a diagram showing face matching zone 301 and passage determination zone 302 in detection area 3 according to embodiment 1. Note that, although the following description will be given taking first gate 11 and the first direction as an example, the same applies to second gate 12 and the second direction.
[0118] The image captured by the first photographing device 21 at the first gate 11 is used for face matching of the person 1 moving in the first direction. In the detection area 3, a predetermined area close to the second gate 12 is set as a face matching zone 301, and a predetermined area close to the first gate 11 is set as a passage determination zone 302. The first edge device 41 may change the display pattern on the first display device 31 depending on whether the person 1 is located in the face matching zone 301 or the passage determination zone 302. Next, the display pattern will be described.
[0119] 14 is a diagram for explaining an example of a display pattern in the first display device 31 according to embodiment 1. Note that, in the following description with reference to FIG. 14, for the display pattern in the second display device 32, the "first edge device 41" can be read as the "second edge device 42," and the "first display device 31" can be read as the "second display device 32."
[0120] If the person 1 is located outside the detection area 3 (outside the zone), the first edge device 41 does not display the matching result image of the person 1 on the first display device 31.
[0121] When person 1 is located in face matching zone 301, first edge device 41 controls the display of first display device 31 according to the following patterns (A0) to (A3). (A0) When a facial image 220 of person 1 exists and the matching status 111 is “matching in progress”, the first edge device 41 displays a matching result image including the facial image 220 of person 1 and the message “matching in progress” on the first display device 31. (A1) When the face image 220 of person 1 exists and the matching status 111 is “matching successful,” the first edge device 41 displays a matching result image including the face image 220 of person 1 and a message “matching successful” on the first display device 31. At this time, the first edge device 41 may output a matching success sound from a speaker provided in the first display device 31. (A2) When the face image 220 of person 1 exists and the matching status 111 is “matching failed”, the first edge device 41 displays a matching result image including the face image 220 of person 1 and a message “matching failed” on the first display device 31. At this time, the first edge device 41 may output a matching failure sound from a speaker provided in the first display device 31. (A3) When the face image 220 of the person 1 does not exist but the body image 200 exists, the first edge device 41 displays a matching result image including the body image 200 of the person 1 and a “Warning” message on the first display device 31. At this time, the first edge device 41 may output a warning sound from a speaker provided in the first display device 31.
[0122] When the person 1 is located in the passage determination zone 302, the first edge device 41 controls the display of the first display device 31 according to the following patterns (A4) to (A7). (A4) When a facial image 220 of person 1 exists and the matching status 111 is “matching in progress”, the first edge device 41 displays a matching result image including the facial image 220 of person 1 and the message “matching in progress” on the first display device 31. (A5) When the face image 220 of person 1 exists and the matching status 111 is “matching successful,” the first edge device 41 displays a matching result image including the face image 220 of person 1 and a message “passed” on the first display device 31. At this time, the first edge device 41 may output a passing sound from a speaker provided in the first display device 31. (A6) When the face image 220 of person 1 exists and the matching status 111 is “matching failed,” the first edge device 41 displays a matching result image including the face image 220 of person 1 and a message “unauthorized passage” on the first display device 31. At this time, the first edge device 41 may output an unauthorized passage sound from a speaker provided in the first display device 31. (A7) When the face image 220 of the person 1 does not exist but the body image 200 exists, the first edge device 41 displays a matching result image including the body image 200 of the person 1 and a message saying "unauthorized passage" on the first display device 31. At this time, the first edge device 41 may output an unauthorized passage sound from a speaker provided in the first display device 31.
[0123] The display priority may be set as follows. The passage determination zone 302 has a higher display priority than the face matching zone 301. The "passing" message has the highest display priority, followed by the "matching successful" message. If there is a face image 220, the display priority of the "matching" message is the lowest. The face image 220 has a higher display priority than the body image 200 alone.
[0124] For example, the display priorities of the above-mentioned patterns (A0) to (A7) may be set as follows in order from lowest to highest: (A3)<(A7)<(A0)<(A4)<(A2)<(A6)<(A1)<(A5)
[0125] <Method for detecting personal information of the same person> 15A and 15B are diagrams for explaining a method for detecting, from a person information table, person information 100 that is presumed to belong to the same person as person 1 indicated by the body detection information in Embodiment 1. Note that in FIG. 15, the description will be given assuming that person 1 is moving in a first direction. In the case where person 1 is moving in a second direction, in the following description referring to FIG. 15, the "first camera device 21" should be read as the "second camera device 22," and the "second camera device 22" should be read as the "first camera device 21."
[0126] A body image 200 of person 1 moving in a first direction captured by the first camera 21 shows the front view of the body of the person 1. Hereinafter, the X and Y coordinates of body coordinates 201A calculated from the front view of the body image 200 will be referred to as the X coordinate 201Ax and the Y coordinate 201Ay of the front of the body, respectively. Furthermore, a face image 220 of person 1 moving in the first direction captured by the first camera 21 shows the front view of the face of the person 1. Hereinafter, the X and Y coordinates of face coordinates 221A calculated from the front view of the face image 220 will be referred to as the X coordinate 221Ax and the Y coordinate 221Ay of the front of the face, respectively.
[0127] The body image 200 of the person 1 moving in the first direction captured by the second image capturing device 22 shows the back side of the body of the person 1. Hereinafter, the X coordinate and the Y coordinate of the body coordinate 201B calculated from the back side body image 200 will be referred to as the X coordinate 201Bx and the Y coordinate 201By of the back side of the body, respectively.
[0128] As shown in FIG. 15A , the Y coordinate 201Ay of the front face and the Y coordinate 201By of the back face are spaced apart by the thickness of the person 1's body in the front-to-back direction. Therefore, if the positional difference between the Y coordinate 201Ay of the front face and the Y coordinate 201By of the back face is smaller than a predetermined threshold W based on the body thickness (hereinafter referred to as the body thickness threshold), it can be assumed that the Y coordinate 201Ay of the front face and the Y coordinate 201By of the back face correspond to the Y coordinates of the same person. The body thickness threshold W may be a value slightly larger than the average body thickness. Similarly, if the positional difference between the Y coordinate 221Ay of the front face and the Y coordinate 201By of the back face is smaller than the body thickness threshold W, it can be assumed that the Y coordinate 221Ay of the front face and the Y coordinate 201By of the back face correspond to the Y coordinates of the same person.
[0129] 15B, if the amount of change in the X component between X-coordinate 201Ax of body coordinate 201A included in the received body detection information (i.e., the current X-coordinate) and X-coordinate 201Ax of body coordinate 201A included in personal information 100 (i.e., the past X-coordinate) is equal to or less than a predetermined threshold (hereinafter referred to as the body width threshold) W2, it can be estimated that X-coordinate 201Ax of the current body coordinate and X-coordinate 201Ax of the past body coordinate are the X-coordinates of the same person. Note that face coordinate 221A may be used instead of body coordinate 201A.
[0130] In other words, if the management device 50 can detect from the personal information table personal information 100 such that the positional difference between the X coordinate of the body coordinate 201 (or face coordinate 221) included in the received body detection information and the X coordinate of the body coordinate 201 (or face coordinate 221) included in the personal information 100 is less than the body width threshold W2, and the positional difference between the Y coordinate of the body coordinate 201 included in the received body detection information and the Y coordinate of the body coordinate 201 or the Y coordinate of the face coordinate 221 included in the personal information 100 is less than the body thickness threshold W, the management device 50 may estimate that the detected personal information 100 is the same person as the person 1 indicated by the body detection information.
[0131] Similarly, if the management device 50 can detect from the personal information table personal information 100 in which the positional difference between the X coordinate of the body coordinate 201 (or face coordinate 221) included in the received body detection information and the X coordinate of the body coordinate 201 (or face coordinate 221) included in the personal information 100 is smaller than the body width threshold W2, and the positional difference between the Y coordinate of the face coordinate 221 included in the received face detection information and the Y coordinate of the body coordinate 201 or the Y coordinate of the face coordinate 221 included in the personal information 100 is smaller than the body thickness threshold W, the management device 50 may estimate that the detected personal information 100 is the same person as the person 1 indicated by the face detection information.
[0132] The above-mentioned conditions regarding the X-coordinate positional difference and the body width threshold W2, and the conditions regarding the Y-coordinate positional difference and the body thickness threshold W are met when the time difference between the face photographing time included in the face detection information or the body photographing time included in the body detection information and the face image update time 108 or the body image update time 105 included in the person information 100 is sufficiently small (i.e., when the time difference is smaller than a predetermined time threshold T). Because person 1 is moving, if the time difference is large, person information 100 that satisfies the above-mentioned conditions regarding the X-coordinate positional difference and the body width threshold W2, and the conditions regarding the Y-coordinate positional difference and the body thickness threshold W is likely to belong to a different person who was in that position in the past.
[0133] 8, the following process may be performed: The management device 50 detects, from the personal information table, personal information 100 for which the time difference between the face capture time included in the face detection information and the face image update time 108 or the body image update time 105 included in the personal information 100 is less than a predetermined time threshold T, the positional difference between the X coordinate of the face coordinate 221 included in the face detection information and the X coordinate of the face coordinate 109 included in the personal information 100 is smaller than a body width threshold W2, and the positional difference between the Y coordinate of the face coordinate 221 included in the face detection information and the Y coordinate of the face coordinate 109 or the Y coordinate of the body coordinate 106 included in the personal information 100 is smaller than a body thickness threshold W.
[0134] 9, the following process may be performed. The management device 50 detects, from the personal information table, personal information 100 in which the time difference between the body photographing time included in the body detection information and the face image update time 108 or the body image update time 105 included in the personal information 100 is less than a predetermined time threshold T, the positional difference between the X coordinate of the body coordinate 201 included in the body detection information and the X coordinate of the body coordinate 106 included in the personal information 100 is smaller than a body width threshold W2, and the positional difference between the X coordinate of the body coordinate 201 included in the body detection information and the Y coordinate of the face coordinate 109 or the Y coordinate of the body coordinate 106 included in the personal information 100 is smaller than a body thickness threshold W.
[0135] The above content can be expressed as follows: That is, if the difference between the position (e.g., body coordinates 201 or face coordinates 221) of person 1 included in a first image (e.g., body image 200 or face image 220) captured by the first image capturing device 21 and the position (e.g., body coordinates 201 or face coordinates 221) of person 1 included in a second image (e.g., body image 200 or face image 220) captured by the second image capturing device 22 is smaller than a predetermined threshold W based on the thickness of the person's body, the management device 50 estimates that the position of person 1 included in the first image and the position of person 1 included in the second image represent the positions of the same person.
[0136] This allows the management device 50 to estimate whether, for example, a first image capturing the front of person 1 and a second image capturing the back of person 1 are of the same person.
[0137] <Method for detecting the direction of movement of a person> FIG. 16 is a diagram for explaining a method for detecting the moving direction of the person 1 according to the first embodiment.
[0138] The management device 50 detects the moving direction of the person 1 by, for example, any one of the following first, second, and third methods.
[0139] <<First Method>> The management device 50 detects the movement direction 112 of the person 1 based on the movement direction of the face coordinates 109 or the body coordinates 106 included in the person information 100 over time. For example, when the management device 50 detects that the Y coordinate of the face coordinates 109 or the Y coordinate of the body coordinates 106 included in the person information 100 is moving in the negative direction of the Y axis over time, the management device 50 detects the movement direction 112 of the person 1 as a first direction. When the management device 50 detects that the Y coordinate of the face coordinates 109 or the Y coordinate of the body coordinates 106 included in the person information 100 is moving in the positive direction of the Y axis over time, the management device 50 detects the movement direction 112 of the person 1 as a second direction.
[0140] <<Second Method>> The management device 50 determines whether the first detected image of the person 1 was captured by the first camera 21 or the second camera 22, based on the camera ID. If the first detected image of the person 1 was captured by the first camera 21, the management device 50 detects the movement direction 112 of the person 1 as the first direction. This is because the person 1 entering the detection area 3 from the second gate 12 is first captured by the first camera 21 provided at the first gate 11. If the first detected image of the person 1 was captured by the second camera 22, the management device 50 detects the movement direction 112 of the person 1 as the second direction. This is because the person 1 entering the detection area 3 from the first gate 11 is first captured by the second camera 22 provided at the second gate 12.
[0141] <<Third Method>> As shown in FIG. 16 , when the face coordinates 109 or body coordinates 106 of person 1 detected for the first time are included in a predetermined area 312 of the detection area 3 that is closer to the second gate 12, the management device 50 detects the movement direction 112 of the person 1 as a first direction. This is because person 1 entering the detection area 3 from the second gate 12 first enters the predetermined area 312 closer to the second gate 12. When the face coordinates 109 or body coordinates 106 of person 1 detected for the first time are included in a predetermined area 311 of the detection area 3 that is closer to the first gate 11, the management device 50 detects the movement direction 112 of person 1 as a second direction. This is because person 1 entering the detection area 3 from the first gate 11 first enters the predetermined area 311 closer to the first gate 11.
[0142] (Embodiment 2) In the second embodiment, a gate system 9 will be described that allows passage even for a person who failed the facial matching described in the first embodiment, if the person satisfies a predetermined condition. For example, when an employee passes through the gate system 9 with a visitor, the employee's facial image has been registered, but the visitor's facial image has not. Therefore, if the gate system 9 described in the first embodiment is used as is, all of the visitors will fail matching. To avoid this, a method of temporarily registering the facial images of all visitors by photographing their faces before they pass through the gate system 9 is conceivable. However, some visitors may be reluctant to have their faces photographed and their facial images registered, even if only temporarily. Furthermore, when there are a large number of visitors, photographing each person's face takes time. Therefore, the following describes a gate system 9 that allows passage for a person (e.g., a visitor) accompanied by a person (e.g., an employee) whose facial image has been registered and whose facial matching is successful, even without registering their facial image.
[0143] In the second embodiment, the same reference numerals may be used to designate components that have already been described in the first embodiment, and the description thereof may be omitted. The content of the first embodiment may also be incorporated into the second embodiment.
[0144] <Setting and processing of tailgating detection zones> FIG. 17 is a diagram showing an example in which face matching zone 301, passage determination zone 302, and tailgating determination zone 303 are set in detection area 3 according to the second embodiment.
[0145] Management device 50 sets face matching zone 301, passage determination zone 302, and tailgating determination zone 303 in detection area 3. For example, as shown in FIG. 17 , management device 50 sets face matching zone 301 and tailgating determination zone 303 on the entrance side in the first direction, and sets passage determination zone 302 on the exit side in the first direction. Note that tailgating determination zone 303 may be set in the entire detection area 3. Also, face matching zone 301 may be read as the first area, and tailgating determination zone 303 may be read as the second area.
[0146] Next, the process performed by the management device 50 when a zone such as that shown in FIG. 17 is set will be described with reference to FIG.
[0147] FIG. 18 is a flowchart showing an example of processing performed by management device 50 when face matching zone 301, tailgating determination zone 303, and passage determination zone 302 are set in detection area 3 according to the second embodiment.
[0148] 18 is a process for one person to be tracked, and the management device 50 executes the process shown in FIG. 18 for each of a plurality of people to be tracked detected from the detection area 3.
[0149] The management device 50 detects a person who has entered the detection area 3 as a person to be tracked by the method described in the first embodiment (S1001).
[0150] The management device 50 determines whether the person to be tracked is located in the face matching zone 301, the tailgating determination zone 303, or the passage determination zone 302 (S1002).
[0151] If the management device 50 determines that the person being tracked is located in the face matching zone 301 (S1002: face matching zone), it executes face matching processing (S1003) and proceeds to step S1010. Details of the face matching processing will be described later (see FIG. 19).
[0152] If management device 50 determines that the person being tracked is located in tailgating determination zone 303 (S1002: tailgating determination zone), it executes face matching and tailgating determination processing (S1004), and proceeds to step S1010. Details of the face matching and tailgating determination processing will be described later (see FIG. 20).
[0153] When the management device 50 determines that the person being tracked is located in the passage determination zone 302 (S1002: passage determination zone), the management device 50 executes a passage determination process (S1005) and proceeds to step S1010. The passage determination process will be described in detail later (see FIG. 21).
[0154] The management device 50 determines whether to end tracking of the person to be tracked (S1010). For example, the management device 50 ends tracking of the person to be tracked when the person to be tracked is no longer detected in the detection area 3. If the management device 50 continues tracking of the person to be tracked (S1010: NO), the processing returns to step S1002, and if the management device 50 ends tracking of the person to be tracked (S1010: YES), the processing ends.
[0155] <Flowchart of face matching process> 19 is a flowchart showing an example of face matching processing according to Embodiment 2. The processing shown in FIG. 19 corresponds to details of the processing in step S1003 shown in FIG.
[0156] The management device 50 performs face matching of the tracking person by the method described in the first embodiment (S1101). At this time, the management device 50 may display a message indicating that matching is in progress on the first display device 31 from the start of face matching until the display of the face matching result.
[0157] The management device 50 determines whether or not the face matching is successful (S1102).
[0158] If the face matching is successful (S1102: YES), the management device 50 displays a message indicating successful matching on the first display device 31 (S1103). Then, the management device 50 assigns a passage permission flag to the person being tracked (S1104), and ends this process.
[0159] If face matching fails (S1102: NO), the management device 50 displays a message indicating that matching has failed on the first display device 31 (S1105), and ends this process.
[0160] Note that, for a tracking person for whom face matching has once succeeded or failed, the management device 50 may omit the processing shown in Fig. 19. After this processing is completed, the processing of step S1010 shown in Fig. 18 is executed.
[0161] <First example: Flowchart of face matching and tailgating determination processing> 20 is a flowchart showing a first example of the face matching and tailgating determination process according to Embodiment 2. The process shown in FIG. 20 corresponds to details of the process of step S1004 shown in FIG.
[0162] The management device 50 performs face matching of the person being tracked using the method described in the first embodiment (S1201).
[0163] The management device 50 determines whether or not the face matching is successful (S1202).
[0164] If the face matching is successful (S1202: YES), the management device 50 displays a message indicating that the tailgating permission mode is in effect in the tailgating determination zone 303 on the first display device 31 (S1203). Then, the management device 50 assigns a passage permission flag to the person being tracked (S1204), and ends this process.
[0165] If face matching fails (S1202: NO), the management device 50 displays a message indicating that tailgating permission determination is in progress on the first display device 31 (S1205). Then, the management device 50 determines whether or not a person whose face has already been successfully matched (i.e., face-registered person 1A) is present within the tailgating determination zone 303, apart from the tracking person whose face matching has failed (S1206).
[0166] If there is no person whose face has already been successfully matched within tailgating determination zone 303 (S1206: NO), management device 50 displays a message indicating that tailgating permission determination is in progress on first display device 31 (S1210), and ends this process.
[0167] If a person whose face has already been successfully matched is present within the tailgating determination zone 303 (S1206: YES), the management device 50 determines that the person being tracked whose face has not been matched within the tailgating determination zone 303 (i.e., person 1B whose face is not registered) is a tailgating person 1C (e.g., a visitor) brought by a person whose face has been successfully matched (e.g., an employee) (S1207).
[0168] The management device 50 displays a message on the first display device 31 to the effect that the passage of the tailgating person 1C is permitted (S1208).
[0169] The management device 50 assigns a passage permission flag to the tracking person whose face matching has failed (that is, the tailgating person 1C) (S1209), and ends this process.
[0170] That is, in the tailgating determination zone 303, the management device 50 performs face matching on the person being tracked who is present in the tailgating determination zone 303, and allows the passage of a face-registered person 1A (e.g., an employee) whose face matching is successful, and also allows the passage of a face-unregistered person 1B (e.g., a visitor) who is present in the tailgating determination zone 303 and whose face matching is unsuccessful as a companion 1C of the face-registered person 1A while the face-registered person 1A is present in the tailgating determination zone 303. On the other hand, if the face-registered person 1A is not present in the tailgating determination zone 303, the management device 50 does not allow the face-unregistered person 1B to pass. Note that the face-registered person 1A may be read as a first person, and the face-unregistered person 1B may be read as a second person.
[0171] As a result, tailgating person 1C, who is brought by face-registered person 1A whose face image has been registered and whose face matching is successful, can pass through passage 2 without undergoing face registration by passing through tailgating determination zone 303. Furthermore, face-unregistered person 1B, who is not brought by face-registered person 1A and whose face matching fails, cannot pass through face matching zone 301, let alone tailgating determination zone 303.
[0172] <Flowchart of Passage Determination Processing> 21 is a flowchart showing an example of the passage determination process according to Embodiment 2. The process shown in FIG. 21 corresponds to details of the process of step S1005 shown in FIG.
[0173] The management device 50 determines whether or not a passage permission flag is assigned to the person to be tracked (S1301).
[0174] If the passage permission flag is assigned to the person to be tracked (S1301: YES), the management device 50 displays a message indicating that the person is permitted to pass on the first display device 31 (S1302), and then ends this process.
[0175] If the passage permission flag is not assigned to the person to be tracked (S1301: NO), the management device 50 displays a message of unauthorized passage on the first display device 31 (S1303).
[0176] The management device 50 transmits the facial image of the person being tracked (including the accompanying person 1C) stored in the memory 1002, who is passing without being assigned a passage permission flag, to, for example, the terminal of a staff member monitoring the entrance and exit (S1304), and then completes this process.
[0177] As a result, a person to be tracked who has been given a passage permission flag in face matching zone 301, and a person to be tracked who has been given a passage permission flag in tailgating determination zone 303 (including tailgating person 1C) can pass through passage determination zone 302.
[0178] <Example 2: Activating the tailgating detection zone according to the movements of the face-registered person> Fig. 22 is a flowchart showing a second example of the face matching and tailgating determination process according to embodiment 2. The process shown in Fig. 22 corresponds to details of the process of step S1004 shown in Fig. 18. Next, with reference to Fig. 22, a process for validating tailgating determination zone 303 when face-registered person 1A is present in tailgating determination zone 303 and has performed a first action will be described.
[0179] The management device 50 performs face matching of the person to be tracked by the method described in the first embodiment (S1401).
[0180] The management device 50 determines whether or not the face matching is successful (S1402).
[0181] First, the case where face matching is successful (S1402: YES) will be described.
[0182] If the face matching is successful (S1402: YES), the management device 50 displays a message indicating that matching is successful on the first display device 31 (S1403).
[0183] The management device 50 assigns a passage permission flag to the person being tracked (S1404).
[0184] The management device 50 determines whether the person being tracked (i.e., face-registered person 1A) is performing a first action (S1405). For example, the management device 50 determines, as the first action, whether face-registered person 1A has been standing still for a certain period of time in tailgating determination zone 303. Note that the first action is not limited to standing still for a certain period of time, and may be, for example, an action in which face-registered person 1A is performing a predetermined pose (for example, raising his / her hand).
[0185] When the management device 50 determines that the tracking person (that is, the face-registered person 1A) is not performing the first action (S1405: NO), the management device 50 ends this process.
[0186] If the management device 50 determines that the person being tracked (i.e., face-registered person 1A) is performing the first action (S1405: YES), it sets the tailgating determination zone 303 to the tailgating permission mode (i.e., enables the tailgating determination zone 303) (S1406).
[0187] The management device 50 displays a message indicating that the tailgating permission mode is in effect on the first display device 31 (S1407), and ends this process.
[0188] Next, the case where face matching is successful (S1402: NO) will be described.
[0189] If the face matching fails (S1402: NO), the management device 50 determines whether the tailgating determination zone 303 is in the tailgating permission mode (S1408).
[0190] If the tailgating determination zone 303 is not in tailgating permission mode (S1408: NO), the management device 50 displays a message on the first display device 31 indicating that matching has failed for the person being tracked (i.e., person 1B whose face is not registered) (S1409), and terminates this process.
[0191] If the tailgating determination zone 303 is in the tailgating permission mode (S1408: YES), the management device 50 displays a message of tailgating permission for the person being tracked (i.e., person 1B whose face is not registered) on the first display device 31 (S1410). Then, the management device 50 assigns a passage permission flag to the person being tracked (i.e., person 1B whose face is not registered) (S1411), and ends this process.
[0192] As a result, while face-registered person 1A is performing the first action in tailgating determination zone 303, management device 50 can place tailgating determination zone 303 in tailgating permission mode and permit face-unregistered person 1B present in tailgating determination zone 303 to pass as companion person 1C of face-registered person 1A. Furthermore, if face-registered person 1A is not performing the first action in tailgating determination zone 303, management device 50 does not place tailgating determination zone 303 in tailgating permission mode. As a result, face-registered person 1A can control whether or not to place tailgating determination zone 303 in tailgating permission mode depending on whether or not he or she performs the first action in tailgating determination zone 303.
[0193] <Third example: Setting a tailgating detection zone around a person whose face has been registered> Fig. 23 is a diagram showing an example of setting a tailgating determination zone 303 around a face-registered person 1A according to embodiment 2. Fig. 24 is a flowchart showing a third example of the face matching and tailgating determination process according to embodiment 2. The process shown in Fig. 24 corresponds to details of the process of step S1004 shown in Fig. 18.
[0194] Next, with reference to FIGS. 23 and 24, a process for setting the tailgating determination zone 303 in a predetermined range including the face-registered person 1A will be described.
[0195] The management device 50 performs face matching of the person to be tracked by the method described in the first embodiment (S1501).
[0196] The management device 50 determines whether or not the face matching is successful (S1502).
[0197] First, the case where face matching is successful (S1502: YES) will be described.
[0198] If the face matching is successful (S1502: YES), the management device 50 displays a message indicating that matching is successful on the first display device 31 (S1503).
[0199] The management device 50 assigns a passage permission flag to the tracking person (that is, the face-registered person 1A) (S1504).
[0200] The management device 50 determines whether or not the tracking person (that is, the face-registered person 1A) is performing the first action (S1505).
[0201] When the management device 50 determines that the tracking person (that is, the face-registered person 1A) is not performing the first action (S1505: NO), the management device 50 ends this process.
[0202] When the management device 50 determines that the person to be tracked (i.e., face-registered person 1A) is performing the first action (S1505: YES), it sets a predetermined range including the position where the person to be tracked (i.e., face-registered person 1A) is present as tailgating determination zone 303 (S1506). For example, as shown in Fig. 23, the management device 50 sets a range of a predetermined radius centered on face-registered person 1A as tailgating determination zone 303. Note that the shape of tailgating determination zone 303 is not limited to a circle, and may be an oval, a rectangle, or the like.
[0203] The management device 50 displays a message indicating that the tailgating permission mode is in effect on the first display device 31 (S1507), and ends this process.
[0204] Next, a case where face matching fails (S1502: NO) will be described.
[0205] If face matching fails (S1502: NO), management device 50 determines whether or not the person to be tracked (i.e., person 1B whose face is not registered) is present in tailgating determination zone 303 (S1508). Note that if tailgating determination zone 303 is not set, management device 50 determines that the person to be tracked (i.e., person 1B whose face is not registered) is not present in tailgating determination zone 303.
[0206] If the person being tracked (i.e., person 1B with an unregistered face) is not present in the tailgating determination zone 303 (S1508: NO), the management device 50 displays a message indicating that matching for the person being tracked (i.e., person 1B with an unregistered face) has failed on the first display device 31 (S1509), and terminates this process.
[0207] If the person to be tracked (i.e., person 1B whose face is not registered) is present in tailgating determination zone 303 (S1508: YES), management device 50 displays a message for allowing tailgating for the person to be tracked (i.e., person 1B whose face is not registered) on first display device 31 (S1510). Then, management device 50 assigns a passage permission flag to the person to be tracked (i.e., person 1B whose face is not registered) (S1511), and ends this process.
[0208] As a result, while face-registered person 1A is performing the first behavior, management device 50 sets tailgating determination zone 303 in a predetermined range including face-registered person 1A, and can permit face-unregistered person 1B present in tailgating determination zone 303 to pass as a tailgating person 1C of face-registered person 1A. Furthermore, if face-registered person 1A is not performing the first behavior, management device 50 does not set tailgating determination zone 303. As a result, face-registered person 1A can control whether or not to set tailgating determination zone 303 depending on whether or not the first behavior is performed.
[0209] <Example 4: A person with a registered face and an unregistered face perform cooperative actions, and the unregistered face is identified as a tailgating person> FIG. 25 is a diagram illustrating a first method according to the second embodiment in which face-registered person 1A and face-unregistered person 1B cooperate to turn face-unregistered person 1B into a tailgating person 1C.
[0210] As shown in FIG. 25, after successfully matching the face of a face-registered person 1A, if the distance L between the face-registered person 1A and the face-unregistered person 1B is equal to or less than a predetermined threshold, and if both the face-registered person 1A and the face-unregistered person 1B perform a first action (for example, an action of standing still for a certain period of time) (i.e., if the face-registered person 1A and the face-unregistered person 1B perform a cooperative action), the management device 50 determines that the face-unregistered person 1B is a person 1C accompanying the face-registered person 1A.
[0211] As a result, if face-registered person 1A and face-unregistered person 1B whose faces have been successfully authenticated do not cooperate with each other, face-unregistered person 1B will not be determined to be a tailgating person 1C. This reduces the possibility of erroneously determining face-unregistered person 1B, who is not a person brought by face-registered person 1A, as a tailgating person 1C.
[0212] FIG. 26 is a diagram illustrating a second method according to the second embodiment in which face-registered person 1A and face-unregistered person 1B cooperate to turn face-unregistered person 1B into a tailgating person 1C.
[0213] 26, after successfully matching the face of the face-registered person 1A, if the distance L between the face-registered person 1A and the face-unregistered person 1B is equal to or less than a predetermined threshold value within the tailgating determination zone 303 and both the face-registered person 1A and the face-unregistered person 1B perform a first action (for example, an action of standing still for a certain period of time) (i.e., the face-registered person 1A and the face-unregistered person 1B perform a cooperative action), the management device 50 determines the face-unregistered person 1B to be a tailgating person 1C of the face-registered person 1A. The tailgating determination zone 303 may be set to the entire detection area 3 as shown in FIG. 26, or may be set to a part of the detection area 3 as shown in FIG. 17.
[0214] As a result, even if the distance L between the face-registered person 1A and the face-unregistered person 1B is equal to or less than a predetermined threshold, and the face-registered person 1A and the face-unregistered person 1B whose faces have been successfully authenticated cooperate with each other, the face-unregistered person 1B will not be determined to be a tailgating person 1C if the face-registered person 1A or the face-unregistered person B is not present in the tailgating determination zone 303. This reduces the possibility of erroneously determining that the face-unregistered person 1B, who is not brought by the face-registered person 1A, is a tailgating person 1C.
[0215] FIG. 27 is a diagram showing an example of a display on the first display device 31 when the process shown in FIG. 25 or FIG. 26 is performed.
[0216] If the management device 50 succeeds in face matching for the face-registered person 1A, it displays a message on the first display device 31 indicating that tailgating permission mode is in effect for the face-registered person 1A, and displays a message indicating that tailgating permission determination is in progress for the face-unregistered person 1B, as shown in Figure 27(a).
[0217] Then, when the distance L between the face-registered person 1A and the face-unregistered person 1B is equal to or less than a predetermined threshold and both the face-registered person 1A and the face-unregistered person 1B perform a first action (for example, an action of standing still for a certain period of time) (i.e., when the face-registered person 1A and the face-unregistered person 1B perform a cooperative action), the management device 50 displays a message to the face-unregistered person 1B on the first display device 31 that tailgating is permitted, as shown in Figure 27(b).
[0218] This allows face-registered person 1A to recognize whether or not the tailgating permission mode is in effect by looking at first display device 31. Also, face-unregistered person 1B can recognize by looking at first display device 31 whether or not tailgating permission determination is in progress and whether or not tailgating has been permitted.
[0219] Fig. 28 is a flowchart showing a fourth example of the face matching and tailgating determination process according to embodiment 2. The process shown in Fig. 28 corresponds to the details of the process of step S1004 shown in Fig. 18. Next, with reference to Figs. 25, 26, 27 and 28, a detailed description will be given of the face matching and tailgating determination process in the case where face-registered person 1A and face-unregistered person 1B cooperate to determine face-unregistered person 1B as tailgating person 1C.
[0220] The processes of S1601 to S1605 shown in Fig. 28 are common to the processes of S1501 to S1505 shown in Fig. 24, and therefore will not be described here. After executing the process of step S1605, management device 50 determines whether or not the person to be tracked (i.e., person 1B whose face is not registered) is present in tailgating determination zone 303 (S1606). Note that, as in Fig. 25, if tailgating determination zone 303 does not exist, step S1606 may be omitted and the process of step S1607 may be executed.
[0221] If it is determined that the person to be tracked (that is, person 1B whose face is not registered) is not present in tailgating zone 303 (S1606: NO), management device 50 ends this process.
[0222] If it is determined that the person being tracked (i.e., person 1B whose face is not registered) is present in the tailgating zone 303 (S1606: YES), the management device 50 displays a message indicating that tailgating is permitted on the first display device 31 (S1607) and terminates this process.
[0223] Next, a case where face matching fails in step S1602 (S1602: NO) will be described.
[0224] If face matching fails (S1602: NO), management device 50 determines (S1608) whether or not the person being tracked (i.e., person 1B whose face is not registered) is present in tailgating determination zone 303. Note that, as shown in FIG. 25, if tailgating determination zone 303 does not exist, step S1608 may be omitted and the process of step S1610 may be executed.
[0225] If it is determined that the person being tracked (i.e., person 1B with an unregistered face) is not present in the tailgating determination zone 303 (S1608: NO), the management device 50 displays a message on the first display device 31 indicating that matching for the person being tracked (i.e., person 1B with an unregistered face) has failed (S1609), and terminates this processing.
[0226] If it is determined that the person being tracked (i.e., person 1B whose face is not registered) is present in the tailgating determination zone 303 (S1608: YES), the management device 50 displays a message indicating that tailgating permission determination is in progress for the person being tracked (i.e., person 1B whose face is not registered) on the first display device 31 (S1610), as shown in FIG. 27(a).
[0227] The management device 50 determines whether the distance L between the person who has successfully passed face matching and is performing the first action (face-registered person 1A) and the person to be tracked (face-unregistered person 1B) is equal to or less than a predetermined threshold (S1611).
[0228] If the distance L between the person (face-registered person 1A) who has successfully completed face matching and is performing the first action and the person being tracked (face-unregistered person 1B) is greater than a predetermined threshold (S1611: NO), the management device 50 displays a message on the first display device 31 indicating that matching has failed for the person being tracked (i.e., face-unregistered person 1B) (S1609), and terminates this processing.
[0229] If the distance L between the person (face-registered person 1A) who has successfully completed face matching and is performing the first action and the person being tracked (face-unregistered person 1B) is less than or equal to a predetermined threshold (S1611: YES), the management device 50 proceeds to the next step S1612.
[0230] The management device 50 determines whether or not the person being tracked (person 1B whose face is not registered) is performing an action in coordination with the first action (S1612).
[0231] If the management device 54 determines that the tracked person (face-unregistered person 1B) is not performing an action coordinated with the first action (S1612: NO), it displays a message indicating that matching has failed for the tracked person (i.e., face-unregistered person 1B) on the first display device 31 (S1609), and terminates this processing.
[0232] When the management device 54 determines that the tracked person (unregistered face person 1B) is performing an action in coordination with the first action (S1612: YES), it displays a message for allowing the tracked person (i.e., unregistered face person 1B) to travel with the person on the first display device 31 (S1613), as shown in Fig. 27(b). Then, the management device 50 assigns a passage permission flag to the tracked person (i.e., unregistered face person 1B) (S1614), and ends this process.
[0233] This allows the management device 50 to perform the operations shown in FIG. 25, FIG. 26, or FIG.
[0234] Note that the processing shown in Fig. 28 described above is based on the processing shown in Fig. 24, but processing similar to that shown in Fig. 28 described above can also be achieved using the processing shown in Fig. 22 or 24 as a base. For example, management device 50 may perform processing in which steps S1408 to S1411 shown in Fig. 22 are replaced with steps S1608 to S1613 shown in Fig. 28. Alternatively, management device 50 may perform processing in which steps S1508 to S1511 shown in Fig. 24 are replaced with steps S1608 to S1613 shown in Fig. 28.
[0235] <Tailgating mode disabled> Fig. 29 is a diagram showing a first display example of first display device 31 when the tailgating permission mode is cancelled according to embodiment 2. Fig. 30 is a diagram showing a second display example of first display device 31 when the tailgating permission mode is cancelled according to embodiment 2.
[0236] 22, while face-registered person 1A is performing the first action, management device 50 sets tailgating determination zone 303 to tailgating permission mode, and during the tailgating permission mode, allows face-unregistered person 1B present within tailgating determination zone 303 to pass as a companion person 1C of face-registered person 1A. In this case, as shown in FIG. 29(a) or 30(a), first display device 31 may display a message indicating that face-registered person 1A is in tailgating permission mode, and a message indicating that tailgating is permitted to tailgating person 1C.
[0237] Here, when the management device 50 detects that the face-registered person 1A has performed a second action different from the first action, the management device 50 may cancel the tailgating permission mode. For example, if the first action is standing still for a certain period of time, the second action may be starting to move from the standing state. For example, if the first action is raising a hand for a certain period of time, the second action may be lowering the raised hand.
[0238] When the management device 50 cancels the tailgating permission mode, the management device 50 may display a message to the face-registered person 1A indicating that the matching was successful and that the tailgating permission mode has been cancelled, as shown in FIG. 29(b) or FIG. 30(a).
[0239] Even if the management device 50 cancels the tailgating permission mode, it may maintain the tailgating permission for the face-unregistered person 1B who was permitted to tailgate before the face-registered person 1A performed the second action, as shown in Figure 29(b).
[0240] Alternatively, when the management device 50 cancels the tailgating permission mode, it may cancel the tailgating permission for the face-unregistered person 1B who was permitted to tailgate before the face-registered person 1A performed the second action, as shown in Figure 30(b), and may display, for example, a warning message.
[0241] When performing the process shown in FIG. 24 or FIG. 28, management device 50 may cancel the setting of tailgating determination zone 303 instead of canceling the above-described tailgating permission mode.
[0242] <Enter whether or not you want to allow tailgating on the device> FIG. 31 is a diagram showing an example of a screen for inputting whether or not to permit tailgating on a terminal according to the second embodiment.
[0243] When the management device 50 detects a tailgating person 1C by the above-described method, it may not immediately grant permission to the tailgating person 1C, but may instead transmit information about the detected tailgating person 1C to a terminal 400 carried by a face-registered person 1A (e.g., an employee) whose face has been successfully matched. The information about the tailgating person 1C may include a facial image of the tailgating person 1C captured by the first image capturing device 21 (or the second image capturing device 22). Examples of the terminal 400 include a tablet terminal, a smartphone, a notebook PC, etc.
[0244] Using the information received from the management device 50, the terminal 400 may display on the screen 401, as shown in FIG. 31, a facial image 410 of the accompanying person 1C, a button to designate the accompanying person 1C as a target for tailgating (hereinafter referred to as the tailgating target button 411), and a button to designate the accompanying person 1C as an ineligible target for tailgating (hereinafter referred to as the ineligible target button 412).
[0245] The face-registered person 1A looks at the face image 410 of the accompanying person 1C displayed on the screen 401 of the terminal 400, and touches the tailgating target button 411 if the accompanying person 1C is to be a target for tailgating (i.e., to be allowed to pass), or touches the non-tailgating target button 421 if the accompanying person 1C is not to be a target for tailgating (i.e., not to be allowed to pass).
[0246] In addition, in Figure 31, a facial image 410 of one accompanying person 1C and corresponding buttons 411, 412 are displayed, but if there are multiple accompanying people 1C, facial images 410 of each accompanying person 1C and corresponding buttons 411, 412 may be displayed.
[0247] The terminal 400 transmits to the management device 50 information relating to the tailgating person 1C and information indicating which button 411 or 412 was pressed in response to the information.
[0248] The management device 50 may determine whether or not to permit passage for each tailgating person 1C based on the information received from the terminal 400. For example, the management device 50 displays a message indicating permission to tailgate for a tailgating person 1C who has touched the tailgating target button 411 on the first display device 31, and displays a warning or a message indicating a verification failure for a tailgating person 1C who has touched the tailgating non-target button 412.
[0249] This allows the face-registered person 1A (e.g., an employee) whose face has been successfully matched to operate the terminal 400 to explicitly indicate the person (e.g., a guest) whom the management device 50 has determined to be the accompanying person 1C whom the management device 50 wishes to permit to pass through. This makes it possible to avoid erroneously granting a pass permit to a person who is not the person brought by the face-registered person 1A.
[0250] <Consider whether or not someone has facial registration, as well as whether or not they have permission to pass through> The management device 50 is not limited to a method of determining whether to permit passage through the gate system 9 based on whether a face image is registered, and may manage a security level for each gate system 9. In this case, the management device 50 manages the security level assigned to each face-registered person 1A and the security level assigned to each gate system 9. If the security level assigned to the gate system 9 through which the face-registered person 1A is about to pass is equal to or lower than the security level assigned to the face-registered person 1A, the management device 50 may determine that the face-registered person 1A has the authority to pass and may not permit the face-registered person 1A to pass if the security level assigned to the gate system 9 is higher than the security level assigned to the face-registered person 1A. In this case, if the management device 50 allows the face-registered person 1A to pass, it also assigns a pass permission flag to the face-unregistered person 1B who is accompanied by the face-registered person 1A, and if it does not allow the face-registered person 1A to pass, it does not need to assign a pass permission flag to the face-unregistered person 1B who is accompanied by the face-registered person 1A.
[0251] As a result, the management device 50 can allow the accompanying person 1C to pass through gate systems 9 for which the face-registered person 1A has passage authority, and can not allow the accompanying person 1C to pass through gate systems 9 for which the face-registered person 1A does not have passage authority.
[0252] <When there are multiple people registered as faces> In the above, for ease of understanding, the case where there is one face-registration person 1A has been described, but there may be a plurality of face-registration people 1A. In this case, the management device 50 may perform any one of the following processes (A1) to (A5).
[0253] (A1) In the example shown in Figure 17, if there is at least one face-registered person 1A in the tailgating determination zone 303, the management device 50 may make a YES judgment in step S1206 of Figure 20, place the tailgating determination zone 303 in tailgating permission mode, and classify the face-unregistered person 1B present in the tailgating determination zone 303 as tailgating person 1C.
[0254] (A2) In the example shown in Fig. 22, if at least one of face-registered persons 1A present in tailgating determination zone 303 is performing the first action, management device 50 may determine YES in step S1405 of Fig. 22, place tailgating determination zone 303 in tailgating permission mode, and classify unregistered face person 1B present in tailgating determination zone 303 as tailgating person 1C. Alternatively, in the example shown in Fig. 22, if all of face-registered persons 1A present in tailgating determination zone 303 are performing the first action, management device 50 may determine YES in step S1405 of Fig. 22, place tailgating determination zone 303 in tailgating permission mode, and classify unregistered face person 1B present in tailgating determination zone 303 as tailgating person 1C.
[0255] (A3) In the example shown in FIG. 23, the management device 50 sets tailgating determination zones 303 around each of the multiple face-registered persons 1A (S1506), and if a face-unregistered person 1B is present in any of the set tailgating determination zones 303, the face-unregistered person 1B may be determined to be a tailgating person 1C.
[0256] (A4) In the example shown in FIG. 25, when the distance L between any one of a plurality of face-registered persons 1A and an unregistered person 1B is equal to or less than a predetermined threshold, and both the face-registered person 1A and the unregistered person 1B perform a first action, the management device 50 may determine that the unregistered person 1B is an accompanying person 1C of the face-registered person 1A.
[0257] (A5) In the example shown in FIG. 26, when the distance L between any one of a plurality of face-registered persons 1A and an unregistered person 1B within the tailgating determination zone 303 is equal to or less than a predetermined threshold, and both the face-registered person 1A and the unregistered person 1B perform a first action, the management device 50 may determine that the unregistered person 1B is an accompanying person 1C of the face-registered person 1A.
[0258] <Registering face images of unregistered people who are allowed to accompany you> The management device 50 may register a facial image of the tailgating person 1C who is permitted to pass in the memory 1002 or the storage 1003 (see FIG. 32 ). This allows the management device 50 to treat the tailgating person 1C, who was initially permitted to pass as a face-unregistered person 1B, as a face-registered person 1A from the next time onwards when passing through the aisle 2. This allows the first tailgating person 1C to pass through the aisle 2 from the next time onwards, even if he or she is not accompanied by the face-registered person 1A.
[0259] When determining whether or not unregistered person 1B is tailgating person 1C, management device 50 may photograph the face of unregistered person 1B using first photographing device 21 or second photographing device 22. If unregistered person 1B is determined to be tailgating person 1C, management device 50 may register the photographed face image. At this time, management device 50 determines whether or not the photographed face image is of a quality usable for face matching. If the quality is usable for face matching, management device 50 may register the photographed face image. If the quality is not usable for face matching, management device 50 may not register the photographed face image. For example, management device 50 may determine that the photographed face image is not of a quality usable for face matching if the photographed face image is a face image wearing a mask, a face image with closed eyes, or a face not facing forward. For example, management device 50 may determine that the photographed face image is of a quality usable for face matching if the photographed face image is a face image not wearing a mask, has its eyes open, and is facing forward.
[0260] Furthermore, when registering the facial image of person 1B whose face has not been registered, the management device 50 may display the facial image to be registered and a message indicating that the facial image will be registered on the first display device 31 (or the second display device 32). This allows the accompanying person 1C to know that his or her facial image has been registered.
[0261] Whether or not to permit registration of the face image of the accompanying person 1C may be instructed by the face-recording person 1A who accompanies the accompanying person 1C.
[0262] (Hardware configuration) FIG. 32 is a diagram illustrating an example of a hardware configuration of a computer according to the present disclosure.
[0263] The above-described first edge device 41, second edge device 42, management device 50, and face matching device 60 may be configured to include a computer 1000 shown in Fig. 32. Furthermore, the processes performed by the above-described first edge device 41, second edge device 42, management device 50, and face matching device 60 may be realized by the computer 1000 shown in Fig. 32 executing a computer program.
[0264] The computer 1000 includes a processor 1001, a memory 1002, a storage 1003, an input device 1004, an output device 1005, a communication device 1006, a graphics processing unit (GPU) 1007, a reading device 1008, and a bus 1009. The processor 1001, the memory 1002, the storage 1003, the input device 1004, the output device 1005, the communication device 1006, the GPU 1007, and the reading device 1008 are connected to the bus 1009 and can send and receive data bidirectionally via the bus 1009.
[0265] The processor 1001 executes a computer program stored in the memory 1002 to realize the processing performed by each of the above-mentioned devices. Therefore, the processing performed by each of the above-mentioned devices can be interpreted as processing mainly performed by the processor 1001. Examples of the processor 1001 include a CPU (Central Processing Unit), an MPU (Micro Processing Unit), a controller, an LSI (Large Scale Integration), an ASIC (Application Specific Integrated Circuit), a PLD (Programmable Logic Device), and an FPGA (Field-Programmable Gate Array).
[0266] The memory 1002 stores computer programs and data used by the computer 1000. The memory 1002 may include a read-only memory (ROM) and a random access memory (RAM). The memory 1002 may be configured as either a volatile storage medium or a non-volatile storage medium.
[0267] The storage 1003 is configured as a non-volatile storage medium, and stores computer programs and data handled by the computer 1000. Examples of the storage 1003 include a hard disk drive (HDD), a solid state drive (SSD), and a flash memory.
[0268] The input device 1004 accepts data to be input to the processor 1001. Examples of the input device 1004 include a keyboard, a mouse, a touchpad, and a microphone.
[0269] The output device 1005 outputs data generated by the processor 1001. For example, the output device 1005 outputs display data generated by the processor 1001 to the first display device 31 or the second display device 32. Alternatively, the output device 1005 may be a display and display the display data itself. Alternatively, the output device 1005 outputs audio data generated by the processor 1001 to a speaker.
[0270] The communication device 1006 transmits and receives data to and from other devices via the communication network 4. The communication device 1006 may include a transmitter for transmitting data and a receiver for receiving data. The communication device 1006 may support either wired communication or wireless communication.
[0271] The GPU 1007 is a device that processes image rendering at high speed. The GPU 1007 may be used for AI processing (e.g., machine learning or deep learning).
[0272] The reading device 1008 is a device that reads data from an external recording medium such as a DVD-ROM (Digital Versatile Disk Read Only Memory) or a USB (Universal Serial Bus) memory.
[0273] (Summary of the second embodiment) The above description of the second embodiment discloses the following techniques.
[0274] <Technology 1> A person matching system (e.g., gate system 9) includes photographing devices (e.g., first photographing device 21, second photographing device 22) that photograph multiple people (1) moving through a predetermined passageway (2), and an information processing device (e.g., management device 50, face matching device 60) that matches the people using facial images (107) of the people photographed by the photographing devices. The information processing device sets a first area (e.g., face matching zone 301) and a second area (e.g., tailgating determination zone 303) in the passageway, and in the first area, performs matching using the facial images of people present in the first area, allowing people who are successfully matched to move through the passageway and not allowing people who are unsuccessful to move through the passageway. In the second area, it performs matching using the facial images of people present in the second area, allowing a first person who is successfully matched to move through the passageway, and allowing a second person who is unsuccessful to move through the passageway while the first person is present in the second area. According to the above configuration, when a first person (e.g., an employee) whose face is successfully matched moves down an aisle with a second person (e.g., a visitor) whose face is unsuccessful, the second person is permitted to move along the aisle by passing through the second area. This allows the first person to move down the aisle with the second person without having to request that the second person register their face image.
[0275] <Technology 2> In the person matching system described in Technology 1, the second area is set in advance within a predetermined range within the passageway, and the information processing device sets the second area to a state in which the second person is permitted to move through the passageway while the first person is present in the second area and when the first person performs a predetermined first action. According to the above configuration, when the first person is present in the second area and has performed the first action, the second person can pass through the second area. This reduces the possibility that the information processing device will allow a person other than the second person accompanied by the first person to move.
[0276] <Technology 3> In the person matching system described in Technology 1 or 2, when the first person performs a predetermined first action in the second area after the matching is successful, the information processing device sets a predetermined range including the first person as the second area, and sets the second area to a state in which the second person is permitted to move through the passage. According to the above configuration, a second area is set around the first person, and the second person in the second area is permitted to move along the passage. This reduces the possibility that the information processing device will permit a person other than the second person accompanied by the first person to move.
[0277] <Technology 4> In the person matching system described in any one of Techniques 1 to 3, the information processing device allows the second person to move through the passage when the distance between the first person and the second person in the second area is equal to or less than a predetermined threshold. According to the above configuration, a second person whose distance to the first person is equal to or less than a predetermined threshold is permitted to move along the passageway, thereby reducing the possibility that the information processing device will permit a person other than the second person accompanied by the first person to move along the passageway.
[0278] <Technology 5> The person matching system described in any one of Techniques 1 to 4 further includes a display device (e.g., a first display device 31, a second display device 32), and when the first person performs the first action, the information processing device causes the display device to display that the second area is in a state that allows the second person to move through the passage. According to the above configuration, the first person and / or the second person can know from the display on the display device whether or not the second area is in a state in which movement is permitted.
[0279] <Technology 6> The person matching system according to any one of techniques 1 to 4 further includes a display device, wherein when the first person performs a predetermined second action while the second area is in a state where movement through the passage is permitted for the second person, the information processing device sets the second area to a state where movement through the passage is not permitted for the second person, and causes the display device to display that the second area is in a state where movement through the passage is not permitted for the second person. According to the above configuration, the first person can switch the state of the passage to one in which movement is not permitted by performing the second action. Furthermore, the first person and / or the second person can know from the display on the display device whether the second area is in a state in which movement is permitted.
[0280] <Technology 7> The person matching system according to any one of techniques 1 to 3 further includes a display device, and when the first person performs a predetermined second action while the second area is in a state where the second person is permitted to move through the passage, the information processing device cancels the setting of the second area and causes the display device to display a message indicating that the second person is not permitted to move through the passage. According to the above configuration, the first person can switch the state of the passage to one in which movement is not permitted by performing the second action. Furthermore, the first person and / or the second person can know from the display on the display device whether the second area is in a state in which movement is permitted.
[0281] <Technology 8> In the person matching system described in Technology 6 or 7, when the first person performs a predetermined second action while the second area is in a state where the second person is permitted to move through the passage, the information processing device continues to maintain the state of permitted tailgating for the second person who was permitted to tailgate before the first person performed the second action. According to the above configuration, even if a first person performs a predetermined second action, the second person who was permitted to tailgate before the first person performed the second action can continue to maintain the state of being permitted to tailgate.
[0282] <Technology 9> In the person matching system described in Technology 6 or Technology 7, when the first person performs a predetermined second action while the second area is in a state where the second person is permitted to move along the aisle, the information processing device cancels the state of tailgating permission for the second person who was permitted to move along the aisle before the first person performed the second action, and displays on the display device that the second person is not permitted to move along the aisle. According to the above configuration, when a first person performs a second action, the tailgating permission state can be cancelled for a second person who was permitted to tailgate before the first person performed the second action.
[0283] <Technology 10> The person matching system according to any one of techniques 1 to 9 further includes a terminal (400), and when the information processing device detects the second person in the second area, the information processing device transmits a facial image of the second person captured by the photographing device to the terminal. According to the above configuration, a face image of the second person detected in the second area is transmitted to the terminal, thereby allowing the person carrying the terminal (e.g., the first person) to know that the second person has been detected.
[0284] <Technology 11> The person matching system described in Technology 10 further includes a display device, wherein the terminal displays a facial image of the second person received from the information processing device, accepts input as to whether or not to allow the second person to move through the passage, and transmits the result of the input to the information processing device, and the information processing device displays information regarding the result of the input received from the terminal on the display device. According to the above configuration, a person carrying the terminal (for example, a first person) can decide which of the detected second persons to allow to move along the passageway, thereby preventing the information processing device from allowing a person who is not accompanied by the first person to move along the passageway.
[0285] <Technology 12> In the person matching system described in any one of Techniques 1 to 11, the information processing device permits the second person to move through the passage when the first person is successfully matched using the facial image and has the authority to move through the passage. According to the above configuration, if the first person whose face has been successfully matched has the authority to move through the passage, the second person is also permitted to move through the passage. As a result, if the first person does not have the authority to move through the passage, the second person accompanied by the first person is not permitted to move through the passage.
[0286] <Technology 13> In the person matching system described in any one of Techniques 1 to 3, the information processing device stores, in a predetermined storage unit, facial images of a quality that can be used for matching using the facial images, from among the facial images of the second person who has been permitted to move through the passageway and that have been photographed by the photographing device. According to the above configuration, among the facial images of the second person, the facial images that can be used for face matching are stored in the storage unit. As a result, from the next time onwards, the second person can move along the passageway without being accompanied by the first person, as face matching is performed using the facial images stored in the storage unit.
[0287] <Technology 14> The person verification system according to Technique 13 further includes a display device, and when the information processing device stores the face image of the second person in the storage unit, the information processing device causes the display device to display a message to that effect. According to the above configuration, the second person can know that the face image of the second person has been stored in the storage unit.
[0288] <Technology 15> An information processing device that matches multiple people moving through a specified passageway using facial images of the people photographed by a photographing device that photographs the people includes a processor (1001) and a memory (1002), and the processor works in cooperation with the memory to set a first area and a second area in the passageway, and in the first area, performs a match using the facial images on people present in the first area, allowing people who are successfully matched to move through the passageway and not allowing people who are unsuccessful to move through the passageway, and in the second area, performs a match using the facial images on people present in the second area, allowing a first person who is successfully matched to move through the passageway and allowing a second person who is unsuccessful to match to move through the passageway while the first person is present in the second area. According to the above configuration, when a first person (e.g., an employee) whose face is successfully matched moves down an aisle with a second person (e.g., a visitor) whose face is unsuccessful, the second person is permitted to move along the aisle by passing through the second area. This allows the first person to move down the aisle with the second person without having to request that the second person register their face image.
[0289] <Technology 16> A person matching method for matching a plurality of people moving through a predetermined passageway using facial images of the people photographed by a photographing device includes setting a first area and a second area in the passageway, performing a match using the facial images of people present in the first area in the first area, allowing people who are successfully matched to move through the passageway, and not allowing people who are unsuccessful to match to move through the passageway, and performing a match using the facial images of people present in the second area in the second area, allowing a first person who is successfully matched to move through the passageway, and allowing a second person who is unsuccessful to match to move through the passageway while the first person is present in the second area. According to the above method, when a first person (e.g., an employee) whose face is successfully matched moves down an aisle with a second person (e.g., a visitor) whose face is unsuccessful, the second person is permitted to move along the aisle by passing through the second area. This allows the first person to move down the aisle with the second person without having to request that the second person register their face image.
[0290] <Technology 17> The person matching program causes a computer to execute the person matching method described in the above technique 14. According to the person matching program, when a first person (e.g., an employee) whose face matches successfully moves down an aisle with a second person (e.g., a visitor) whose face matches unsuccessfully, the second person is permitted to move along the aisle by passing through the second area. This allows the first person to move down the aisle with the second person without having to request that the second person register their face image.
[0291] Although the embodiments have been described above with reference to the accompanying drawings, the present disclosure is not limited to such examples. It is clear that a person skilled in the art can conceive of various modifications, alterations, substitutions, additions, deletions, and equivalents within the scope of the claims, and it is understood that these also fall within the technical scope of the present disclosure. Furthermore, the components in the above-described embodiments may be combined in any manner without departing from the spirit of the invention. [Industrial Applicability]
[0292] The technology of the present disclosure is useful for realizing a system, device, method, computer program, etc. that takes into account a person who does not have the right of passage being accompanied by a person who has the right of passage. [Explanation of symbols]
[0293] 1 person 1A Face Registration Person 1B Person with unregistered face 1C Tailgating person 2 aisles 3 Detection Area 4. Communication Network 9 Gate System 10 Gate device 11 Gate 1 12 Gate 2 21 First Imaging Device 22 Second imaging device 31 1st display device 32 2nd display device 41 First Edge Device 42 Second Edge Device 50 Management device 60 Face matching device 100 Personal information 102 First image capture time 103 Image update time 104 Body Images 105 Body image update time 106 Body coordinates 107 Facial Images 108 Face image update time 109 Face Coordinates 110 Face Matching ID 111 Matching status 112 Direction of movement 200 Body Images 201, 201A, 201B body coordinates 202 Body coordinates 220 facial images 221, 221A face coordinates 301 Face Matching Zone 302 Passage Judgment Zone 303 Tailgating Judgment Zone 311 Designated area closer to Gate 1 312 Designated area closer to the second gate 400 terminals 401 screen 411 Tailgating Target Button 412 Tailgating Exclusion Button 1000 computers 1001 processor 1002 memory 1003 Storage 1004 Input Device 1005 Output Device 1006 Communication equipment 1007 GPU 1008 Reading device 1009 Bus
Claims
1. An image capturing device that captures images of a plurality of people moving along a predetermined passageway; an information processing device that collates a face image of a person captured by the photographing device, The information processing device includes: A first area and a second area are set in the passage, In the first area, a match is performed using the face image for a person present in the first area, and a person who has been successfully matched is permitted to move through the passage, and a person who has been unsuccessful in the match is not permitted to move through the passage; In the second area, a match is performed using the face image of a person present in the second area, and a first person who has been successfully matched is permitted to move along the passage, and a second person who has not been successfully matched is permitted to move along the passage while the first person is present in the second area. Person matching system.
2. The second area is preset within a predetermined range within the passage, the information processing device sets the second area to a state in which movement of the second person through the passage is permitted while the first person is present in the second area and when the first person performs a predetermined first action; The person verification system according to claim 1 .
3. when the first person performs a predetermined first action in the second area after the matching is successful, the information processing device sets a predetermined range including the first person as the second area, and sets the second area to a state in which movement of the second person through the passage is permitted. The person verification system according to claim 1 .
4. the information processing device permits movement of the second person through the passage when a distance between the first person and the second person in the second area is equal to or smaller than a predetermined threshold value; The person verification system according to any one of claims 1 to 3.
5. Further comprising a display device, and when the first person performs the first action, the information processing device causes the display device to display a message indicating that the second area is in a state in which movement of the second person through the passage is permitted. The person verification system according to claim 2 or 3.
6. Further comprising a display device, When the first person performs a predetermined second action in a state in which the second area permits the second person to move along the passage, the information processing device sets the second area to a state in which the second person is not permitted to move along the passage, and causes the display device to display a message indicating that the second area is in a state in which the second person is not permitted to move along the passage. The person verification system according to claim 2 or 3.
7. Further comprising a display device, when the first person performs a predetermined second action in a state in which the second area permits the second person to move through the passage, the information processing device cancels the setting of the second area and causes the display device to display a message indicating that the second person is not permitted to move through the passage. The person verification system according to claim 3 .
8. When the first person performs a predetermined second action in a state in which the second area permits the second person to move through the passage, the information processing device continues to maintain a state of tailgating permission for the second person who was permitted to tailgate before the first person performed the second action. The person verification system according to claim 6.
9. When the first person performs a predetermined second action in a state in which the second area permits the second person to move along the passage, the information processing device cancels the state of tailgating permission for the second person who was permitted to move along the passage before the first person performed the second action, and causes the display device to display a message indicating that the second person is not permitted to move along the passage. The person verification system according to claim 6.
10. Further comprising a terminal; When the information processing device detects the second person in the second area, the information processing device transmits a face image of the second person photographed by the photographing device to the terminal. The person verification system according to any one of claims 1 to 3.
11. Further comprising a display device, the terminal displays the face image of the second person received from the information processing device, receives an input of whether or not to permit the second person to move through the passage, and transmits a result of the input to the information processing device; the information processing device causes the display device to display information regarding the result of the input received from the terminal; The person verification system according to claim 10.
12. the information processing device permits the second person to move through the passage when the first person is successful in matching using the face image and has authority to move through the passage; The person verification system according to any one of claims 1 to 3.
13. the information processing device stores in a predetermined storage unit, among the facial images of the second person who has been permitted to move through the passage and which have been photographed by the photographing device, a facial image of a quality usable for matching using the facial image; The person verification system according to any one of claims 1 to 3.
14. Further comprising a display device, When the facial image of the second person is stored in the storage unit, the information processing device causes the display device to display a message to that effect. The person verification system according to claim 13.
15. An information processing device that matches a person using face images of a person captured by a photographing device that photographs a plurality of people moving along a predetermined passage, the information processing device comprising a processor and a memory, the processor working in cooperation with the memory, A first area and a second area are set in the passage, In the first area, a match is performed using the face image for a person present in the first area, and a person who has been successfully matched is permitted to move through the passage, and a person who has been unsuccessful in the match is not permitted to move through the passage; In the second area, a match is performed using the face image of a person present in the second area, and a first person who has been successfully matched is permitted to move along the passage, and a second person who has not been successfully matched is permitted to move along the passage while the first person is present in the second area. Information processing device.
16. 1. A person verification method for verifying a person by using face images of a plurality of people photographed by an imaging device that photographs the people moving along a predetermined passageway, comprising: A first area and a second area are set in the passage, In the first area, a match is performed using the face image for a person present in the first area, and a person who has been successfully matched is permitted to move through the passage, and a person who has been unsuccessful in the match is not permitted to move through the passage; In the second area, a match is performed using the face image of a person present in the second area, and a first person who has been successfully matched is permitted to move along the passage, and a second person who has not been successfully matched is permitted to move along the passage while the first person is present in the second area. Person matching methods.
17. A method for verifying a person according to claim 16, Person matching program.
Citation Information
Patent Citations
Entrance management system
JP2020088462A
