Authentication apparatus, authentication method, and recording medium

Abstract

To provide an authentication device, an authentication method and a recording medium for securing the chance of the face authentication of a person to be authenticated by performing the face authentication of the person to be authenticated when the face of the person to be authenticated is included in an image while tracking an authenticating person.SOLUTION: The authentication device 1 includes an acquisition unit that acquires an image including a predetermined region through which a person to be authenticated passes, a tracking unit that tracks the person to be authenticated on the basis of a tracking region of the person to be authenticated including a head of the person to be authenticated in the image, and a face authentication unit that performs face authentication of the person to be authenticated in a case where a face of the person to be authenticated is included in the image while the tracking unit is tracking the person to be authenticated.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] The present disclosure relates to the technical fields of an authentication device, an authentication method, and a recording medium. [Background technology]

[0002] Patent Literature 1 describes a technology that provides a digital image of at least one user detected while passing through a gateless controlled area, extracts a first face detection data set from the provided digital image of the detected user, provides a second face detection data set stored in a wearable device carried by the detected user using an electronic reader, applies facial recognition to the first and second face detection data sets to determine whether the user associated with the first face detection data set is the same as the user associated with the second face detection data set, and identifies the detected users as authorized if they are identical, and as unauthorized otherwise. Patent Literature 2 describes a technology that a server device that acquires a facial image of a search target and acquires a search start area for the search target transmits the search area facial image and instructions for imaging and authentication operations to a camera device that is provided in a vehicle and has a processing unit that recognizes faces included in the image and authenticates whether the recognized face matches a face included in the facial image, and the camera device performs an imaging operation based on the imaging operation instruction if it finds itself within the search area based on its current location. Patent document 3 describes a technology for outputting output information based on first information, which includes identification information assigned to a mobile terminal and location information regarding the location of the mobile terminal, for identifying an identified object within a target space, and second information, which is information based on an image of the target space and for identifying an identified object within the target space. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Special Publication No. 2020-524928 [Patent Document 2] Patent Publication No. 2021-121050 [Patent Document 3] Japanese Patent Application Publication No. 2018-182705 Summary of the Invention [Problem to be solved by the invention]

[0004] The present disclosure aims to provide an authentication device, an authentication method, and a recording medium that aim to improve upon the techniques described in prior art documents. [Means for solving the problem]

[0005] One aspect of the authentication device includes an acquisition means for acquiring an image including a predetermined area through which a person to be authenticated passes, a tracking means for tracking the person to be authenticated based on a tracking area of ​​the person to be authenticated that includes the head of the person to be authenticated in the image, and a face authentication means for performing face authentication of the person to be authenticated when the face of the person to be authenticated is included in the image while the tracking means is tracking the person to be authenticated.

[0006] One aspect of the authentication method involves acquiring an image including a predetermined area through which the person to be authenticated passes, tracking the person based on a tracking area of ​​the person to be authenticated that includes the person's head in the image, and performing facial authentication of the person to be authenticated if the person's face is included in the image while being tracked.

[0007] In one embodiment of the recording medium, a computer program is recorded on a computer to execute an authentication method in which an image including a predetermined area through which a person to be authenticated passes is acquired, the person is tracked based on a tracking area of ​​the person to be authenticated that includes the person's head in the image, and if the person's face is included in the image while the person to be authenticated is being tracked, the face of the person to be authenticated is authenticated. [Brief explanation of the drawings]

[0008] [Figure 1] FIG. 1 is a block diagram showing the configuration of an authentication device in the first embodiment. [Figure 2] FIG. 2 is a block diagram showing the configuration of an authentication device in the second embodiment. [Figure 3] FIG. 3 is a conceptual diagram showing an example of a situation in which the authentication device according to the second embodiment is applied. [Figure 4] FIG. 4 is a conceptual diagram showing an example of a determination operation performed by the authentication device in the second embodiment. [Figure 5] FIG. 5 is a flowchart showing the flow of authentication operations performed by the authentication device in the second embodiment. [Figure 6] FIG. 6 is a block diagram showing the configuration of an authentication device in the third embodiment. [Figure 7] FIG. 7 is a flowchart showing the flow of authentication operations performed by the authentication device in the third embodiment. [Figure 8] FIG. 8 is a flowchart showing the flow of the display control operation performed by the authentication device in the third embodiment. [Figure 9] FIG. 9 shows an example of a display on a display in the third embodiment. [Figure 10] FIG. 10 is a block diagram showing the configuration of an authentication device in the fourth embodiment. [Figure 11] FIG. 11 is a flowchart showing the flow of the ReID operation performed by the authentication device in the fourth embodiment. [Figure 12] FIG. 12 is a flowchart showing the flow of authentication operations performed by the authentication device in the fourth embodiment. [Figure 13] FIG. 13 is a block diagram showing the configuration of an authentication device in the fifth embodiment. [Figure 14] FIG. 14 is a conceptual diagram showing an example of a situation in which the authentication device according to the fifth embodiment is applied. [Figure 15] FIG. 15 is a conceptual diagram showing a first application example of the authentication device in the sixth embodiment. [Figure 16] FIG. 16 is a conceptual diagram showing a second application example of the authentication device in the sixth embodiment. [Figure 17] FIG. 17 is a conceptual diagram showing an example of a tracking operation of the authentication device in the seventh embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0009] Hereinafter, embodiments of an authentication device, an authentication method, and a recording medium will be described with reference to the drawings. [1: First embodiment]

[0010] A first embodiment of an authentication device, an authentication method, and a recording medium will be described below. In the following, the first embodiment of the authentication device, the authentication method, and the recording medium will be described using an authentication device 1 to which the first embodiment of the authentication device, the authentication method, and the recording medium is applied. [1-1: Configuration of authentication device 1]

[0011] The configuration of an authentication device 1 in the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of an authentication device 1 in the first embodiment.

[0012] As shown in Fig. 1, the authentication device 1 includes an acquisition unit 11, a tracking unit 12, and a face authentication unit 13. The acquisition unit 11 acquires an image IMG including a predetermined area through which the person to be authenticated passes. The tracking unit 12 tracks the person to be authenticated based on a tracking area of ​​the person to be authenticated that includes the head of the person to be authenticated in the image IMG. If the face of the person to be authenticated is included in the image IMG while the tracking unit 12 is tracking the person to be authenticated, the face authentication unit 13 performs face authentication of the person to be authenticated. [1-2: Technical Effects of Authentication Device 1]

[0013] If the face of the person to be authenticated is included in the image IMG while the tracking unit 12 is tracking the person to be authenticated, the face of the person to be authenticated is authenticated, so that an opportunity for face authentication of the person to be authenticated can be secured. [2: Second embodiment]

[0014] A second embodiment of the authentication device, authentication method, and recording medium will be described below. The second embodiment of the authentication device, authentication method, and recording medium will be described below using an authentication device 2 to which the second embodiment of the authentication device, authentication method, and recording medium is applied. [2-1: Configuration of authentication device 2]

[0015] The configuration of the authentication device 2 in the second embodiment will be described with reference to Fig. 2. Fig. 2 is a block diagram showing the configuration of the authentication device 2 in the second embodiment.

[0016] 2, the authentication device 2 includes a calculation device 21 and a storage device 22. The authentication device 2 may further include a communication device 23, an input device 24, and an output device 25. However, the authentication device 2 does not necessarily include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0017] The arithmetic device 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (e.g., an input device 24 described later) (not shown) included in the authentication device 2. The arithmetic device 21 may acquire (i.e., download or read) the computer program from a device (not shown) located outside the authentication device 2 via the communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, logical functional blocks for executing operations to be performed by the authentication device 2 are realized within the arithmetic device 21. That is, the arithmetic device 21 can function as a controller for realizing logical function blocks for executing the operations (in other words, processing) that the authentication device 2 should perform.

[0018] FIG. 2 shows an example of logical functional blocks implemented in the computing device 21 to perform authentication operations. As shown in FIG. 2, the computing device 21 implements an acquisition unit 211 as a specific example of "acquisition means," a tracking unit 212 as a specific example of "tracking means," a face authentication unit 213 as a specific example of "authentication means," and a determination unit 214 as a specific example of "determination means." If face authentication of the person being tracked by the tracking unit 212 has not been successful even once until the person leaves a predetermined area, the determination unit 214 determines that face authentication of the person has failed. However, the determination unit 214 does not necessarily have to be implemented in the computing device 21. Details of the operations of the acquisition unit 211, the tracking unit 212, the face authentication unit 213, and the determination unit 214 will be described later.

[0019] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data temporarily used by the arithmetic device 21 when the arithmetic device 21 is executing a computer program. The storage device 22 may store data to be stored long-term by the authentication device 2. The storage device 22 may include at least one of a random access memory (RAM), a read-only memory (ROM), a hard disk drive, a magneto-optical disk drive, a solid-state drive (SSD), and a disk array device. In other words, the storage device 22 may include a non-transitory recording medium. The storage device 22 may store a facial feature database FC and an ID correspondence table IDT. The facial feature database FC may store an authentication ID as the ID of a registered person and the feature of a facial image of the registered person (sometimes referred to as "facial feature"). The ID correspondence table IDT will be described in detail with reference to Table 1. However, the storage device 22 does not necessarily have to store at least one of the facial feature database FC and the ID correspondence table IDT.

[0020] The communication device 23 is capable of communicating with devices external to the authentication device 2 via a communication network (not shown). The communication device 23 may acquire an image IMG used in the authentication operation from a camera CAM (described later) via the communication network.

[0021] The input device 24 is a device that accepts information input to the authentication device 2 from outside the authentication device 2. For example, the input device 24 may include an operation device (for example, at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the authentication device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the authentication device 2.

[0022] The output device 25 is a device that outputs information to the outside of the authentication device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (a so-called display) that can display an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (a so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (a so-called printer) that can print desired information on paper. [2-2: Application example of authentication device 2]

[0023] Next, an application example of the authentication device 2 in the second embodiment will be described with reference to Fig. 3. Fig. 3 is a conceptual diagram showing an example of a scene in which the authentication device 2 in the second embodiment is applied. The authentication device 2 in the second embodiment may be applied to an authentication operation of a person to be authenticated to determine whether or not the person to be authenticated is permitted to enter a restricted area LA provided in, for example, a theme park, an airport, a train station, an office, an event venue, etc. For example, a mechanism for authenticating a person to be authenticated, in which a general flapper gate or a face recognition terminal is installed, may be replaced with the authentication device 2 that authenticates the person by passing through the authentication area RA.

[0024] In the scene illustrated in FIG. 3, consider a passageway with people flowing from left to right toward the restricted area LA shown on the right side of FIG. 3. A virtual authentication area RA is provided just before the entrance to the restricted area LA. The authentication device 2 authenticates people present in the authentication area RA as those to be authenticated. The authentication area RA may be, for example, an area of ​​5 meters by 5 meters. Five to six people to be authenticated can pass through the 5-meter width at the same time, and 20 to 30 people to be authenticated may exist and pass through the authentication area RA. In the scene illustrated in FIG. 3, as indicated by the arrows, authenticatees A, B, and C are moving from the authentication area RA toward the restricted area LA.

[0025] The camera CAM may be installed near the exit of the authentication area RA. The camera CAM may capture an image of a predetermined space to capture an image of the person to be authenticated. The camera CAM may be a high-resolution camera to perform facial authentication in the authentication area RA with high accuracy. For example, the camera CAM may be a 4K resolution camera. The camera CAM may be installed at a position or height that does not impede the passage of the person to be authenticated. The camera CAM may be installed at a height higher than the head of the person to be authenticated. The camera CAM may be installed at a height of, for example, 2.5 meters. The camera CAM may capture an area including the authentication area RA. The authentication area RA does not have to be an area captured in the image IMG, and may be an area with a slightly narrower angle of view.

[0026] The person to be authenticated may not be explicitly informed that they are being authenticated; for example, they may be lined up and authenticated in order. The person to be authenticated may be authenticated while passing through the authentication area RA. The authentication device 2 may be able to authenticate the person to be authenticated while walking normally, or the person to be authenticated may be authenticated without stopping and being aware that they are being authenticated. The person to be authenticated may be authenticated even if they turn around or crouch while passing through the authentication area RA, temporarily obscuring their face. The authentication device 2 may also be able to authenticate multiple people to the authentication device 2 administrator or the like at the same time. The authentication device 2 administrator or the like may be able to immediately check the authentication results of the people to be authenticated. The person to be authenticated can experience smooth authentication. For example, using a flapper gate or a facial recognition terminal to direct the person to the camera and authenticate the person to be authenticated one by one can improve convenience for the person to be authenticated.

[0027] Furthermore, for example, in walk-through authentication, there are restrictions on installation locations, such as adding lanes, in order to improve throughput. In contrast, with the authentication of this embodiment, there are no restrictions on locations for improving throughput. Furthermore, since the authentication device 2 has a high degree of freedom, it is also suitable for situations where the design of the space is important, such as theme parks.

[0028] To make authentication more successful, it is desirable for the person to be authenticated to stay in the authentication area RA for the same amount of time on average. Therefore, for example, an operational rule may be established to prevent the person to be authenticated from moving from the back to the front and entering from the left or right areas shown in the drawing. Also, an eye-catching image may be installed near the camera CAM to make it easier to include the person to be authenticated's face in the image IMG. [2-3: Tracking operation performed by the tracking unit 212]

[0029] The tracking unit 212 tracks the person to be authenticated based on the tracking area of ​​the person to be authenticated, including the head of the person to be authenticated, in the image IMG. The head can be detected even when the person to be authenticated is facing away from the front, such as facing backwards or sideways. Furthermore, if the person to be authenticated overlaps with another person to be authenticated, the whole body cannot be seen, making stable tracking by detecting the whole body difficult. In contrast, particularly when the camera CAM is installed at a higher height than the head, the head of the person to be authenticated is less likely to overlap with other people to be authenticated and is therefore more easily detected. Therefore, stable tracking can be achieved by detecting the head. Even when the head overlaps with other people to be authenticated, head detection is often successful. Therefore, in the second embodiment, the tracking unit 212 tracks the person to be authenticated using the head.

[0030] The "tracking area of ​​the person to be authenticated including the head" may be an area including the head and shoulders of the person to be authenticated, an area of ​​the upper body of the person to be authenticated, or an area of ​​the whole body of the person to be authenticated. In the following, an example will be described in which the "tracking area of ​​the person to be authenticated including the head" is the "head".

[0031] When an image IMG is input, the tracking unit 212 may output the head positions associated with tracking IDs for the number of people to be authenticated present in the image IMG. The tracking ID may not be an ID that identifies the person to be authenticated, but may be an ID that associates the same person present in different image IMGs.

[0032] The image IMG may be one frame of a video. The tracking performed by the tracking unit 212 may be an operation of identifying and associating the same person to be authenticated between the previous frame and the current frame. The tracking performed by the tracking unit 212 may be performed by determining the head position in the image IMG. Since the movement of the person to be authenticated is often small and the change in head position is often small between frames, the tracking unit 212 can perform association based on the head image and head position. For example, optical flow may be used to associate the person to be authenticated between frames. The tracking performed by the tracking unit 212 may be performed by determining the identity of the head pattern or by determining the similarity of the feature amounts of the head image. In this embodiment, the head image does not need to be used to identify the individual, but may be used to associate the person to be authenticated between frames.

[0033] The tracking unit 212 may use the frame immediately preceding the current frame as the previous frame, or may use a frame two or more frames preceding the current frame. The tracking unit 212 may also use multiple frames prior to the current frame. For example, the tracking unit 212 may use information from the past five frames to track the previous frame and the current frame. The tracking unit 212 may also use information from all past frames to track the current frame.

[0034] The tracking unit 212 may detect and track the head of the person to be authenticated while passing through the authentication area RA. The tracking unit 212 may continue to track a person to be authenticated who is passing through the authentication area RA and whose face authentication has not been successful, or a person to be authenticated whose face is not visible. The tracking unit 212 may detect and continue to track the upper body including the head even when the face cannot be detected. The tracking unit 212 may continue to track even when the person to be authenticated turns away or is blocked by another person. [2-4: Target of face authentication operation performed by authentication device 2]

[0035] When the face of the person to be authenticated is detected, the face authentication unit 213 may perform face authentication in the following cases (1) to (3). (1): In the case of a subject to which a new tracking ID is associated The face authentication unit 213 may perform face authentication on a person to be authenticated who has newly appeared in the image IMG, to identify who the person is. (2): In the case of a person to be authenticated who is not a new tracking ID but has a tracking ID that has not yet been successfully authenticated. The face authentication unit 213 may perform face authentication because it has not yet identified who the person to be authenticated is. If the person to be authenticated is not registered in the face feature database FC, the case (2) applies. On the other hand, even in the case of (2), face authentication does not have to be performed for all frames. For example, the face authentication unit 213 may perform face authentication for every predetermined number of frames. The predetermined number may be determined according to the frame rate and the processing speed of face authentication. Alternatively, the predetermined number may be increased to reduce the processing load of face authentication. Furthermore, the face authentication unit 213 may perform face authentication when the face image is of a predetermined quality or higher. For example, if the person to be authenticated is facing sideways, it is possible that authentication will not be successful no matter how many times face authentication is performed. In this case, for example, the face authentication unit 213 may perform face authentication when the face of the person to be authenticated is facing forward. (3): If the person has already been successfully authenticated, but a certain period of time has passed since the successful authentication. In case the person to be authenticated has been replaced due to a tracking error by the tracking unit 212 or the like, the face authentication unit 213 may perform face authentication again on the person to be authenticated for which a certain time has passed since the previous matching time.

[0036] For example, if the average time it takes a person to pass through the authentication area RA is 5 seconds, the determination unit 214 may determine that the person to be authenticated may have been replaced if 10 seconds or more have passed since the last verification time. The elapsed period may be determined arbitrarily depending on the size of the authentication area RA and the type of location to which it is applied. For example, in a theme park, the movement speed of the person to be authenticated may be slower than in an airport, so the elapsed period may be set longer. Alternatively, in a theme park, the person to be authenticated is likely to turn their face in various directions, making face detection relatively difficult, so the elapsed period may be set longer.

[0037] The face authentication unit 213 may perform face authentication in at least one of the following cases: when the tracking ID is seen for the first time; when the tracking ID has not been successfully authenticated before; and when a predetermined period of time has passed since the last successful face authentication. [2-5: ID correspondence table IDT]

[0038] The determination unit 214 may associate the tracking ID of the person being tracked with an authentication ID registered in advance in the face feature amount database FC, and register the ID in an ID correspondence table IDT, an example of which is shown in Table 1 below. [Table 1: ID correspondence table IDT]

[0039] [Table 1]

[0040] If the face authentication unit 213 can authenticate that the person is a registered person through face matching, the determination unit 214 may store the tracking ID and the authentication ID of the registered person in association with each other in the ID correspondence table IDT. A registered person is a person whose facial features are stored in the facial feature database FC.

[0041] Furthermore, the determination unit 214 may store the time at which the matching was performed as the "last matching time" in the ID correspondence table IDT, regardless of whether the authentication was successful or unsuccessful. The determination unit 214 may determine that tracking may have failed if a predetermined period of time or more has passed since the last matching time. That is, even if authentication has already been successful, i.e., an authentication ID has been registered, the face authentication unit 213 may perform an authentication operation if a predetermined period of time or more has passed since the last matching time. Furthermore, even if authentication has not yet been successful, i.e., an authentication ID has not been registered, the determination unit 214 can determine that face detection has been successful by referring to the last matching time. [2-6: Determination operation by the determination unit 214]

[0042] The determination unit 214 may determine that face authentication of the person to be authenticated has been successful if authentication is successful at any timing while tracking the person to be authenticated. The determination unit 214 may determine that face authentication of the person to be authenticated has been successful if authentication is successful within a predetermined time. For example, the determination unit 214 may determine that face authentication of the person to be authenticated has been successful if authentication is successful within 8 seconds from the start of tracking of the person to be authenticated. The determination unit 214 may determine that face authentication of the person to be authenticated has been successful if authentication is successful while the person is staying in a predetermined space. The determination unit 214 may also determine that face authentication is successful if the face is seen at least once within the authentication area RA and face authentication is successful. When the person to be authenticated leaves the authentication area RA, the determination unit 214 may refer to the ID correspondence table IDT to determine whether face authentication of the person to be authenticated has been successful at least once within the authentication area RA.

[0043] Even if the person to be authenticated while passing through the authentication area RA turns around or is blocked by another person to be authenticated in front of them, and the facial area is hidden, if the face is detected in a case other than the above and authentication is successful, the judgment unit 214 may judge that authentication is successful.

[0044] On the other hand, if a person to be authenticated passes by while hiding their face the whole time, their face cannot be seen and therefore facial authentication cannot be performed. If the face cannot be recognized because it is not visible, the determination unit 214 may determine that authentication has failed. Furthermore, the determination unit 214 may determine that authentication has failed if face authentication has never been successful while the person to be authenticated is passing through the authentication area RA.

[0045] FIG. 4 is a conceptual diagram showing an example of authentication operation by authentication device 2 in the second embodiment. Assume that person-to-be-authenticate A is tracked throughout his / her movement within authentication area RA, but face detection has not been possible. In other words, assume that face authentication operation has not been possible. In this case, determination unit 214 may determine that authentication of person-to-be-authenticate A has failed when person-to-be-authenticate A leaves authentication area RA. For example, person-to-be-authenticate A faces forward near the entrance to authentication area RA, but his / her face is hidden by others and cannot be seen, and even near the exit of authentication area RA, he / she faces sideways and cannot be seen. In this case, face matching of person-to-be-authenticate A is not possible, and therefore determination unit 214 determines that authentication of person-to-be-authenticate A has failed.

[0046] Assume that face detection of person-to-be-authentication B was not possible when person-to-be-authentication B left the authentication area RA, but face authentication was successful while person-to-be-authentication B was moving within the authentication area RA. In this case, the determination unit 214 may determine that authentication of person-to-be-authentication B was successful when person-to-be-authentication B left the authentication area RA. For example, person-to-be-authentication B looked down at the exit of the authentication area RA and so face detection was not possible, but face detection was possible while person-to-be-authentication B was staying in the authentication area RA, face authentication by the face authentication unit 213 was successful, and tracking by the tracking unit 212 continued, so the determination unit 214 may determine that authentication of person-to-be-authentication B was successful.

[0047] Although person C is tracked throughout the movement of person C within the authentication area RA, for example, he / she is looking to the side or is hidden by person B, and therefore face detection is not possible. Furthermore, face matching fails when person C leaves the authentication area RA. In this case, the determination unit 214 may determine that authentication of person C has failed when person C leaves the authentication area RA. For example, person C is looking to the side near the entrance of the authentication area RA and his / her face cannot be detected, but his / her face is detected at the exit of the authentication area RA and face authentication is performed, but the face cannot be matched with any of the registered persons. In this case, the determination unit 214 may determine that authentication of person C has failed. [2-7: Authentication operation for each image IMG]

[0048] The flow of authentication operations performed by the authentication device 2 in the second embodiment will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of authentication operations performed by the authentication device 2 in the second embodiment. The operations from "Start" to "End" shown in Fig. 5 may be operations for each image IMG.

[0049] 5, the acquisition unit 211 acquires an image IMG including a predetermined area through which the person to be authenticated passes (step S20). The acquisition unit 211 may acquire an image IMG including a predetermined area through which the person to be authenticated passes at predetermined time intervals. The acquisition unit 211 may acquire one frame of a moving image from the camera CAM. The acquisition unit 211 may acquire 30 frames per second from the camera CAM.

[0050] The tracking unit 212 tracks the person to be authenticated based on the head of the person to be authenticated in the image IMG (step S21). The tracking unit 212 may detect the head from the acquired image IMG and identify the tracking ID.

[0051] The tracking unit 212 determines whether the number of heads is one or more (step S22). If the number of heads is one or more (step S22: Yes), the determination unit 214 refers to the corresponding tracking ID column in the ID correspondence table IDT (step S23). The determination unit 214 determines whether the authentication ID is included in the corresponding tracking ID column in the ID correspondence table IDT (step S24). If the authentication ID is included in the corresponding tracking ID column (step S24: Yes), the determination unit 214 determines whether it is within a predetermined period since the authentication of the corresponding authenticated person was successful (step S25). The determination unit 214 may also determine whether the elapsed time from the previous matching time included in the ID correspondence table IDT is within a predetermined period. If it is within the predetermined period since the authentication of the authenticated person was successful (step S25: Yes), the process proceeds to step S29.

[0052] If the authentication ID is not included in the corresponding tracking ID field (step S24: No), or if a predetermined period of time has elapsed since the authentication target was successfully authenticated (step S25: No), the face authentication unit 213 determines whether the face of the corresponding person to be authenticated has been detected (step S26). If the face of the corresponding person to be authenticated has been detected (step S26: Yes), the face authentication unit 213 performs face matching (step S27). The face authentication unit 213 may extract features of the detected face image and match them with features registered in a face feature database FC to identify the person to be authenticated.

[0053] If the determination unit 214 determines that face authentication of the person to be authenticated being tracked by the tracking unit 212 has been successful in the past, the face authentication unit 213 may not perform face authentication even if the face of the person to be authenticated is included in the image IMG. On the other hand, even if the determination unit 214 determines that face authentication of the person to be authenticated being tracked by the tracking unit 212 has been successful in the past, the face authentication unit 213 may perform face authentication if a predetermined period has passed since the previous successful face authentication. If the face of the person to be authenticated is included in the image IMG while the tracking unit 212 is tracking the person to be authenticated, the face authentication unit 213 may perform face authentication of the person to be authenticated. If the face of the person to be authenticated cannot be detected (step S26: No), the process proceeds to step S29.

[0054] The determination unit 214 updates the ID correspondence table IDT based on the result of the face authentication (step S28). The tracking unit 212 determines whether or not there is an unprocessed head (step S29). If there is an unprocessed head (step S29: Yes), the process proceeds to step S24. If there is no unprocessed head (step S29: No), the operation ends.

[0055] The authentication threshold for a person to be authenticated who is in the back area of ​​the authentication area RA may be different from the authentication threshold for a person to be authenticated who is in the front area. For example, the resolution of the facial image of a person to be authenticated may be lower in the back area than in the front area. Therefore, the authentication threshold for a person to be authenticated who is in the back area may be set lower than the authentication threshold for a person to be authenticated who is in the front area. Alternatively, a person to be authenticated who is in the back area may have a higher chance of being authenticated than a person to be authenticated who is in the front area. Therefore, the authentication threshold for a person to be authenticated who is in the back area may be set higher than the authentication threshold for a person to be authenticated who is in the front area. [2-8: Technical Effects of Authentication Device 2]

[0056] If the face authentication of the person being tracked has never been successful before the person to be authenticated leaves the specified area, it is determined that the face authentication of the person to be authenticated has failed, so that it is possible to properly determine that a person whose face authentication has not been successful is a person whose face authentication has failed while ensuring an opportunity for face authentication of the person to be authenticated. [3: Third embodiment]

[0057] A third embodiment of an authentication device, an authentication method, and a recording medium will be described below. The third embodiment of the authentication device, the authentication method, and a recording medium will be described below using an authentication device 3 to which the third embodiment of the authentication device, the authentication method, and a recording medium is applied. [3-1: Configuration of authentication device 3]

[0058] The configuration of the authentication device 3 in the third embodiment will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the configuration of the authentication device 3 in the third embodiment.

[0059] 6, the authentication device 3 in the third embodiment differs from the authentication device 2 in the second embodiment in that the calculation device 21 includes a display control unit 315. Other features of the authentication device 3 may be the same as other features of the authentication device 2. [3-2: Display Operation Performed by Authentication Device 3]

[0060] The flow of the authentication result display operation performed by the authentication device 3 in the third embodiment will be described with reference to Figures 7 to 9. Figure 7 is a flowchart showing the flow of the authentication operation performed by the authentication device 3 in the third embodiment. The operations from "Start" to "End" shown in Figure 7 may be performed for each image IMG.

[0061] As shown in Fig. 7, the authentication operation performed by the authentication device 3 in the third embodiment differs from the authentication operation performed by the authentication device 2 in the second embodiment in that an authentication result display operation (step S30) is performed after an ID correspondence table IDT update operation (step S28). Fig. 8 is a flowchart showing the flow of the authentication result display operation (step S30 shown in Fig. 7) performed by the authentication device 3 in the third embodiment. That is, the authentication device 3 in the third embodiment may update the authentication result display every time one image IMG is processed.

[0062] In the third embodiment, the output device 25 may include a display device (referred to as "display D") capable of displaying an image showing information to be output. FIG. 9 shows an example of a screen showing the authentication result displayed on display D in the third embodiment. Display D in the third embodiment may display a screen for confirmation by a security guard monitoring the authentication area RA or a security guard monitoring at an entrance / exit management control center. Display control unit 315 may control the display on display D.

[0063] As shown in Fig. 8, determination unit 214 determines whether or not face authentication of the person to be authenticated has been successful (step S31). If face authentication of the person to be authenticated has been successful (step S31: Yes), display control unit 315 displays an image in which information indicating success of face authentication of the person to be authenticated is superimposed on the face of the person to be authenticated included in image IMG (step S32). In the example screen shown in Fig. 9, the information indicating success of face authentication of the person to be authenticated is a thin solid rectangle. If face authentication of person to be authenticated C has been successful, display control unit 315 may display an image in which a thin solid rectangle is superimposed on the face of person to be authenticated C included in image IMG.

[0064] Furthermore, the determination unit 214 may accept a person whose face could not be detected. For example, face authentication of person D is successful shortly after entering the authentication area RA, but near the exit of the authentication area RA, the person turns away, and their face is not detected and therefore face authentication is not possible. In this case, the determination unit 214 may determine that face authentication of person D is successful. The display control unit 315 may display an image in which a thin solid-line rectangle is superimposed on the face of person D from the time face authentication of person D is successful while person D is in the authentication area RA.

[0065] The determination unit 214 determines whether the person to be authenticated is present within the predetermined area (step S33). If the person to be authenticated is present within the predetermined area, i.e., if facial authentication of the person to be authenticated has not yet been successful until the tracked person to be authenticated leaves the predetermined area (step S33: Yes), the display control unit 315 displays an image by superimposing information indicating that facial authentication is currently being performed on the face of the person to be authenticated included in the image IMG (step S34). In the example screen shown in FIG. 9, the information indicating that facial authentication is currently being performed on the person to be authenticated is a dashed rectangle. If person A is currently being authenticated, the display control unit 315 may display an image by superimposing a dashed rectangle on the face of person A included in the image IMG.

[0066] The determination in step S33 may be a determination by determination unit 214 of whether or not a predetermined period has elapsed since the person to be authenticated appeared in the predetermined area. Alternatively, the determination in step S33 may be a determination by determination unit 214 of whether or not the length of time that tracking unit 212 has been tracking the person to be authenticated is less than a predetermined period. Determination unit 214 may determine whether authentication has been successful or unsuccessful while the person to be authenticated is in a predetermined time-space.

[0067] If the person to be authenticated is not present within the predetermined area, that is, if the tracked person to be authenticated has left the predetermined area but face authentication of the person to be authenticated has never been successful (step S33: No), the determination unit 214 determines that face authentication of the person to be authenticated has failed. In this case, the display control unit 315 displays an image IMG by superimposing information indicating that face authentication of the person to be authenticated has failed on the face of the person to be authenticated included in the image IMG (step S35).

[0068] In the example screen shown in Fig. 9, information indicating failure of face authentication of the person to be authenticated is a rectangle drawn in a thick solid line. If face authentication of person to be authenticated B fails, display control unit 315 may display an image in which a rectangle drawn in a thick solid line is superimposed on the face of person to be authenticated B included in image IMG. Furthermore, in the example screen shown in Fig. 9, if face authentication of person to be authenticated B fails, display control unit 315 may highlight face image B of person to be authenticated B in a corner of the screen.

[0069] The determination unit 214 may determine that only a person to be authenticated whose face authentication has never been successful while passing through the authentication area RA has failed authentication. That is, the display control unit 315 may display an image with a dashed rectangle superimposed on the face of the person to be authenticated who is determined to have failed while the person to be authenticated is in the authentication area RA, and when the person to be authenticated arrives at the exit of the authentication area RA, may display an image with a thick solid rectangle superimposed on the face of the person to be authenticated. That is, when person to be authenticated A arrives at the exit of the authentication area RA without ever succeeding in face authentication, the display control unit 315 may display an image with a thick solid rectangle superimposed on the face of person to be authenticated A included in the image IMG.

[0070] The display control unit 315 may display differently depending on whether facial authentication is unsuccessful because the face of the person to be authenticated included in the image IMG is inappropriate for facial authentication or whether facial authentication is unsuccessful even though the face of the person to be authenticated included in the image IMG is appropriate for facial authentication. That is, after step S33: No, a step in which the determination unit 214 determines whether facial authentication has been performed while tracking the person to be authenticated, and a step in which the display control unit 315 performs different displays may be performed. For example, the determination unit 214 may refer to the ID correspondence table IDT and determine whether facial authentication has been performed while tracking the person to be authenticated depending on whether the previous matching time has been registered.

[0071] Facial recognition may not be successful because the face of the person to be authenticated contained in the image IMG is inappropriate for facial recognition, including cases where the person to be authenticated is constantly facing backward, downward, sideways, etc., and therefore a face appropriate for recognition cannot be detected; the person is dressed unusually, for example, with a beard or wearing glasses; there is a problem with the camera CAM; or the image quality is poor due to blurring or occlusion. A case where facial recognition is not successful even though the face of the person to be authenticated contained in the image IMG is appropriate for facial recognition may be when the person to be authenticated is determined to be an unregistered person. Specifically, the similarity between the feature values ​​of the facial image of the person to be authenticated and the feature values ​​of the facial image of any registered person does not exceed a threshold.

[0072] The determination unit 214 may perform real-time personal authentication on all of the authentication-subjects passing through the authentication area RA, and identify all authentication-subjects whose faces cannot be detected and unregistered authentication-subjects. Then, the display control unit 315 may notify the authentication-subjects through a display on the display D before they leave the vicinity of the exit of the authentication area RA. [3-3: Technical Effects of Authentication Device 3]

[0073] The administrator can check the facial authentication status of each person being authenticated in real time by viewing the display D. For example, a person being authenticated who is near the exit of the authentication area RA but has an image indicating "authentication in progress" superimposed on them may be a person who has "failed authentication," and the administrator can focus on that person. In addition, the display control unit 315 displays different information depending on the cause of "authentication failure," allowing the administrator to know the reason for the facial authentication failure. [4: Fourth embodiment]

[0074] A fourth embodiment of an authentication device, an authentication method, and a recording medium will be described below. The fourth embodiment of the authentication device, the authentication method, and the recording medium will be described below using an authentication device 4 to which the fourth embodiment of the authentication device, the authentication method, and the recording medium is applied. [4-1: Configuration of authentication device 4]

[0075] The configuration of the authentication device 4 in the fourth embodiment will be described with reference to Fig. 10. Fig. 10 is a block diagram showing the configuration of the authentication device 4 in the fourth embodiment.

[0076] 10, the authentication device 4 in the fourth embodiment differs from the authentication device 2 in the second embodiment and the authentication device 3 in the third embodiment in that the calculation device 21 includes a storage control unit 416 and a correspondence unit 417, and the storage device 22 stores a ReID feature database RIC. Other features of the authentication device 4 may be the same as other features of the authentication device 2 and the authentication device 3. However, the ReID feature database RIC does not have to be stored in the storage device 22. [4-2:ReID operation]

[0077] The ReID operation may be an operation of linking and managing the tracking IDs of the corresponding authenticated person tracked in different cases. The different cases may be at different times, different places, or when using images IMG acquired from different cameras CAM.

[0078] The tracking unit 212 may have difficulty in tracking if it cannot detect the head for a predetermined time, for example, one second. That is, if 30 frames are captured per second, the tracking unit 212 may be unable to detect the head for 30 frames. The ReID operation may be a process that is performed when tracking of a person to be authenticated is interrupted, when a new person to be authenticated appears, or the like. The ReID operation may also be a response operation when a tracking error occurs and different tracking IDs are assigned to the same person. The ReID operation may also be an operation that is effective in identifying the same person.

[0079] The ReID feature database RIC may store the tracking ID of the person to be authenticated and the ReID feature of the person to be authenticated in association with each other. The ReID feature may be a feature for identifying the person to be authenticated. The ReID feature may be a feature extracted from an image of a rectangular area including the person to be authenticated. The ReID feature may be a feature extracted from a whole-body image of the person to be authenticated. The ReID feature may be a feature extracted from an upper-body image of the person to be authenticated. The upper-body image of the person to be authenticated may include at least the clothing worn by the person to be authenticated. In other words, in the ReID operation, it may be determined whether the people to be authenticated are the same person based on whether the clothing worn is the same. The ReID feature may be a feature extracted from a head image of the person to be authenticated.

[0080] The storage control unit 416 stores information about a tracked authenticated person who is an authenticated person who has been tracked by the tracking unit 212 in the ReID feature database RIC. When the head of a non-tracked authenticated person who is an authenticated person who has been newly tracked by the tracking unit 212 is included in the image IMG and information about a tracked authenticated person who is the same person as the non-tracked authenticated person is stored in the ReID feature database RIC, the association unit 417 associates the non-tracked authenticated person with the corresponding tracked authenticated person.

[0081] The matching operation using the ReID feature may be able to determine whether the person to be authenticated is the same person with higher accuracy than the tracking operation by the tracking unit 212. Furthermore, the matching operation using the ReID feature may have a lighter operational load than the matching operation between facial features. That is, the ReID operation may be able to determine whether the person to be authenticated matches with higher accuracy than tracking by the tracking unit 212, and may have a smaller operational load than the matching operation between facial features. In the fourth embodiment, even if tracking by the tracking unit 212 fails, tracking may be able to continue without performing face authentication. [4-3: Target of ReID operation]

[0082] The tracking ID of a tracked subject who has been tracked is called a "corresponding tracking ID." The association unit 417 may perform a ReID operation on a subject whose tracking ID is not linked to a corresponding tracking ID and whose face has not been successfully authenticated. The association unit 417 may perform a ReID operation when a ReID feature extraction area RICA of the subject is included in the image IMG. The ReID feature extraction area RICA of the subject may be a body area of ​​the subject, an upper body area of ​​the subject, or an area including part of clothing worn by the subject.

[0083] If the head of the person to be authenticated is included in the image IMG but the tracking unit 212 is unable to track it, and if the ReID feature extraction area RICA of the person to be authenticated is included in the image IMG, the matching unit 417 may determine that the people are the same person based on the physical features included in the ReID feature extraction area RICA of the person to be authenticated.

[0084] Note that the ReID operation may not be performed if there is no doubt that the person is the same. If there is no doubt that the person is the same, it is sufficient to perform only the head tracking operation. [4-4: ID correspondence table IDT]

[0085] For example, the ID correspondence table IDT of the fourth embodiment may have the structure shown in Table 2 below. [Table 2: ID correspondence table IDT]

[0086] [Table 2] That is, compared to the ID correspondence table IDT of the second embodiment, the ID correspondence table IDT of the fourth embodiment may further include a column of a corresponding tracking ID and a column of a tracking start time. [4-5: ReID operation for each image IMG]

[0087] The flow of the ReID operation performed by the authentication device 4 in the fourth embodiment will be described with reference to Fig. 11. Fig. 11 is a flowchart showing the flow of the ReID operation performed by the authentication device 4 in the fourth embodiment.

[0088] 11, the determination unit 214 determines whether information about the corresponding person to be authenticated is stored in the ReID feature database RIC (step S41). The determination unit 214 may determine at least one of whether the ReID feature of the corresponding person to be authenticated is stored in the ReID feature database RIC and whether the tracking ID of the corresponding person to be authenticated is associated with the corresponding tracking ID in the ID correspondence table IDT.

[0089] That is, in step S41, if the ReID feature of the relevant authenticated person is not stored in the ReID feature database RIC, this may be the case where the relevant authenticated person is a non-tracked authenticated person who is newly tracked by the tracking unit 212. If the head of the newly tracked authenticated person is included in the image IMG, the tracking unit 212 may determine that the relevant authenticated person is a newly tracked non-tracked authenticated person.

[0090] If the ReID feature of the corresponding person to be authenticated is not stored in the ReID feature database RIC (step S41: No), the storage control unit 416 determines whether the ReID feature extraction area RICA of the corresponding person to be authenticated in the image IMG is of a predetermined quality or higher (step S42). The ReID feature extraction area RICA in the image IMG being of a predetermined quality or higher may be of a quality that allows ReID features to be extracted from the ReID feature extraction area RICA in the image IMG. If the ReID feature extraction area RICA in the image IMG is of a predetermined quality or higher (step S42: Yes), the storage control unit 416 extracts ReID features from the ReID feature extraction area RICA in the image IMG (step S43).

[0091] The association unit 417 compares the ReID feature of the relevant authentication subject with the ReID feature of the tracked authentication subject already stored in the ReID feature database RIC (step S44). The association unit 417 determines whether the comparison between the ReID feature of the relevant authentication subject and the ReID feature of the tracked authentication subject already stored in the ReID feature database RIC is successful (step S45). If the comparison is successful (step S45: Yes), that is, if the ReID feature of the tracked authentication subject who is the same person as the relevant authentication subject is stored in the ReID feature database RIC, the association unit 417 associates the relevant authentication subject with the tracked authentication subject (step S47). For example, in the example shown in Table 2, the tracking ID of the relevant authentication subject is "6", and it is determined that the person is the same as the authentication subject with tracking ID "1". In this case, the association unit 417 may add "1" to the corresponding tracking ID field for the person to be authenticated whose tracking ID is "6", and may also add the same authentication ID "0105" as the person to be authenticated whose tracking ID is "1" and the previous matching time "0:02:30" to the authentication ID field. Also, if the matching is successful (step S45: Yes), the storage control unit 416 may store the ReID feature of the person to be authenticated in the ReID feature database RIC.

[0092] If the matching fails (step S45: No), determination unit 214 determines whether a predetermined period of time has passed since the tracking of the corresponding authentication-subject started (step S46). Determination unit 214 may determine whether the predetermined period of time has passed by referring to the tracking start time of the corresponding authentication-subject in the ID correspondence table IDT.

[0093] If the predetermined time has passed (step S46: Yes), the process proceeds to step S47. For example, in the example shown in Table 2, it is assumed that the determination unit 214 determines that the predetermined time has passed for the person to be authenticated whose tracking ID is "5". In this case, the determination unit 214 may add "N / A" to the corresponding tracking ID column for the person to be authenticated whose tracking ID is "5". Furthermore, if the predetermined time has passed (step S46: Yes), the storage control unit 416 may store the ReID feature of the person to be authenticated in the ReID feature database RIC.

[0094] In step S41, if the ReID feature of the corresponding authentication subject is stored in the ReID feature database RIC, the ReID operation is terminated. In step S41, if the ReID feature of the corresponding authentication subject is stored in the ReID feature database RIC, this may be the case where the association unit 417 has already matched the ReID feature of the corresponding authentication subject with the ReID feature of the tracked authentication subject that is already stored in the ReID feature database RIC. Alternatively, this may be the case where the association unit 417 has not successfully matched the ReID feature of the corresponding authentication subject with the ReID feature of the tracked authentication subject that is already stored in the ReID feature database RIC, even though a predetermined period has elapsed since tracking of the corresponding authentication subject began.

[0095] In step S42, if the quality of the ReID feature extraction area RICA in the image IMG is lower than a predetermined level, the ReID operation is terminated. In step S46, if a predetermined period of time has not elapsed since the start of tracking of the person to be authenticated, the ReID operation is terminated. In this case, the ReID operation may also be performed in the next frame. In the fourth embodiment, the ReID feature is compared for a newly tracked person to be authenticated, which reduces the processing load compared to when the ReID feature is compared for each frame. On the other hand, the tracking accuracy may be improved by comparing the ReID feature for each frame. [4-6: Authentication operation performed by authentication device 4]

[0096] The flow of authentication operations performed by the authentication device 4 in the fourth embodiment will be described with reference to Fig. 12. Fig. 12 is a flowchart showing the flow of authentication operations performed by the authentication device 4 in the fourth embodiment. The operations from "START" to "END" shown in Fig. 12 may be operations for each frame.

[0097] As shown in Fig. 12, the authentication operation performed by authentication device 4 differs from the authentication operation performed by authentication device 2 shown in Fig. 5 in that a ReID operation (step S40) is performed at the beginning of processing for each tracked person to be authenticated. The ReID operation (step S40) is as described with reference to Fig. 11.

[0098] As shown in the example of Table 2, when a person to be authenticated whose tracking ID is "6" is associated with a person to be authenticated whose tracking ID is "1", the association unit 417 may set the time of the last successful authentication for tracking ID "6" to "00:02:30". That is, in the determination of step S25, the determination unit 214 may use the time of the person to be authenticated whose tracking ID is associated.

[0099] Also, for example, when the person to be authenticated leaves the authentication area RA, the association unit 417 may integrate multiple tracking IDs using the ReID feature, and the judgment unit 214 may make a final judgment as to whether or not face authentication was successful.

[0100] In the fourth embodiment, similarly to the third embodiment, the authentication result display operation (step S30) may be executed after the ID correspondence table IDT update operation (step S28). [4-7: Technical Effects of Authentication Device 4]

[0101] Even if tracking of the person to be authenticated fails, the same person is associated and the success or failure of face authentication is determined, so that an opportunity for face authentication of the person to be authenticated can be secured. [5: Fifth embodiment]

[0102] A fifth embodiment of an authentication device, an authentication method, and a recording medium will be described below. The fifth embodiment of an authentication device, an authentication method, and a recording medium will be described below using an authentication device 5 to which the fifth embodiment of the authentication device, the authentication method, and the recording medium is applied. [5-1: Configuration of authentication device 5]

[0103] The configuration of the authentication device 5 in the fifth embodiment will be described with reference to Fig. 13. Fig. 13 is a block diagram showing the configuration of the authentication device 5 in the fifth embodiment.

[0104] 13, the authentication device 5 in the fifth embodiment differs from the authentication device 2 in the second embodiment to the authentication device 4 in the fourth embodiment in that the calculation device 21 includes a change unit 518. Other features of the authentication device 5 may be the same as other features of the authentication devices 2 to 4.

[0105] The change unit 518 changes at least one of the size of the predetermined area and the length of time that the tracking unit 212 tracks the person to be authenticated, depending on the number of people to be authenticated who exist in the predetermined area. For example, when there are a large number of people to be authenticated who exist in the predetermined area, people often overlap with each other, which can make face detection difficult. Therefore, the chances of face detection can be increased by changing at least one of the size of the predetermined area and the length of time that the tracking unit 212 tracks the person to be authenticated. [5-2: Application example of authentication device 5]

[0106] An application example of the authentication device 5 in the fifth embodiment will be described with reference to Fig. 14. Fig. 14 is a conceptual diagram showing an example of a scene in which the authentication device 5 in the fifth embodiment is applied. The authentication device 5 in the fifth embodiment may also be applied to the authentication operation of a person to be authenticated, for example, to determine whether or not the person to be authenticated is permitted to enter a restricted area LA set up in a theme park, airport, train station, office, event venue, or the like.

[0107] The change unit 518 changes at least one of the size of the predetermined area and the length of time that the tracking unit 212 tracks the person to be authenticated, depending on the number of people to be authenticated who exist in the predetermined area. For example, normally, the authentication device 6 authenticates people who exist in the authentication area RA1 as people to be authenticated, but if there are many people to be authenticated who exist in the authentication area RA1, the authentication device 6 may authenticate people who exist in both the authentication area RA1 and the authentication area RA2 as people to be authenticated.

[0108] Each of the authentication areas RA1 and RA2 may be, for example, an area of ​​5 meters by 5 meters. That is, for example, while the length of the direction of travel of the person to be authenticated is normally 5 meters, if there are a large number of people to be authenticated in the authentication area RA1, the length of the direction of travel of the person to be authenticated may be 10 meters, which is the length of the authentication area RA1 plus the length of the authentication area RA2.

[0109] The authentication device 5 acquires an image IMG from a camera CAM. The camera CAM may be installed near the exit of the authentication area RA1. For example, the camera CAM normally captures an image of an area including the authentication area RA1, but if there are a large number of people to be authenticated in the authentication area RA1, the camera CAM may capture an image of an area including the authentication areas RA1 and RA2. The camera CAM may be an imaging device similar to the camera CAM used in the second to fourth embodiments. [5-3: Technical Effects of Authentication Device 5]

[0110] For example, when there are many people to be authenticated in a predetermined area, overlapping of people often occurs, making face detection difficult, but by changing at least one of the size of the predetermined area and the length of time that the tracking unit 212 tracks the people to be authenticated, it is possible to ensure opportunities for face authentication of the people to be authenticated. This increases the chances that the face authentication of the people to be authenticated will ultimately be successful. [6: Sixth embodiment]

[0111] A sixth embodiment of an authentication device, an authentication method, and a recording medium will be described below. In the following, the sixth embodiment of an authentication device, an authentication method, and a recording medium will be described using an authentication device 6 to which the sixth embodiment of the authentication device, the authentication method, and the recording medium is applied.

[0112] The authentication device 6 in the sixth embodiment differs from the authentication devices 2 in the second embodiment to 5 in the fifth embodiment in how to handle a failed face authentication. Other features of the authentication device 6 may be the same as other features of the authentication devices 2 to 5.

[0113] In the sixth embodiment, the acquisition unit 211 acquires an image IMG from each of a plurality of imaging devices capable of generating an image IMG by capturing an image of a person to be authenticated. Also, in the sixth embodiment, if face authentication using an image IMG captured by a camera CAM1 serving as a first imaging device among the plurality of imaging devices is unsuccessful, the face authentication unit 213 performs face authentication using an image IMG captured by a camera CAM2 serving as a second imaging device different from camera CAM1 among the plurality of imaging devices. [6-1-1: First application example of authentication device 6]

[0114] A first application example of the authentication device 6 in the sixth embodiment will be described with reference to Fig. 15. Fig. 15 is a conceptual diagram showing a first example of a situation in which the authentication device 6 in the sixth embodiment is applied. The authentication device 6 in the sixth embodiment may also be applied to the authentication operation of a person to be authenticated, for example, to determine whether or not the person to be authenticated is permitted to enter a restricted area LA set up in a theme park, airport, train station, office, event venue, or the like.

[0115] In the scene illustrated in Fig. 15, the authentication device 6 authenticates people present in the authentication area RA1 and the authentication area RA2 as persons to be authenticated. Each of the authentication area RA1 and the authentication area RA2 may be, for example, an area of ​​5 meters by 5 meters. That is, in the scene illustrated in Fig. 16, the length of the direction of travel of the person to be authenticated may be 10 meters. In a first application example of the authentication device 6 in the sixth embodiment, the length of the direction of travel of the person to be authenticated may be longer than in the case illustrated in Fig. 3.

[0116] In the scene illustrated in Fig. 15, it is assumed that the person to be authenticated moves from authentication area RA1 to authentication area RA2. In the scene illustrated in Fig. 15, each of person to be authenticated A, person to be authenticated B, and person to be authenticated C moves from authentication area RA1 to authentication area RA2 and moves toward restricted area LA.

[0117] In the scene illustrated in FIG. 15, the authentication device 6 acquires images IMG from each of cameras CAM1 and CAM2. Camera CAM1 may be installed near the exit of authentication area RA1. Camera CAM1 may capture an image of an area including authentication area RA1. Camera CAM2 may be installed near the exit of authentication area RA2. Camera CAM2 may capture an image of an area including authentication area RA2. Each of cameras CAM1 and CAM2 may be an imaging device similar to the camera CAM used in the second to fifth embodiments.

[0118] That is, each of the authenticatee A, the authenticatee B, and the authenticatee C may be imaged by camera CAM1 in authentication area RA1, and may be imaged by camera CAM2 in authentication area RA2. For each of the authenticatee A, the authenticatee B, and the authenticatee C, if face authentication using the image IMG captured by camera CAM1 is unsuccessful, the face authentication unit 213 may perform face authentication using the image IMG captured by camera CAM2. Furthermore, even if face authentication using the image IMG captured by camera CAM2 is unsuccessful, the determination unit 214 may determine that face authentication is successful if face authentication using the image IMG captured by camera CAM1 is successful. [6-1-2: ID correspondence table IDT for the first application example]

[0119] For example, the ID correspondence table IDT of the first application example of the sixth embodiment may be as shown in Table 3 below. [Table 3: ID correspondence table IDT]

[0120] [Table 3]

[0121] The tracking ID "C1" is the tracking ID of the person to be authenticated C whose image is captured by camera CAM1. The person to be authenticated C has successfully undergone facial authentication using the image IMG captured by camera CAM1. The tracking ID "C2" is the tracking ID of the person to be authenticated C whose image is captured by camera CAM2. The tracking ID "C2" may be associated with the tracking ID "C1" by the ReID operation described above. In other words, the ReID operation may be performed between different cameras CAM1 and CAM2. Even if the facial authentication using the image IMG captured by camera CAM2 is not successful, the determination unit 214 may determine that the facial authentication of the person to be authenticated C has been successful because the facial authentication using the image IMG captured by camera CAM1 was successful.

[0122] The tracking ID "B1" is the tracking ID of the person to be authenticated B, whose image is captured by the camera CAM1. The person to be authenticated B has not been successfully authenticated using the image IMG captured by the camera CAM1. The tracking ID "B2" is the tracking ID of the person to be authenticated B, whose image is captured by the camera CAM2. The person to be authenticated B has been successfully authenticated using the image IMG captured by the camera CAM2. The tracking ID "B2" may be associated with the tracking ID "B1" by the ReID operation described above. If the face authentication using the image IMG captured by the camera CAM1 is not successful, the face authentication unit 213 may perform face authentication using the image IMG captured by the camera CAM2. The determination unit 214 may determine that the face authentication has been successful if the face authentication using the image IMG captured by the camera CAM2 is successful, even if the face authentication using the image IMG captured by the camera CAM1 is not successful. In the case shown in Table 3 above, face authentication using image IMG captured by camera CAM2 is successful, so determination unit 214 may determine that face authentication of person B is successful.

[0123] The tracking ID "A1" is the tracking ID of the person to be authenticated A captured by the camera CAM1. The person to be authenticated A has not been successfully authenticated using the image IMG captured by the camera CAM1. The tracking ID "A2" is the tracking ID of the person to be authenticated A captured by the camera CAM2. The tracking ID "A2" may be associated with the tracking ID "A1" by the ReID operation described above. If the face authentication using the image IMG captured by the camera CAM1 is not successful, the face authentication unit 213 may perform face authentication using the image IMG captured by the camera CAM2. The determination unit 214 may determine that the face authentication has been successful if the face authentication using the image IMG captured by the camera CAM2 is successful, even if the face authentication using the image IMG captured by the camera CAM1 is not successful. In the case shown in Table 3 above, if facial authentication using the image IMG captured by camera CAM2 has not yet been successful, but facial authentication using the image IMG captured by camera CAM2 is successful before the person to be authenticated A leaves the authentication area RA2, the judgment unit 214 may judge that facial authentication of the person to be authenticated A has been successful.

[0124] That is, if a camera CAM is provided in an area (e.g., authentication area RA2) in front (e.g., 5 to 10 meters in front) of the normal authentication area RA (e.g., authentication area RA1), and if face authentication is successful there, the determination unit 214 may determine that face authentication is successful even if face authentication is not successful in the normal authentication area RA. Alternatively, if face authentication is not successful in the normal authentication area RA (e.g., authentication area RA2), an image IMG may be acquired from a camera CAM (e.g., camera CAM1) provided so as to be able to capture an area behind the normal authentication area (e.g., authentication area RA1), and face authentication may be continued while tracking the person to be authenticated. That is, in the sixth embodiment, the success rate of face authentication can be increased by expanding both the time and space in the front and back. In this way, by increasing the number of camera CAMs, the authentication area can be expanded compared to normal.

[0125] In the fourth embodiment, correspondence is established between tracking images taken at different times by the same camera CAM, but in the sixth embodiment, correspondence is established between tracking images taken at different times by different camera CAMs. Images IMG taken by multiple camera CAMs can share the identity of the person being tracked. [6-2-1: Second Application Example of Authentication Device 6]

[0126] A second application example of the authentication device 6 in the sixth embodiment will be described with reference to Fig. 16. Fig. 16 is a conceptual diagram showing a second example of a situation in which the authentication device 6 in the sixth embodiment is applied.

[0127] In the scene illustrated in Fig. 16, the authentication device 6 authenticates people present in the authentication areas RA1 and RA2 as subjects to be authenticated. Each of the authentication areas RA1 and RA2 may be, for example, an area of ​​5 meters by 5 meters. That is, in the scene illustrated in Fig. 16, the width of the authentication area RA may be 10 meters. In a second application example of the authentication device 6 in the sixth embodiment, the width of the authentication area RA may be wider than that shown in Fig. 3.

[0128] In the scene illustrated in Fig. 16, the authentication device 6 acquires images IMG from each of the cameras CAM1 and CAM2. The camera CAM1 may be installed near the exit of the authentication area RA1. The camera CAM1 may capture an image of an area including the authentication area RA1. The camera CAM2 may be installed near the exit of the authentication area RA2. The camera CAM2 may capture an image of an area including the authentication area RA2.

[0129] In the scene illustrated in Fig. 16, the person to be authenticated may not only move within either authentication area RA1 or authentication area RA2, but may also move from authentication area RA1 to authentication area RA2, or from authentication area RA2 to authentication area RA1. In the scene illustrated in Fig. 16, person to be authenticated A is moving toward restriction area LA within authentication area RA1. Person to be authenticated B is moving toward restriction area LA while moving from authentication area RA1 to authentication area RA2. Person to be authenticated C is moving toward restriction area LA while moving from authentication area RA2 to authentication area RA1.

[0130] That is, the person to be authenticated B may be imaged by camera CAM1 in authentication area RA1, and by camera CAM2 in authentication area RA2. For the person to be authenticated B, if face authentication using the image IMG captured by camera CAM1 is unsuccessful, the face authentication unit 213 may perform face authentication using the image IMG captured by camera CAM2. Furthermore, even if face authentication using the image IMG captured by camera CAM2 is unsuccessful, the determination unit 214 may determine that face authentication is successful if face authentication using the image IMG captured by camera CAM1 is successful.

[0131] Furthermore, the person to be authenticated C may be imaged by camera CAM2 in authentication area RA2, and may be imaged by camera CAM1 in authentication area RA1. For the person to be authenticated C, the face authentication unit 213 may perform face authentication using the image IMG captured by camera CAM1 when face authentication using the image IMG captured by camera CAM2 is unsuccessful. Furthermore, the determination unit 214 may determine that face authentication is successful when face authentication using the image IMG captured by camera CAM2 is successful, even when face authentication using the image IMG captured by camera CAM1 is unsuccessful. [6-2-2: ID correspondence table IDT for the second application example]

[0132] For example, the ID correspondence table IDT of the second application example of the sixth embodiment may be as shown in Table 4 below. [Table 4: ID correspondence table IDT]

[0133] [Table 4]

[0134] Tracking ID "B1" is the tracking ID of person B who is imaged by camera CAM1. Person B has been successfully authenticated using the image IMG captured by camera CAM1. Tracking ID "B2" is the tracking ID of person B who is imaged by camera CAM2. Tracking ID "B2" may be associated with tracking ID "B1" by the ReID operation described above. Even if face authentication using the image IMG captured by camera CAM2 is not successful, the determination unit 214 may determine that face authentication of person B has been successful because face authentication using the image IMG captured by camera CAM1 was successful.

[0135] The tracking ID "C2" is the tracking ID of the person to be authenticated C whose image is captured by the camera CAM2. The person to be authenticated C has not been successfully authenticated using the image IMG captured by the camera CAM2. The tracking ID "C1" is the tracking ID of the person to be authenticated C whose image is captured by the camera CAM1. The tracking ID "C1" may be associated with the tracking ID "C2" by the ReID operation described above. The face authentication unit 213 may perform face authentication using the image IMG captured by the camera CAM1 when face authentication using the image IMG captured by the camera CAM2 is unsuccessful. The determination unit 214 may determine that face authentication is successful when face authentication using the image IMG captured by the camera CAM1 is successful, even when face authentication using the image IMG captured by the camera CAM2 is unsuccessful. In the case shown in Table 4 above, since face authentication using the image IMG captured by the camera CAM1 is successful, the determination unit 214 may determine that face authentication of the person to be authenticated C has been successful.

[0136] The tracking ID "A1" is the tracking ID of the person to be authenticated A captured by the camera CAM1. When the person to be authenticated A moves from the authentication area RA1 to the authentication area RA2, a new tracking ID may be assigned to the person to be authenticated A, and the new tracking ID may be associated with the tracking ID "A1" by the ReID operation described above. [6-2: Technical Effects of Authentication Device 6]

[0137] In the sixth embodiment, by performing a ReID operation in addition to a tracking operation, it is possible to continue tracking the same person to be authenticated. The ReID operation allows the authentication device 6 to perform a tracking operation using images IMG acquired by different cameras CAM. This, for example, eliminates the need to issue warnings multiple times. Examples of warnings include warnings for not wearing a mask, warnings for people with high body temperatures, etc. [7: Seventh embodiment]

[0138] A seventh embodiment of an authentication device, an authentication method, and a recording medium will be described below. In the following, the seventh embodiment of an authentication device, an authentication method, and a recording medium will be described using an authentication device 7 to which the seventh embodiment of the authentication device, the authentication method, and the recording medium is applied.

[0139] The authentication device 7 in the seventh embodiment is different from the authentication devices 2 in the second embodiment to 6 in the sixth embodiment in the area of ​​the image IMG used for face authentication. Other features of the authentication device 7 may be the same as other features of the authentication devices 2 to 6.

[0140] When the person to be authenticated in the image IMG is no longer included in the image IMG, the entire head of the person to be authenticated is gradually included in the image IMG, but the portion of the head that is not included in the image IMG gradually increases, and eventually the entire head is no longer included in the image IMG. For example, if only a portion of the person to be authenticated is included in the image IMG, tracking by the tracking unit 212 may fail. For example, the tracking unit 212 may not determine that a portion of the person to be authenticated is the same person's head, but rather that it is a new head, and associate a new tracking ID with the portion of the person to be authenticated. In this case, even if the association unit 417 performs a ReID operation, it may not be associated with the tracking ID of the person to be authenticated. This may result in a tracking ID that cannot be associated with any tracking ID.

[0141] Therefore, in the seventh embodiment, the tracking unit 212 tracks the person to be authenticated based on the head of the person to be authenticated within a predetermined area of ​​the image IMG. The tracking unit 212 may track the person to be authenticated if the head of the person to be authenticated is included within the predetermined area of ​​the image IMG.

[0142] Fig. 17 is a conceptual diagram showing an example of the tracking operation of the authentication device in the seventh embodiment. For example, as shown in Fig. 17, among the heads included in the image IMG, a head included in an area hidden by a mask M may be excluded from the tracking target. Alternatively, when the tracking unit 212 determines that the head included in an area hidden by the mask M is a new head, it is not necessary to associate a new tracking ID with the new head. Alternatively, even when the tracking unit 212 has succeeded in tracking the head included in an area hidden by the mask M, the authentication device 7 may omit the operations of steps S23 to S28.

[0143] Since the tracking unit 212 tracks the person to be authenticated based on the head of the person to be authenticated within a predetermined area of ​​the image IMG, it is possible to prevent failure in tracking the person to be authenticated.

[0144] Although the above embodiment has been described using face authentication as an example, other authentication methods such as iris authentication may be used to perform authentication while tracking. Also, multimodal authentication using both face authentication and iris authentication may be used to perform authentication while tracking.

[0145] In addition, in each of the above-described embodiments, an authenticated person not wearing a mask may be detected from the image IMG, and an announcement to the authenticated person to wear a mask may be played, or an administrator may speak to the authenticated person to wear a mask. In particular, when performing the ReID operation and continuously tracking a specific authenticated person, there is no need to issue multiple warnings. Furthermore, face recognition and body surface temperature measurement may be performed simultaneously without contact, and an authenticated person whose body surface temperature exceeds a set value may be identified, and an administrator may speak to the authenticated person and urge them to take their temperature. [8: Note]

[0146] The following additional notes are provided regarding the above-described embodiment. [Appendix 1] an acquisition means for acquiring an image including a predetermined area through which a person to be authenticated passes; a tracking means for tracking the person to be authenticated based on the head of the person to be authenticated in the image; a face authentication means for performing face authentication of the person to be authenticated when the face of the person to be authenticated is included in the image while the tracking means is tracking the person to be authenticated; An authentication device comprising: [Appendix 2] The tracking device further includes a determination unit that determines that face authentication of the person to be authenticated has failed if the face authentication of the person to be authenticated has not been successful even once until the person to be authenticated being tracked by the tracking unit leaves the predetermined area. 10. The authentication device of claim 1. [Appendix 3] If the face authentication of the person to be authenticated is successful, information indicating the success of the face authentication of the person to be authenticated is superimposed on the face of the person to be authenticated included in the image; If the facial authentication of the person to be authenticated has never been successful until the person to be tracked moves out of the predetermined area, information indicating that the person to be authenticated is currently undergoing facial authentication is superimposed on the face of the person to be authenticated included in the image; a display means for displaying an image with information indicating a failure in face authentication of the person to be authenticated superimposed on the face of the person to be authenticated included in the image when the determination means determines that face authentication of the person to be authenticated has failed; 3. The authentication device according to claim 1 or 2, further comprising: [Appendix 4] The display means displays differently depending on whether face authentication is unsuccessful because the face of the person to be authenticated included in the image is inappropriate for face authentication or whether face authentication is unsuccessful even though the face of the person to be authenticated included in the image is appropriate for face authentication. 2. The authentication device according to claim 1 . [Appendix 5] a storage means for storing information about a tracked authenticated person who has been tracked by the tracking means; a correspondence association means for associating the non-tracked authenticated person with the corresponding tracked authenticated person when the head of the non-tracked authenticated person who is the authenticated person newly tracked by the tracking means is included in the image and information about a tracked authenticated person who is the same person as the non-tracked authenticated person is stored in the storage means; 5. The authentication device according to any one of claims 1 to 4, further comprising: [Appendix 6] a change means for changing at least one of the size of the predetermined area and the length of time during which the tracking means tracks the person to be authenticated, in accordance with the number of people to be authenticated who are present in the predetermined area; 6. The authentication device according to any one of Supplementary notes 1 to 5, further comprising: [Appendix 7] the acquisition means acquires the image from each of a plurality of imaging devices capable of generating the image by capturing an image of the person to be authenticated; When the face authentication using the image captured by a first image capturing device among the plurality of image capturing devices is unsuccessful, the face authentication means performs the face authentication using the image captured by a second image capturing device different from the first image capturing device among the plurality of image capturing devices. 10. An authentication device according to any one of claims 1 to 6. [Appendix 8] The tracking means tracks the person to be authenticated based on the head of the person to be authenticated within a predetermined area of ​​the image. 8. An authentication device according to any one of appendices 1 to 7. [Appendix 9] Acquire an image including a predetermined area through which the person to be authenticated passes; tracking the subject based on the subject's head in the image; If the face of the person to be authenticated is included in the image while the person to be authenticated is being tracked, face authentication of the person to be authenticated is performed. Authentication method. [Appendix 10] On the computer, Acquire an image including a predetermined area through which the person to be authenticated passes; tracking the subject based on the subject's head in the image; If the face of the person to be authenticated is included in the image while the person to be authenticated is being tracked, face authentication of the person to be authenticated is performed. A recording medium on which a computer program for executing an authentication method is recorded.

[0147] At least some of the constituent elements of each of the above-described embodiments can be appropriately combined with at least some of the other constituent elements of each of the above-described embodiments. Some of the constituent elements of each of the above-described embodiments may not be used. Furthermore, to the extent permitted by law, the disclosures of all documents (e.g., published patent applications) cited in this disclosure are incorporated by reference as part of the description of this disclosure.

[0148] This disclosure may be modified as appropriate within the scope of the claims and the technical idea that can be read from the entire specification. Authentication devices, authentication methods, and recording media that involve such modifications are also included in the technical idea of ​​this disclosure. [Explanation of symbols]

[0149] 1,2,3,4,5,6,7 Authentication device 11,211 Acquisition Department 12,212 Tracking Department 13,213 Face Recognition Unit 214 Judgment section 315 Display control unit 416 Memory control unit 417 Correspondence Section 518 Changes IDT ID Correspondence Table FC Facial Feature Database RA Certification Area LA Restricted Area RIC ReID feature database RICA ReID feature extraction area

Claims

1. an acquisition means for acquiring a first image that is an image including a predetermined area through which a person to be authenticated passes; a setting means for setting a tracking area for each of the persons to be authenticated within the image; a tracking means for tracking the person to be authenticated based on the tracking area; a face authentication means for performing face authentication using an image of the person to be authenticated acquired while the tracking means is tracking the person to be authenticated; a display control means for controlling the display of the first image and a second image in which the authentication result of the person to be authenticated is superimposed on an image of the person to be authenticated acquired while tracking the person to be authenticated; and An information processing device comprising:

2. when the person to be authenticated is present in the first image, the display control means superimposes information related to authentication on the person to be authenticated in the first image.

2. The information processing apparatus according to claim 1, wherein:

3. the display control means superimposes and displays information related to the authentication until the person to be authenticated leaves the predetermined area.

3. The information processing apparatus according to claim 2, wherein:

4. The information related to the authentication includes first information indicating that the person to be authenticated has been successfully authenticated, and second information indicating that the person to be authenticated is currently being authenticated.

4. The information processing apparatus according to claim 3,

5. The information regarding the authentication includes third information indicating that the person to be authenticated was not successfully authenticated.

5. The information processing apparatus according to claim 4,

6. the first information is displayed as a solid-line rectangle superimposed on the person to be authenticated, and the second information is displayed as a dashed-line rectangle superimposed on the person to be authenticated; 6. The information processing apparatus according to claim 5,

7. the third information is displayed on the subject of authentication as a rectangle with a thicker solid line than the solid line representing the first information; 7. The information processing apparatus according to claim 6,

8. If the person to be authenticated leaves the predetermined area while being authenticated, the face authentication means determines that authentication of the person to be authenticated has failed, and displays the first image and the second image together.

4. The information processing device according to claim 1, wherein the information processing device is a computer.

9. the display control means displays differently depending on whether face authentication is unsuccessful because the face of the person to be authenticated is inappropriate for face authentication or whether face authentication is unsuccessful despite the face of the person to be authenticated being appropriate for face authentication.

4. The information processing device according to claim 1, wherein the information processing device is a computer.

10. acquiring a first image that is an image including a predetermined area through which a person to be authenticated passes; setting a tracking area for each of the persons to be authenticated within the image; Tracking the authenticatee based on the tracking area; performing face authentication using an image of the person to be authenticated acquired while the tracking means is tracking the person to be authenticated; controlling the display to display the first image and a second image in which the authentication result of the person to be authenticated is superimposed on an image of the person to be authenticated acquired while tracking the person to be authenticated; An information processing method including:

11. On the computer, acquiring a first image that is an image including a predetermined area through which a person to be authenticated passes; setting a tracking area for each of the persons to be authenticated within the image; Tracking the authenticatee based on the tracking area; performing face authentication using an image of the person to be authenticated acquired while the tracking means is tracking the person to be authenticated; controlling the display to display the first image and a second image in which the authentication result of the person to be authenticated is superimposed on an image of the person to be authenticated acquired while tracking the person to be authenticated; A computer program for executing an information processing method including the steps of:

Images