Authentication device, authentication system, control method of authentication device, and program

Abstract

To provide an authentication device, an authentication system, a control method of an authentication device and a program which suppress occurrence of erroneous authentication in person collation.SOLUTION: In an enter / leave management system having a face authentication device, a control part and a display part, an authentication device 101 has: a feature vector extraction part for extracting a feature vector of a person to be authenticated from an image obtained by imaging the person to be authenticated who is going to enter / leave a predetermined section; and a person to be authenticated specification part for specifying whether the person to be authenticated is a permitted person who is permitted to enter / leave the predetermined section or a non-permitted person who is not permitted to enter / leave the predetermined section, by collating a feature vector of the permitted person, a feature vector of the non-permitted person, and a feature vector of the person to be authenticated extracted by the feature vector extraction means.SELECTED DRAWING: Figure 3

Description

[Technical field]

[0001] The present invention relates to a person matching process. [Background technology]

[0002] In recent years, many techniques have been proposed to extract useful information by advanced image processing. In particular, face recognition, which uses a multi-layered neural network called a deep net to compare an input face image with face images of multiple people registered in advance and determine who the input face image is, has been actively researched and developed. Deep nets are also called deep neural networks or deep learning.

[0003] Furthermore, with the improvement in the accuracy of face recognition, the development of entrance / exit management systems using face recognition devices is also actively underway. In this entrance / exit management system, a facial image of a person to be authenticated (authenticated person) is captured, and facial feature information (feature vector) specific to that person obtained from the facial image is compared with registered face information (registered feature vector) that is registered in advance as a person permitted to enter. Then, based on the comparison result, it is determined whether or not the authenticated person is a person permitted to enter. Then, the entrance / exit management system performs control so as to open the door to the controlled area (area for entry and exit) only if it is determined that the person is permitted to enter.

[0004] Patent Document 1 discloses a technique for reducing the load on a computer in face recognition processing by setting only people who are permitted to enter and exit a closed space (controlled area) as the search target (match target) range. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] International Publication No. 2018 / 110012 Summary of the Invention [Problem to be solved by the invention]

[0006] However, while the method described in Patent Document 1 can reduce the processing load on a computer, it searches only for people who are permitted to enter and exit a controlled area (authorized people).When face recognition processing is performed on an image, if the photographing conditions when the image was taken or the person to be authenticated is not registered as an authorized person, a so-called erroneous authentication may occur in which a person who is not actually an authorized person is authorized to enter and exit.

[0007] The present invention has been made in consideration of the above problems, and has an object to suppress the occurrence of erroneous authentication in person matching. [Means for solving the problem]

[0008] In order to achieve the above-mentioned object, an authentication device as one aspect of the present invention has a feature vector extraction means for extracting a feature vector of a person to be authenticated from an image captured of the person attempting to enter or exit a specified area, and an identification means for identifying whether the person to be authenticated is an authorized person or an unauthorized person by comparing the feature vector of an authorized person who is a person who is authorized to enter or exit the specified area and the feature vector of an unauthorized person who is a person who is not authorized to enter or exit the specified area with the feature vector of the person to be authenticated extracted by the feature vector extraction means. Effect of the Invention

[0009] According to the present invention, it is possible to suppress the occurrence of erroneous authentication in person matching. [Brief description of the drawings]

[0010] [Figure 1] FIG. 2 is a block diagram showing the configuration of an entrance / exit management system. [Diagram 2] FIG. 2 is a diagram illustrating a hardware configuration of the face authentication device. [Diagram 3] FIG. 1 is a block diagram showing a configuration of a face authentication device. [Figure 4] 13 is a diagram showing a state of management of person IDs and their feature vectors in a feature vector management unit. FIG. [Diagram 5] 11 is a flowchart showing a process flow in the entrance and exit management system. [Figure 6] 11 is a flowchart showing a process flow in the entrance and exit management system. [Figure 7] FIG. 11 is a diagram illustrating an example of a matching result. [Figure 8] FIG. 1 is a block diagram showing a configuration of a face authentication device. [Figure 9] 11 is a flowchart showing a process flow of the entrance and exit management system. [Figure 10] 11 is a flowchart showing a process flow of the entrance and exit management system. [Figure 11] FIG. 1 is a block diagram showing a configuration of a face authentication device. [Figure 12] 11 is a flowchart showing a process flow of the entrance and exit management system. [Figure 13] 11 is a flowchart showing a process flow of the entrance and exit management system. [Figure 14] FIG. 2 is a block diagram showing the configuration of an entrance / exit management system. [Figure 15] 11 is a flowchart showing a process flow of the entrance and exit management system. [Figure 16] 11 is a flowchart showing a process flow of the entrance and exit management system. [Figure 17] 11 is a flowchart showing a process flow of the entrance and exit management system. [Figure 18] 13 is a diagram showing a state of management of person IDs and their feature vectors in a feature vector management unit. FIG. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0011] Hereinafter, preferred embodiments of the present invention will be described in detail with reference to the accompanying drawings. Note that the configurations shown in the following embodiments are merely examples, and the present invention is not limited to the configurations shown in the drawings.

[0012] <Embodiment 1> In this embodiment, a detailed description will be given of a case where a biometric authentication method (technique) is applied to face authentication. Furthermore, a description will be given of an example where a face authentication device that performs face authentication is used in an entrance / exit management system.

[0013] In face recognition, a feature extractor is used to extract information (feature vectors) that distinguish a person from a face image. In face recognition, for example, if the similarity score obtained by comparing two feature vectors obtained from two images is high (above a certain threshold), the faces in the two images are determined to be of the same person. On the other hand, if the similarity score is low (below a certain threshold), the faces in the two images are determined to be of different people.

[0014] In the entrance / exit management system 100 including the face recognition device 101 of this embodiment, face recognition extracts a face area of ​​a person from an image obtained by photographing (capturing) the person, and judges whether or not the person is a person permitted to enter by comparing the face area with face images of people registered in advance. If the person is judged to be permitted to enter, the entrance / exit management system 100 controls the opening of the gate or unlocks the electric lock on the door to allow the person to pass through or enter. In the face recognition according to this embodiment, the face image of the person to be authenticated is not only compared with face images of persons permitted to enter, but also compared with face images of persons not permitted to enter. On the other hand, if the person is judged to be not permitted to enter, the entrance / exit management system 100 prohibits the person from passing through by not opening the gate or unlocking the electric lock.

[0015] In this embodiment, an admitted person refers to a person who is permitted to pass through or enter a specific controlled area using the entrance / exit management system 100. Moreover, a non-admitted person does not refer to all people (in the world) other than the admitted people. In other words, a non-admitted person refers to a person who may use the entrance / exit management system 100 to enter a specific controlled area, but is not permitted to enter the controlled area.

[0016] For example, consider an entrance / exit management system for a special laboratory in a company. Since entry to the company premises is basically limited to employees, only employees have the possibility of using the laboratory entrance / exit management system 100. Only a limited number of employees among the company's employees are permitted to enter the laboratory as permitted persons. In such a case, employees other than the permitted persons are prohibited from entering. In other words, in an environment in which only certain restricted persons (e.g., employees) can use the entrance / exit management system 100, people other than permitted persons who may use the entrance / exit management system 100 are prohibited from entering.

[0017] The entrance / exit management system 100 according to the first embodiment will be described in detail below with reference to Fig. 1. Fig. 1 is a block diagram showing an example of the configuration of the entrance / exit management system 100 according to the first embodiment. The entrance / exit management system 100 according to the first embodiment includes a face recognition device 101, a control unit 102, and a display unit 103.

[0018] The face recognition device 101 is a device that performs face recognition using a feature extractor that extracts information (feature vector) that distinguishes a person from a face image. The face recognition device 101 will be described later with reference to FIG.

[0019] The control unit 102 includes a CPU (Central Processing Unit), a memory (storage unit), etc., and is configured by at least one computer. The control unit 102 also performs overall control of the entire entrance and exit management system 100 including the face recognition device 101 according to a program stored in the memory.

[0020] Furthermore, the control unit 102 controls entry and exit to a predetermined controlled area according to the information of the person to be authenticated output from the face authentication device 101. That is, it controls permission or denial of the entry (admission) of the person to be authenticated into a predetermined controlled area. The information of the person to be authenticated output from the face authentication device 101 includes the person ID of the person to be authenticated and information on whether the person to be authenticated is a person permitted to enter or a person not permitted to enter (hereinafter referred to as the person to be authenticated information). For example, if the person to be authenticated is a person permitted to enter, the gate to the controlled area is opened and the electric lock on the door is unlocked. On the other hand, if the person to be authenticated is a person not permitted to enter, the gate is not opened and the electric lock is not unlocked, thereby prohibiting the person to be authenticated from entering (admission).

[0021] The display unit 103 is a display device (display device) such as a display or a monitor. In the first embodiment, the display unit 103 displays the authenticated person information output from the face authentication device 101 on a screen, and presents the authenticated person information to the authenticated person (user). A detailed description of the display unit 103 will be given later.

[0022] In addition to the above-mentioned configuration, the entrance and exit management system 100 may have a sound output device configured with a mechanism capable of outputting sound such as a speaker. The sound output device outputs, as voice information, the contents to be conveyed to the person to be authenticated based on the person to be authenticated information.

[0023] 2 is a diagram showing an example of a hardware configuration of the face authentication device 101. The face authentication device 101 includes a CPU 201, a ROM 202, a RAM 203, a secondary storage device 204, an imaging unit 205, and a communication unit 206.

[0024] The functions realized by the hardware shown in Fig. 2 may be realized, for example, by automatically generating a dedicated circuit on an FPGA (Field Programmable Gate Array) from a predetermined program using a predetermined compiler. Alternatively, a gate array circuit may be formed in the same manner as an FPGA, and the functions may be realized as hardware. Alternatively, the functions may be realized by an ASIC (Application Specific Integrated Circuit).

[0025] The CPU 201 is a central processing unit and performs overall control of the face recognition device 101. The CPU 201 also reads out control programs recorded in a ROM (Read Only Memory) 202 and executes various processes. The ROM 202 is a non-volatile memory and stores the program of the first embodiment and other programs and data required for control.

[0026] The RAM (Random Access Memory) 203 is a volatile memory and is used as a temporary storage area such as a main memory or a work area. The secondary storage device 204 stores image and video data, data to be processed, programs, etc. The secondary storage device 204 is configured by a non-volatile storage medium such as an HDD, a flash memory, or an SD card.

[0027] The imaging unit 205 is composed of an imaging lens and an imaging sensor such as a CCD or CMOS. The imaging unit 205 captures an image of a person or the like who is to be subjected to face authentication, and acquires the image. In this embodiment, the imaging unit 205 captures an image of a person (person to be authenticated) who wishes to enter a predetermined controlled area, and acquires image data including the face of the person to be authenticated from the captured image. The communication unit 206 performs communication processing with the control unit 102, the display unit 103, or an external device such as a client device or a server, either wired or wirelessly.

[0028] The face authentication device 101 may have a display device, an input device, etc. In this case, the display device may be configured as a display device such as a display or monitor that displays images and setting contents. The input device may be configured as a device for allowing a user to manually perform settings, such as a keyboard, a mouse, or a touch panel.

[0029] 3 is a block diagram showing the functional configuration of the face authentication device 101. The face authentication device 101 includes an image acquisition unit 301, a face detection unit 302, a feature vector extraction unit 303, a feature vector management unit 304, and an authentication target person identification unit 305.

[0030] 3, a program for providing the function of each functional block is stored in a memory such as ROM 202. The program is then read into RAM 203 and executed by CPU 201 to realize the functions. Note that the configuration of the functional blocks shown in FIG. 3 is an example, and multiple functional blocks may constitute one functional block, or any functional block may be divided into blocks performing multiple functions.

[0031] The image acquisition unit 301 acquires an image to be subjected to face authentication. In the first embodiment, the image acquisition unit 301 acquires a captured image (image data) including the face of a person (person to be authenticated) who wants to enter a predetermined controlled area from the imaging unit 205. The acquired image data is output (transmitted) to the face detection unit 302. The image acquisition unit 301 may acquire image data from a source other than the imaging unit 205. For example, the image may be acquired from a memory or storage medium in which image data including the face of the person to be authenticated captured by the imaging unit 205 or an imaging device (camera) other than the imaging unit 205 is stored. Also, for example, image data including the face of the person to be authenticated may be acquired from an imaging device other than the imaging unit 205. In that case, the imaging unit 205 may be excluded from the configuration of the face authentication device 101.

[0032] The face detection unit 302 detects the face of a person in the image data output from the image acquisition unit 301. The image data of the detected face region is normalized to a fixed size by a predetermined method and output to the feature vector extraction unit 303 as a face region image. The face detection method may be an existing publicly known method. For example, a method may be used in which shapes corresponding to components of the face region such as the nose, mouth, and eyes are extracted, the size of the face is estimated from the size of both eyes and the distance between them, and an area surrounded by an area of ​​the estimated size based on a position corresponding to the center of the nose is regarded as the face region.

[0033] The feature vector extraction unit 303 performs a feature vector extraction process on the face area image output from the face detection unit 302. This process makes it possible to extract the feature vector of the person to be authenticated. The feature vector extraction unit 303 then outputs the extracted feature vector to the person to be authenticated identification unit 305. The feature vector extraction unit 303 is realized by a neural network in which feature extraction parameters obtained by machine learning using training data are set. In other words, the feature vector extraction unit 303 is configured by a trained deep net.

[0034] The feature vector management unit 304 manages feature vectors extracted from image data of persons permitted to enter and persons not permitted to enter captured in advance. The method of extracting feature vectors from these image data may be the same as the procedure described above. In other words, face detection may be performed on the image data of persons permitted to enter and persons not permitted to enter, and feature vectors may be extracted from face area images obtained as a result of the detection. In this embodiment, the feature vector management unit 304 manages the person ID of a person permitted to enter and its feature vector in association (linked) relation. Furthermore, the person ID of a person not permitted to enter and its feature vector in association relation.

[0035] FIG. 4 is a diagram showing, in a table, the state of management of person IDs and their feature vectors in the feature vector management unit 304. Table 401 shows how person IDs and their feature vectors are managed for people permitted to enter. For example, it is shown that a person with person ID ID_1 is a person permitted to enter, and its feature vector is Feature_1. Similarly, people with person IDs ID_2 and ID_5 are also managed as people permitted to enter. Table 402 shows how person IDs and their feature vectors are managed for people not permitted to enter. For example, it is shown that a person with person ID ID_3 is a person not permitted to enter, and its feature vector is Feature_3. Similarly, people with person IDs ID_4, ID_6, ID_7, and ID_8 are also managed as people not permitted to enter.

[0036] The person to be authenticated identifying unit 305 collates the feature vector of the image data sent from the feature vector extracting unit 303 with the feature vector managed by the feature vector managing unit 304 to identify whether the person to be authenticated is permitted to enter or not permitted to enter. Specifically, the feature vector extracted by the feature vector extracting unit 303 is collated with the feature vector of the permitted person and the feature vector of the not permitted person managed by the feature vector managing unit 304 to calculate a similarity score. Here, the cosine distance between both feature vectors is taken as the similarity score. It can be said that the larger the similarity score is, the more similar the two feature vectors are. Then, the person to be authenticated identifying unit 305 compares the maximum similarity score (maximum value of the similarity scores) among the calculated similarity scores with a predetermined threshold value set in advance to identify the ID of the person to be authenticated. In addition, based on the identified ID information, it is identified whether the person to be authenticated is permitted to enter or not permitted to enter.

[0037] Here, for example, it is assumed that ID information and feature vectors are associated and managed in the feature vector management unit 304 as shown in Fig. 4. In this case, since there are eight feature vectors managed, eight similarity scores are calculated in the person-to-be-authenticated identification unit 305. If the maximum similarity score (maximum similarity score) among the eight calculated similarities exceeds a preset predetermined threshold, the person ID corresponding to the feature vector used when the maximum similarity score was calculated is identified as the ID of the person-to-be-authenticated.

[0038] For example, assume that the similarity score between the feature vector of the input image sent from the feature vector extraction unit 303 and Feature_2 in Fig. 4 is the largest among the eight calculated similarity scores and is greater than a preset threshold. In this case, ID_2, which is the person ID corresponding to Feature_2, becomes the ID of the person to be authenticated. Furthermore, since ID_2 is a person permitted to enter, the person to be authenticated is identified as ID_2, who is also permitted to enter.

[0039] Similarly, assume that the similarity score between the feature vector of the input image transmitted from the feature vector extraction unit 303 and Feature_7 in Fig. 4 is the largest among the eight calculated similarity scores and is greater than a preset threshold. In this case, ID_7, which is the person ID corresponding to Feature_7, becomes the ID of the person to be authenticated. Furthermore, since ID_7 is a person not permitted to enter, the person to be authenticated is identified as ID_7, who is also a person not permitted to enter.

[0040] In this way, when the maximum similarity score exceeds a predetermined threshold and is a similarity score calculated from a person permitted to enter, the person to be authenticated is identified as a person permitted to enter by the person to be authenticated identification unit 305. On the other hand, when the calculated maximum similarity score does not exceed the predetermined threshold, the person to be authenticated is determined to be a person not permitted to enter.

[0041] A person (person to be authenticated) who uses the entrance / exit management system 100 of this embodiment is classified as either a person permitted to enter or a person not permitted to enter. Here, if the image of the person to be authenticated captured by the imaging unit 205 is an image that is inappropriate for face authentication due to some reason (for example, face orientation or lighting conditions), accurate face authentication may not be possible. In order to prevent erroneous identification even in such a case, in this embodiment, a predetermined threshold is set in advance, and whether the calculated similarity score is reliable or not is confirmed depending on whether the threshold is exceeded. The predetermined threshold can be arbitrarily set or changed by a person who has authority over entry and exit to the controlled area, such as the administrator of the entrance / exit management system 100.

[0042] It may also happen that a person who is neither permitted nor not permitted to enter (for example, a malicious intruder) uses the entrance and exit management system 100. Therefore, by setting a predetermined threshold as described above and comparing the calculated similarity score with the predetermined threshold, it is possible to prevent such a person from being erroneously permitted to enter.

[0043] The above is the hardware configuration and functional configuration of the face authentication device 101. Here, the display unit 103 will be described in detail. As described above, the display unit 103 displays (presents to the user) the authenticated person information (such as the person ID of the authenticated person and information on whether the authenticated person is permitted to enter or not permitted to enter) output from the face authentication device 101. This allows the authenticated person to check the authenticated person information on the screen of the display unit 103. The control unit 102 controls the display of the authenticated person information on the display unit 103. The CPU 201 controls the display of the authenticated person information on the display unit 103.

[0044] Here, for example, if the person to be authenticated is identified by the person to be authenticated identification unit 305 as ID_2, who is a person permitted to enter, the person ID of the person to be authenticated, ID_2, and a message indicating that entry has been permitted are displayed on the display unit 103. As an example of the display, "You are ID_2, aren't you? You are permitted to enter the room" is displayed (display pattern 1).

[0045] Furthermore, in the case where the person to be authenticated is identified by the person to be authenticated identification unit 305 as ID_7, who is a person not permitted to enter the room, the person ID of the person to be authenticated, ID_7, and a message indicating that entry is not permitted are displayed on the display unit 103. As an example of the display, "You are ID_7, aren't you? Entry is not permitted" is displayed (display pattern 2).

[0046] In addition, if the maximum similarity score does not exceed the threshold and the authenticated person identification unit 305 determines that entry is not permitted, the person ID information is not displayed and a message such as "Entry is not permitted" is simply displayed (display pattern 3).

[0047] In this way, the information according to the result of the matching by the authenticated person identification unit 305 is displayed on the display unit 103, so that the authenticated person can know that he or she is permitted to enter or leave. Furthermore, even if the authenticated person is determined not to be permitted to enter, he or she can know the reason why.

[0048] For example, when a message is displayed together with the person ID as in display pattern 2, the person knows that he or she is not qualified to enter the room in the first place. In such a case, for example, the person can apply for permission to enter the room to the administrator of the entry and exit management system 100. If the administrator approves the application for permission to enter the room, the person who made the application will be managed as a person permitted to enter the room, and entry will be permitted smoothly from the next time.

[0049] Also, for example, when the person's ID is not displayed and only a message that entry is not permitted is displayed as in display pattern 3, it is possible that the captured image was inappropriate for face recognition. In such a case, it is possible to improve the possibility of entry being permitted by changing the direction of capture from the current position and capturing an image again, or by adjusting the surrounding lighting before capturing an image.

[0050] By displaying such information on the display unit 103, it becomes possible to clearly inform the authenticated person of the reason why entry was denied. This also makes clear the measures that the authenticated person who was determined to be denied entry should take, which leads to improved convenience of the entrance / exit management system 100 using face recognition.

[0051] Next, the process flow of the entrance / exit management system 100 will be described with reference to Fig. 5 and Fig. 6, focusing on the process in the face authentication device 101 of the first embodiment. Fig. 5 and Fig. 6 are flowcharts showing the process flow of the entrance / exit management system in the first embodiment. Each process below is realized by the control unit 102 executing a program stored in memory. Alternatively, it may be realized by the CPU 201 executing a program stored in the ROM 202 or the like. Also, by adding an S to the beginning of each process (step), the notation of the process (step) is omitted.

[0052] In S501, the image acquisition unit 301 acquires a captured image of a person to be recognized who is a person to be subjected to face recognition, from the imaging unit 205. After that, the image acquisition unit 301 outputs image data of the acquired captured image to the face detection unit 302.

[0053] In S502, the face detection unit 302 performs face detection on the image data of the captured image output in S501. Then, the image data of the face area detected by the face detection unit 302 is normalized to a fixed size by a predetermined method and output to the feature vector extraction unit 303 as a face area image.

[0054] In S503, the feature vector extraction unit 303 extracts a feature vector from the face area image output in S502. After that, the feature vector extraction unit 303 outputs the extracted feature vector to the authentication-target person identification unit 305.

[0055] In S504, the authenticated person identification unit 305 compares the feature vector extracted by the feature vector extraction unit 303 in S503 with the feature vector managed by the feature vector management unit 304, and calculates a similarity score. The feature vectors managed by the feature vector management unit 304 include feature vectors of permitted and unpermitted persons. That is, the authenticated person identification unit 305 calculates a similarity score by comparing the feature vectors of permitted and unpermitted persons with the feature vector extracted by the feature vector extraction unit 303. When the feature vector management unit 304 manages one feature vector, one similarity score is calculated. When the feature vector management unit 304 manages multiple feature vectors, the number of similarity scores calculated is equal to the number of feature vectors managed by the feature vector management unit 304.

[0056] In S505, the authenticated person identification unit 305 compares the maximum value of the similarity scores calculated in S504 with a predetermined threshold value, and determines whether the maximum value of the similarity scores exceeds the predetermined threshold value. If it is determined that the calculated maximum value of the similarity scores does not exceed the predetermined threshold value, the process proceeds to S506. On the other hand, if it is determined that the calculated maximum value of the similarity scores exceeds the predetermined threshold value, the process proceeds to S508.

[0057] In S506, the authentication-target person identification unit 305 determines that the authentication-target person is a person not permitted to enter. In S507, since the authentication-target person is determined to be a person not permitted to enter in S506, the control unit 102 prohibits the authentication-target person from entering (admitting) the room by not opening the gate or unlocking the electric lock. Furthermore, a message indicating that entry / exit is not permitted is displayed on the screen of the display unit 103. Specifically, a message such as the above-mentioned display pattern 3 is displayed on the screen of the display unit 103 to notify the authentication-target person that entry / exit is not permitted. Then, the process ends.

[0058] In S508, the authenticated person identification unit 305 identifies the person ID of the person with the maximum similarity score (maximum similarity score) as the ID of the authenticated person. In addition, it is determined whether the maximum similarity score was obtained by matching with the feature vector of the permitted person (whether the maximum similarity score is the similarity score of the feature vector of the permitted person). If it is determined that the maximum similarity score was obtained by matching with the feature vector of the permitted person, the process proceeds to S511. On the other hand, if the maximum similarity score was not obtained by matching with the feature vector of the permitted person (if it was obtained by matching with the feature vector of the prohibited person), the process proceeds to S509. This process identifies whether the authenticated person is a person permitted to enter or a person prohibited from entering.

[0059] In S509, the authenticated person identification unit 305 determines that the authenticated person is not permitted to enter. In S510, since it has been determined in S509 that the authenticated person is not permitted to enter, the control unit 102 prohibits the authenticated person from entering (admitting) the room by not opening the gate or unlocking the electric lock. Furthermore, a message indicating that entry / exit is not permitted is displayed on the screen of the display unit 103. Specifically, the person ID and a message, etc. of the authenticated person are displayed on the screen of the display unit 103 as in the above-mentioned display pattern 2, to notify the authenticated person that entry / exit is not permitted. Then, the process ends.

[0060] In S511, the authenticated person identification unit 305 determines that the authenticated person is permitted to enter. In S512, since it has been determined in S511 that the authenticated person is permitted to enter, the control unit 102 permits the authenticated person to enter (enter) the room by opening the gate or unlocking the electric lock on the door. Furthermore, a message indicating that entry / exit is not permitted is displayed on the screen of the display unit 103. Specifically, as in the above-mentioned display pattern 1, the person ID and a message, etc. of the authenticated person are displayed on the screen of the display unit 103 to notify the authenticated person that entry / exit is permitted. Then, the process ends.

[0061] If the entrance and exit management system 100 has a sound output device, in S507, S510, and S512, in addition to displaying the authenticated person information, etc. on the display unit 103, sound based on the result identified by the authenticated person identification unit 305 may be output from the sound output device. By outputting sound based on the processing result from the sound output device as well, the authenticated person can recognize the authentication result not only visually but also aurally. It is also possible to output only sound based on the result identified by the authenticated person identification unit 305 from the sound output device without displaying on the display unit 103.

[0062] As described above, according to the entrance / exit management system 100 of the first embodiment, even if the person to be authenticated is determined to be denied entry, the person to be authenticated can know the reason. Therefore, the person to be authenticated can take appropriate measures according to the cause. This leads to improved reliability of the entrance / exit management system 100 using the face recognition device 101. Furthermore, if appropriate measures are taken, it is possible to improve the expectation of being allowed to enter the room from the next time onwards, which also leads to improved convenience.

[0063] A person (person to be authenticated) who uses the entrance and exit management system 100 of the first embodiment is classified as either a person permitted to enter or a person not permitted to enter. This makes it possible to reduce erroneous authentication (suppression of erroneous authentication) caused by mistakenly entering the person ID of a person to be authenticated.

[0064] In face recognition, if the feature vector of the person to be authenticated is registered in advance, the feature vector obtained from the captured image during matching is matched with the registered feature vector of the person to be authenticated, and the person is correctly authenticated as the person. In other words, the similarity score obtained when matching with the person's feature vector is almost always the maximum similarity score. However, if the feature vector of the person to be authenticated is not registered, the score with another registered person may slightly exceed the threshold, and the person may be mistaken for another person and be erroneously authenticated. This erroneous authentication is particularly likely to occur when the person to be authenticated is not registered, but a person who closely resembles the person to be authenticated (for example, a brother, sister, relative, etc.) is registered.

[0065] In the entrance / exit management system 100 of this embodiment, the feature vector of the person to be authenticated is registered and becomes a subject of matching, regardless of whether the person is permitted to enter or not permitted to enter. This makes it possible to prevent erroneous authentication as described above, and improves the reliability of the entrance / exit management system 100 including the face recognition device 101.

[0066] FIG. 7 is a schematic diagram showing how a person to be authenticated is matched with persons permitted to enter and persons not permitted to enter.

[0067] In Fig. 7, the person ID of the person to be authenticated is H. The person IDs of the permitted persons to be compared are denoted by A, B, and C. When the feature vector of the person to be authenticated is compared with the feature vector of an permitted person whose person ID is A, the similarity score is assumed to be 0.10. When the feature vector of the person to be authenticated is compared with the feature vector of an permitted person whose person ID is B, the similarity score is assumed to be 0.25. When the feature vector of the person to be authenticated is compared with the feature vector of an permitted person whose person ID is C, the similarity score is assumed to be 0.40.

[0068] The person IDs of the persons not permitted to enter, who are the objects of comparison, are denoted as D, E, F, G, and H. It is assumed that the similarity score is 0.05 when the feature vector of the person to be authenticated is compared with the feature vector of the person not permitted to enter whose person ID is D. It is assumed that the similarity score is 0.20 when the feature vector of the person to be authenticated is compared with the feature vector of the person not permitted to enter whose person ID is E. It is assumed that the similarity score is 0.30 when the feature vector of the person to be authenticated is compared with the feature vector of the person not permitted to enter whose person ID is F. It is assumed that the similarity score is 0.30 when the feature vector of the person to be authenticated is compared with the feature vector of the person not permitted to enter whose person ID is G. It is assumed that the similarity score is 0.80 when the feature vector of the person to be authenticated is compared with the feature vector of the person not permitted to enter whose person ID is H. It is assumed that the threshold value is 0.35.

[0069] 7, if the person to be authenticated is matched only with persons permitted to enter, the maximum similarity score will be 0.40. The maximum similarity score exceeds the threshold value of 0.35. In this case, the person to be authenticated (person ID H), who should not have been permitted to enter, will be identified (authenticated) as person ID C and will be permitted to enter. In other words, the person to be authenticated will be erroneously authenticated as someone else, and as a result, the entrance and exit management system will not operate correctly.

[0070] However, in the entrance and exit management system 100 of the first embodiment, the person to be authenticated is not only compared with persons permitted to enter, but also compared with persons not permitted to enter. Taking FIG. 7 as an example, by also comparing with persons not permitted to enter, the maximum similarity score is 0.80. The maximum similarity score is 0.80, which exceeds the threshold value of 0.35. As a result, the person to be authenticated (person ID H) is identified (authenticated) as a person not permitted to enter, and is controlled as not permitted to enter.

[0071] In this way, by comparing the person to be authenticated not only with persons permitted to enter but also with persons not permitted to enter, the feature vector of the person to be authenticated is always compared with the feature vector of the person registered. Therefore, according to the entry / exit management system 100 of the first embodiment, the similarity score with the feature vector of the person registered can be significantly higher than the similarity score when compared with another person. This makes it possible to suppress erroneous authentication in which the person ID of the person to be authenticated is mistaken for the ID of another person.

[0072] <Embodiment 2> The following describes the entrance / exit management system 100 in the second embodiment. In the second embodiment, an example in which a face recognition device is used in the entrance / exit management system will also be described. Note that in the second embodiment, differences from the entrance / exit management system 100 in the first embodiment will be described, and descriptions of similar points and configurations (hardware configurations and functional configurations) will be omitted.

[0073] In the second embodiment, an example will be shown in which a method for reducing the cost (processing amount or processing time) required for the process of identifying a person to be authenticated, which is carried out by the person to be authenticated identification unit 805, is used.

[0074] Fig. 8 is a block diagram showing the configuration of a face authentication device 801 in embodiment 2. In Fig. 8, the same functional units as those in Fig. 3 are given the same numbers as those in Fig. 3, and the description thereof will be omitted.

[0075] As in the first embodiment, the authenticated person identification unit 805 compares the feature vector extracted from the image data by the feature vector extraction unit 303 with the feature vector managed by the feature vector management unit 304, and calculates a similarity score. Here, the authenticated person identification unit 805 of the second embodiment first compares the feature vector extracted from the image data by the feature vector extraction unit 303 with the feature vector of the person permitted to enter managed by the feature vector management unit 304 as the feature vector to be compared. Hereinafter, the similarity score obtained at this time will be referred to as the permitted person similarity score.

[0076] Furthermore, the authentication target person identification unit 805 compares the maximum similarity score among the calculated authorized person similarity scores with a predetermined threshold value set in advance. If the maximum similarity score among the authorized person similarity scores exceeds the predetermined threshold value, the person ID corresponding to the feature vector used when the maximum similarity score was calculated is identified as the ID of the authentication target person. Also, the authentication target person is identified as a person permitted to enter the room.

[0077] On the other hand, if the maximum similarity score among the permitted person similarity scores does not exceed the threshold, the feature vector extracted from the image data by the feature vector extraction unit 303 is compared with the feature vector of a person not permitted to enter managed by the feature vector management unit 304. The similarity score obtained at this time is referred to as the prohibited person similarity score.

[0078] Furthermore, the authentication target person identification unit 805 compares the maximum similarity score among the unauthorized person similarity scores with a preset threshold value. If the maximum similarity score among the unauthorized person similarity scores exceeds the threshold value, the person ID corresponding to the feature vector used when the maximum similarity score was calculated is identified as the ID of the authentication target person. Also, the authentication target person is identified as a person not permitted to enter. On the other hand, if the maximum similarity score among the unauthorized person similarity scores does not exceed the threshold value, the authentication target person is identified as a person not permitted to enter.

[0079] In this way, according to the authentication-target person identification unit 805 of the second embodiment, it is possible to determine whether or not to compare the feature vector of an unauthorized person with the extracted feature vector of the authentication-target person, based on the result of the comparison with the feature vector of a permitted person. In other words, when the authentication-target person is a person permitted to enter, it is possible to omit the comparison between the feature vector extracted from the image data by the feature vector extraction unit 303 and the feature vector of the person not permitted to enter. In this way, it is possible to reduce the processing amount and processing time of the comparison process.

[0080] Next, the process flow of the entrance / exit management system 100 will be described with reference to Figs. 9 and 10, focusing on the process in the face authentication device 801 of the second embodiment. Figs. 9 and 10 are flowcharts showing the process flow of the entrance / exit management system 100 in the second embodiment. Each process below is realized by the control unit 102 executing a program stored in memory. Alternatively, it may be realized by the CPU 201 executing a program stored in the ROM 202 or the like. Moreover, by adding an S to the beginning of each process (step), the notation of the process (step) is omitted. Moreover, in Figs. 9 and 10, the same numbers as in Figs. 5 and 6 are assigned to steps that perform the same processes (operations) as in Figs. 5 and 6, and their explanations are omitted.

[0081] In S904, the authenticated person identification unit 805 compares the feature vector extracted from the image data by the feature vector extraction unit 303 in S503 with the feature vector of the entry-allowed person managed in the feature vector management unit 304, and calculates an allowed person similarity score. When there is one feature vector of the entry-allowed person managed in the feature vector management unit 304, one allowed person similarity score is calculated. When there are multiple feature vectors of the entry-allowed person managed in the feature vector management unit 304, the same number of allowed person similarity scores are calculated as the number of managed feature vectors.

[0082] In S905, the authenticated person identification unit 805 compares the maximum value (maximum similarity score) of the permitted person similarity scores calculated in S904 with a predetermined threshold, and determines whether or not the maximum value of the permitted person similarity scores exceeds the predetermined threshold. If it is determined that the calculated maximum value of the permitted person similarity scores does not exceed the predetermined threshold, the process proceeds to S906. On the other hand, if it is determined that the calculated maximum value of the permitted person similarity scores exceeds the predetermined threshold, the process proceeds to S511, and then to S512.

[0083] In S906, the authenticated person identification unit 805 compares the feature vector extracted from the image data by the feature vector extraction unit 303 in S503 with the feature vector of the entry prohibited person managed in the feature vector management unit 304, and calculates a similarity score to the entry prohibited person. When there is one feature vector of the entry prohibited person managed in the feature vector management unit 304, one similarity score to the entry prohibited person is calculated. When there are multiple feature vectors of the entry prohibited person managed in the feature vector management unit 304, the same number of similarity scores to the entry prohibited person are calculated as the number of the feature vectors of the entry prohibited person managed.

[0084] In S907, the authentication target person identification unit 805 compares the maximum value (maximum similarity score) of the unauthorized person similarity scores calculated in S906 with a predetermined threshold, and judges whether the maximum value of the unauthorized person similarity scores exceeds the predetermined threshold. If it is judged that the calculated maximum value of the unauthorized person similarity scores does not exceed the predetermined threshold, the process proceeds to S509, and then to S510. On the other hand, if it is judged that the calculated maximum value of the unauthorized person similarity scores exceeds the predetermined threshold, the process proceeds to S506, and then to S507.

[0085] If the entrance and exit management system 100 has a sound output device, in S507, S510, and S512, in addition to displaying the authenticated person information, etc. on the display unit 103, sound based on the result identified by the authenticated person identification unit 805 may be output from the sound output device. By outputting sound based on the processing result from the sound output device, the authenticated person can recognize the authentication result not only visually but also aurally. It is also possible to output only sound based on the result identified by the authenticated person identification unit 805 from the sound output device without displaying on the display unit 103.

[0086] As described above, according to the entrance / exit management system 100 including the face authentication device 801 of the second embodiment, when the person to be authenticated is a person permitted to enter, it is possible to omit the comparison of the feature vector extracted by the feature vector extraction unit 303 with the feature vector of a person not permitted to enter. This makes it possible to reduce the amount of processing and the processing time of the comparison process. In particular, when the entrance / exit management system 100 of the second embodiment is used in a place where the number of people not permitted to enter is much greater than the number of people permitted to enter, the effect of reducing the amount of processing and the processing time is high. For example, the reduction effect is particularly high in a situation where the access restrictions are strict, such as a highly confidential server room, where only a portion of many employees are permitted to enter.

[0087] <Embodiment 3> The following describes the entrance / exit management system 100 in the third embodiment. In the third embodiment, an example in which a face recognition device is used in the entrance / exit management system will also be described. Note that in the third embodiment, differences from the entrance / exit management systems 100 in the first and second embodiments will be described, and descriptions of similarities and configurations (hardware configurations and functional configurations) will be omitted.

[0088] In the second embodiment, a method for reducing the processing amount and processing time of the matching process was described. Here, when configured as in the second embodiment, depending on the degree to which a predetermined threshold is set, if the matching result is as shown in Fig. 7, there is a possibility that the person to be authenticated (person ID H) will be erroneously authenticated as person ID C. In the third embodiment, a case will be described in which a reduction in the processing amount is achieved while suppressing erroneous authentication that may occur in the configuration of the second embodiment.

[0089] Fig. 11 is a block diagram showing the configuration of a face recognition device 1101 according to embodiment 3. In Fig. 11, the same functional units as those in Fig. 3 are given the same numbers as those in Fig. 3, and the description thereof will be omitted.

[0090] The authentication-subject person identification unit 1105 compares the feature vector extracted from the image data sent from the feature vector extraction unit 303 with the feature vector managed by the feature vector management unit 304, and calculates a similarity score.

[0091] Here, in the third embodiment, two thresholds are set in advance as thresholds for comparison with the similarity score. These are called the first threshold and the second threshold, respectively. Furthermore, the first threshold is larger than the second threshold (a value larger than the first threshold). Note that the threshold used for comparison with the similarity score in the first and second embodiments corresponds to the second threshold in the third embodiment. Therefore, in the third embodiment, the first threshold is set to a value larger than the threshold in the first and second embodiments.

[0092] The authenticated person identification unit 1105 first compares the feature vector extracted from the image data by the feature vector extraction unit 303 with the feature vector of the person permitted to enter managed by the feature vector management unit 304, and calculates a similarity score. The similarity score obtained at this time is called the permitted person similarity score.

[0093] Furthermore, the authentication target person identification unit 1105 compares the maximum similarity score among the permitted person similarity scores with a first threshold value set in advance. If the maximum similarity score among the permitted person similarity scores exceeds the first threshold value, the person ID corresponding to the feature vector used when the maximum similarity score was calculated is identified as the ID of the authentication target person. Also, the authentication target person is identified as a person permitted to enter the room.

[0094] On the other hand, if the maximum similarity score among the permitted person similarity scores does not exceed the first threshold, the feature vector extracted from the image data by the feature vector extraction unit 303 is compared with the feature vector of the person not permitted to enter managed by the feature vector management unit 304. Then, a similarity score is calculated. The similarity score obtained at this time is referred to as the prohibited person similarity score.

[0095] Furthermore, the authentication target person identification unit 1105 compares the maximum similarity score among the combined permitted person similarity score and non-permitted person similarity score with a second threshold. If the maximum similarity score exceeds the second threshold, the person ID corresponding to the feature vector used when the maximum similarity score was calculated is identified as the ID of the authentication target person. Also, depending on whether the identified person ID is a permitted person or a non-permitted person, it is determined whether the authentication target person is a permitted person or a non-permitted person.

[0096] If the maximum similarity score among the combined permitted person similarity score and non-permitted person similarity score does not exceed the second threshold, the person to be authenticated is identified as a person not permitted to enter the room.

[0097] As described above, the authenticated person identification unit 1105 first compares the permitted person similarity score with the first threshold. As described above, the first threshold is a value greater than the second threshold. Therefore, when the maximum permitted person similarity score exceeds the first threshold, it means that the authenticated person is authenticated as a permitted person with a fairly high degree of certainty. In other words, the comparison with the feature vector of a person not permitted to enter can be omitted only when the person is authenticated with a high degree of certainty in comparison with a permitted person, which makes it possible to suppress erroneous authentication.

[0098] Next, the process flow of the entrance / exit management system 100 in the third embodiment will be described with reference to Figs. 12 and 13, focusing on the process in the face authentication device 1101. Figs. 12 and 13 are flowcharts showing the process flow of the entrance / exit management system 100 in the third embodiment. Each process is realized by the control unit 102 executing a program stored in the memory. Alternatively, each process may be realized by the CPU 201 executing a program stored in the ROM 202 or the like. Each process (step) is represented by adding an S to the beginning, so that the description of the process (step) is omitted. In Figs. 12 and 13, the same numbers as in Figs. 5, 6, and 9 are assigned to steps that perform the same processes (operations) as in Figs. 5, 6, and 9, and the description thereof is omitted.

[0099] In S1205, the authenticated person identification unit 1105 compares the maximum value (maximum similarity score) of the permitted person similarity scores calculated in S904 with a first threshold value, and determines whether or not the maximum value of the permitted person similarity scores exceeds a predetermined threshold value. If it is determined that the calculated maximum value of the permitted person similarity scores does not exceed the first threshold value, the process proceeds to S906. If it is determined that the calculated maximum value of the permitted person similarity scores exceeds the first threshold value, the process proceeds to S511, and then to S512.

[0100] In S1207, the authenticated person identification unit 1105 compares the maximum value (maximum similarity score) among the calculated permitted person similarity scores and non-permitted person similarity scores with a second threshold, and determines whether the maximum similarity score exceeds the second threshold. If it is determined that the calculated maximum similarity score does not exceed the second threshold, the process proceeds to S506, and then to S507. On the other hand, if it is determined that the maximum similarity score exceeds the second threshold, the process proceeds to S508.

[0101] In S508, the same process as in the first embodiment is performed, and if it is determined that the maximum similarity score was obtained by matching with the feature vector of an authorized person, the process proceeds to S511 and then to S512. On the other hand, if the maximum similarity score was not obtained by matching with the feature vector of an authorized person (if it was obtained by matching with the feature vector of an unauthorized person), the process proceeds to S509 and then to S510.

[0102] If the entrance and exit management system 100 has a sound output device, in S507, S510, and S512, in addition to displaying the authenticated person information, etc. on the display unit 103, sound based on the result identified by the authenticated person identification unit 1105 may be output from the sound output device. By outputting sound based on the processing result from the sound output device, the authenticated person can recognize the authentication result not only visually but also aurally. It is also possible to output only sound based on the result identified by the authenticated person identification unit 1105 from the sound output device without displaying on the display unit 103.

[0103] In this way, in the face authentication device 1101 of the third embodiment, it is possible to suppress erroneous authentication by performing matching processing using two thresholds, the first threshold and the second threshold. For example, in the example of Fig. 7, if the first threshold is set to 0.60 and the second threshold is set to 0.35, the person to be authenticated can be correctly authenticated without erroneous authentication.

[0104] As described above, according to the entrance / exit management system 100 including the face recognition device 1101 of the third embodiment, it is possible to omit matching with the feature vector of a person not permitted to enter only when matching with an authorized person is performed with a high degree of certainty. This realizes a reduction in the amount of processing and processing time of the matching process while suppressing erroneous recognition. Suppressing erroneous recognition leads to an improvement in the reliability of face recognition, and a reduction in the amount of processing or processing time leads to an improvement in the convenience of face recognition. Achieving both improved reliability and improved convenience can also promote the spread of face recognition.

[0105] <Embodiment 4> The following describes the entrance / exit management system 1400 in the fourth embodiment. In the fourth embodiment, a case will be described in which, if the maximum similarity score does not exceed a predetermined threshold, a person is required to present personal identification information other than biometric authentication such as an ID card, and a decision is made as to whether or not to permit entry. Note that in the fourth embodiment, differences from the entrance / exit management system 100 in the first to third embodiments will be described, and explanations of similar points and configurations (hardware configurations and functional configurations) will be omitted.

[0106] In the first to third embodiments, if the maximum similarity score does not exceed a preset threshold, the person to be authenticated is not allowed to enter the room without the person ID being specified. There are two possible reasons for this phenomenon:

[0107] The first possibility is that the image captured of the person to be identified was inappropriate for facial recognition (the person was not facing the camera, the surrounding lighting conditions were poor), and the second possibility is that the person to be identified was not intended to use the access control system (such as a malicious intruder trying to force their way in).

[0108] In the case where the image capturing conditions are poor as in the former (first case), the possibility of identifying the person's ID increases if the image capturing conditions are adjusted and the image is captured again. However, it is not a desirable situation for a malicious person to repeatedly attempt face recognition as in the latter (second case). In order to avoid such a situation, in the fourth embodiment, when the maximum similarity score does not exceed a predetermined threshold set in advance, the person is required to present personal identification information other than biometric authentication (other than biometric information) such as an ID card, and a decision is made as to whether or not to allow entry.

[0109] Fig. 14 is a block diagram showing an example of the configuration of an entrance / exit management system 1400 according to embodiment 4. In Fig. 14, a face authentication device 101, a control unit 102, and a display unit 103 have the same configurations as those in embodiment 1, and therefore the description thereof will be omitted.

[0110] The ID card reader 1401 reads the person ID stored in the ID card presented by the person to be authenticated. The read person ID is output to the face authentication device 101, so that it can be determined whether the person who has had the ID card read by the ID card reader (the person currently using the ID card reader) is a person permitted to enter or not permitted to enter.

[0111] Next, the process flow of the entrance / exit management system 1400 will be described with reference to Figs. 15 and 16, focusing on the process in the face authentication device 101 of the fourth embodiment. Figs. 15 and 16 are flowcharts showing the process flow of the entrance / exit management system 1400 of the fourth embodiment. Each process below is realized by the control unit 102 executing a program stored in a memory. Alternatively, it may be realized by the CPU 201 executing a program stored in the ROM 202 or the like. Moreover, by adding an S to the beginning of each process (step), the notation of the process (step) is omitted. Moreover, in Figs. 15 and 16, the same numbers as in Figs. 5 and 6 are assigned to steps that perform the same process (operation) as in Figs. 5 and 6, and the description thereof is omitted.

[0112] In S1501, the control unit 102 requests the person to be authenticated to present an ID card (requests the presentation of ID information on the ID card). At this time, the control unit 102 may, for example, display a message to have the ID card reader 1401 read the ID card on the display unit 103. Alternatively, if the entrance and exit management system 1400 has an audio output device, the control unit 102 may output a message urging the person to have the ID card reader 1401 read the ID card from the audio output device. Alternatively, while displaying the above message on the screen, the control unit 102 may also notify the person to be authenticated of the message urging the person to have the ID card read by audio. For example, when displaying a message to have the ID card reader 1401 read the ID card on the display unit 103, the control unit 102 may display a message urging the person to have the ID card read within a predetermined time. Note that this process may be performed by the CPU 201.

[0113] In S1502, the control unit 102 determines whether or not an IC card has not been presented. If it is determined that an IC card has not been presented, the process proceeds to S1506. For example, the control unit 102 may determine that an IC card has not been presented when IC card information is not output from the IC card reader 1401 to the face authentication device 101, or when a predetermined time has elapsed. On the other hand, if it is determined that an IC card has been presented, the process proceeds to S1503. For example, the control unit 102 may determine that an IC card has been presented when it has read the IC card into the IC card reader 1401 within a predetermined time and has detected that the IC card information has been output to the face authentication device 101. This process may be performed by the CPU 201.

[0114] In S1503, the control unit 102 judges whether the ID card is genuine (the recorded ID information is correct). If it is judged that the ID card is genuine, the process proceeds to S1504. On the other hand, if it is judged that the ID card is not genuine, the process proceeds to S1506. The control unit 102 may judge that the ID card is not genuine, for example, when the given ID information has expired, or the ID information belongs to a person who does not have the authority to enter or exit, or the ID information belongs to a person who has already left the company. This process may be performed by the CPU 201.

[0115] In S1504, the authenticated person identification unit 305 determines that the authenticated person is permitted to enter. In S1505, since it was determined in S1504 that the authenticated person is permitted to enter, the control unit 102 permits the authenticated person to enter (enter) the room by opening the gate or unlocking the electric lock on the door. Furthermore, the control unit 102 displays on the screen of the display unit 103 that entry / exit is not permitted. Specifically, as in the above-mentioned display pattern 1, the person ID and a message, etc. of the authenticated person are displayed on the screen of the display unit 103 to notify the authenticated person that entry / exit is permitted. Then, the process ends.

[0116] In S1506, the authentication-target person identification unit 305 determines that the authentication-target person is not permitted to enter. In S1507, since it has been determined in S1506 that the authentication-target person is not permitted to enter, the control unit 102 prohibits the authentication-target person from entering (entering) the room by not opening the gate or unlocking the electric lock. Furthermore, the control unit 102 displays on the screen of the display unit 103 that entry / exit is not permitted. Specifically, the control unit 102 notifies the authentication-target person that entry / exit is not permitted by displaying a message such as the above-mentioned display pattern 2 or display pattern 3 on the screen of the display unit 103. Then, the processing ends.

[0117] If the entrance and exit management system 1400 has a sound output device, in S510, S512, S1505, and S1507, in addition to displaying the authenticated person information, etc. on the display unit 103, sound based on the result identified by the authenticated person identification unit 305 may be output from the sound output device. By outputting sound based on the processing result from the sound output device, the authenticated person can recognize the authentication result not only visually but also aurally. It is also possible to output only sound based on the result identified by the authenticated person identification unit 305 from the sound output device without displaying on the display unit 103.

[0118] As described above, according to the entry / exit management system 1400 of embodiment 4, even if the maximum similarity score does not exceed a predetermined threshold, it is possible to use an ID card or the like to identify whether the authenticated person is permitted or not permitted to enter, and to control entry.

[0119] Fig. 17 is a flowchart showing the process flow of the entrance / exit management system 1400 when an ID card is used, which is partially different from Fig. 15 and Fig. 16. Here, in the process methods shown in Fig. 15 and Fig. 16, there is a risk that someone who happens to find an ID card may enter the room. Therefore, Fig. 17 shows a process method that eliminates this risk.

[0120] 17 shows the process after it is determined that the maximum value of the similarity score calculated in S505 of FIG. 15 does not exceed a predetermined threshold (when the determination in S505 is No). That is, this process is started when the determination in S505 is No. In addition, in the case of the process in FIG. 17, not only the person ID information but also the face image (or its feature vector) of the legitimate ID card holder is stored in the ID card. Each process below is realized by the control unit 102 executing a program stored in the memory. Alternatively, it may be realized by the CPU 201 executing a program stored in the ROM 202 or the like. Moreover, by adding an S to the beginning of each process (step), the notation of the process (step) is omitted.

[0121] In S1701, the control unit 102 requests the person to be authenticated to present an ID card (requests the presentation of ID information on the ID card). At this time, the control unit 102 may, for example, display a message to have the ID card reader 1401 read the ID card on the display unit 103. Alternatively, if the entrance and exit management system 1400 has an audio output device, the control unit 102 may output a message urging the person to have the ID card reader 1401 read the ID card from the audio output device. Alternatively, while displaying the above message on the screen, the control unit 102 may also notify the person to be authenticated of the message urging the person to have the ID card read by audio. For example, when displaying a message to have the ID card reader 1401 read the ID card on the display unit 103, the control unit 102 may display a message urging the person to have the ID card read within a predetermined time. Note that this process may be performed by the CPU 201.

[0122] In S1702, the control unit 102 determines whether or not an IC card has not been presented. If it is determined that an IC card has not been presented, the process proceeds to S1708. For example, the control unit 102 may determine that an IC card has not been presented when IC card information is not output from the IC card reader 1401 to the face authentication device 101, or when a predetermined time has elapsed. On the other hand, if it is determined that an IC card has been presented, the process proceeds to S1703. For example, the control unit 102 may determine that an IC card has been presented when it has read the IC card into the IC card reader 1401 within a predetermined time and has detected that the IC card information has been output to the face authentication device 101. This process may be performed by the CPU 201.

[0123] In S1703, the control unit 102 judges whether the ID card is genuine (the recorded ID information is correct). If it is judged that the ID card is genuine, the process proceeds to S1704. On the other hand, if it is judged that the ID card is not genuine, the process proceeds to S1708. The control unit 102 may judge that the ID card is not genuine, for example, when the given ID information has expired, or the ID information belongs to a person who does not have the authority to enter or exit, or the ID information belongs to a person who has already left the company. This process may be performed by the CPU 201.

[0124] In S1704, the authentication-target person identification unit 305 compares the feature vector of the face image stored in the ID card with the feature vector extracted from image data capturing the authentication-target person, and calculates a similarity score.

[0125] In S1705, the authentication-subject person identification unit 305 compares the similarity score calculated in S1703 with a predetermined threshold and determines whether the similarity score exceeds the predetermined threshold. If it is determined that the calculated similarity score does not exceed the predetermined threshold, the process proceeds to S1706. On the other hand, if it is determined that the calculated similarity score exceeds the predetermined threshold, the process proceeds to S1707.

[0126] In S1706, the authentication target person identification unit 305 judges the authentication target person as a person not permitted to enter. Then, since the authentication target person is judged as a person not permitted to enter, the control unit 102 prohibits the authentication target person from entering (entering) the room by not opening the gate or unlocking the electric lock as entry control. Furthermore, the display unit 103 displays the fact that entry / exit is not permitted on the screen. Specifically, the display unit 103 displays a message such as the above-mentioned display pattern 2 or display pattern 3 or a warning message on the screen of the display unit 103 to notify the authentication target person that entry / exit is not permitted. Then, the process ends. In such a case, the presented ID card does not belong to the authentication target person, and it is judged that the authentication target person is not a regular holder. For example, the above-mentioned judgment is made when a found ID card is used.

[0127] In S1707, the authentication target person identification unit 305 refers to the person ID stored in the ID card and identifies whether the authentication target person is permitted to enter the room or not permitted to enter the room. Then, the control unit 102 controls entry based on the identification result. After that, the process ends. In such a case, it is determined that the presented ID card belongs to the authentication target person and that the authentication target person is a legitimate holder.

[0128] In S1708, the authentication-target person identification unit 305 determines that the authentication-target person is not permitted to enter. Then, since it has been determined that the authentication-target person is not permitted to enter, the control unit 102 prohibits the authentication-target person from entering (admitting) the room by not opening the gate or unlocking the electric lock as entry control. Furthermore, a message indicating that entry / exit is not permitted is displayed on the screen of the display unit 103. Specifically, a message such as the above-mentioned display pattern 2 or display pattern 3 or a warning message is displayed on the screen of the display unit 103 to notify the authentication-target person that entry / exit is not permitted. Then, the processing ends.

[0129] Furthermore, when the process of S1706 or S1708 occurs (when the authenticated person identification unit 305 determines that the authenticated person is not permitted to enter), a notification may be sent to the administrator of the entrance / exit management system 1400 or a person in a position such as a security guard. In this way, it is possible to build an entrance / exit management system 1400 that is firmly protected from intruders.

[0130] <Other embodiments> In the above-described first to fourth embodiments, the permitted and unpermitted persons are fixed. However, depending on the operation of the entrance and exit management system, the permitted and unpermitted persons may not be fixed, but may vary over time.

[0131] For example, the same person may be permitted to enter at a certain time (e.g., daytime) but not permitted to enter at another time (e.g., nighttime). Also, when the person's attributes (e.g., organization affiliation or position within a company) change, they may go from permitted to not be permitted to enter, or vice versa. Also, there is a practice of setting a time limit for permitted people to enter, so that when the time limit expires, they become not permitted to enter.

[0132] In such a case, even when the management of a certain person ID is changed from an entry permitted person to an entry prohibited person, the reason is clear. In the above-mentioned embodiments 1 to 4, the authenticated person information is displayed on the display unit 103, but in this example, in addition to the authenticated person information, the reason for the change in management (for example, the reason why a person permitted to enter has become a person prohibited from entering: the reason for the change to prohibited) may also be displayed.

[0133] FIG. 18 is a diagram showing an example in which the characteristic vector management unit 304 manages the reason for transition to unpermission in addition to the person ID and the characteristic vector.

[0134] Here, for example, if a person with ID_7 uses the entry / exit management system 100 as the person to be authenticated, the control unit 102 will display on the display unit 103, "You are ID_7, aren't you? Entry is not permitted. Your entry permission has expired." By displaying a message or the like explaining the reason for the change in management in this way, it becomes more persuasive when entry is not permitted, and users of the entry / exit management system 100, including the person to be authenticated, can more easily handle the situation appropriately.

[0135] Furthermore, in the entrance / exit management system 100, even if the person to be authenticated is not permitted to enter as explained in the above first to fourth embodiments, the entrance / exit management system 100 may be able to know the person ID of the person to be authenticated. In this example, by utilizing this, when it is determined that the person is not permitted to enter, the control unit 102 or the CPU 201 may notify (report) the identified person ID to the administrator of the entrance / exit management system 100 (inform the administrator). This allows the administrator who receives the notification to carry out operations such as re-managing the notified person ID as a person permitted to enter.

[0136] For example, when a person who is denied entry due to "expired permission" is determined to be denied entry, it is possible to re-manage them as permitted entry. Setting an expiration date for permission to enter is an effective method to prevent a person who has left the company or been transferred to a new workplace from being managed as permitted entry indefinitely. However, if a person is denied entry when the expiration date has passed and this operation state is left in place, there is a risk that user convenience will decrease.

[0137] Here, for example, even if a person is once transferred to a person not permitted to enter due to expiration of the period or the like, by subsequently using the entry / exit management system 100, it is possible to re-register the person as a person permitted to enter at that time or thereafter.

[0138] Also, although the display examples of the display unit 103 have been shown in the above embodiment, the present invention is not limited to these display examples. For example, assume that the maximum similarity score is relatively high but does not exceed the threshold (S507 in the first embodiment). In such a case, the person ID corresponding to the feature vector used when the maximum similarity score is calculated may be displayed, and a message such as "Is this Mr. ID_○? The matching result score was not sufficient, so please face the front correctly, stand still, and retake the image." may be displayed.

[0139] Or, for security reasons, in order to prevent a possible intruder from being given information about which registered person he resembles, a message such as "This person may be a registered person, but the matching score was insufficient and we were unable to allow him in. Please face the camera correctly, stand still, and retake the image" may be displayed. In this way, various forms for improving user convenience are possible for the entrance and exit management system 100 of the present invention.

[0140] Although the preferred embodiments of the present invention have been described in detail above, the present invention is not limited to such specific embodiments, and various modifications and variations are possible within the scope of the gist of the present invention described in the claims.

[0141] Furthermore, the functions of the above-described embodiments are not only realized by the computer reading and executing the program code, but also include an operating system (OS) running on the computer performing all or part of the actual processing based on the instructions of the program code.

[0142] Furthermore, the functions of the above-described embodiments may be realized by the following method: Program code read from a recording medium is written into a memory provided in a function expansion card inserted into a computer or a function expansion unit connected to a computer. Then, based on the instructions of the program code, a CPU provided in the function expansion card or function expansion unit performs some or all of the actual processing.

[0143] When the present invention is applied to the above-mentioned recording medium, the recording medium stores program codes corresponding to the respective flowcharts described above.

[0144] The disclosure of this embodiment includes the following configuration, system, method, and program.

[0145] (Configuration 1) a feature vector extraction means for extracting a feature vector of a person to be authenticated from an image captured of the person who is about to enter or exit a predetermined area; and a determination means for determining whether the person to be authenticated is the authorized person or the unauthorized person by comparing a feature vector of an authorized person who is authorized to enter the specified area and a feature vector of an unauthorized person who is not authorized to enter the specified area with the feature vector of the person to be authenticated extracted by the feature vector extraction means. 13. An authentication device comprising:

[0146] (Configuration 2) The authentication device described in configuration 1, characterized in that the identification means compares a maximum value of the similarity score obtained by performing the matching with a predetermined threshold, and identifies whether the person to be authenticated is the authorized person or the unauthorized person based on the result of the comparison.

[0147] (Configuration 3) When the maximum value of the similarity scores exceeds the predetermined threshold, the identification means further determines whether or not the maximum value of the similarity scores is a similarity score of a feature vector of the authorized person; 3. The authentication device according to configuration 2, wherein the identification means identifies whether the person to be authenticated is the authorized person or the unauthorized person based on a result of the judgment.

[0148] (Configuration 4) The authentication device described in any one of configurations 1 to 3, characterized in that the identification means determines whether or not to compare the feature vector of the unauthorized person with the feature vector of the person to be authenticated extracted by the feature vector extraction means, based on the result of the comparison with the feature vector of the authorized person.

[0149] (Configuration 5) The authentication device described in any one of configurations 1 to 3, characterized in that the identification means compares the similarity score for the authorized person obtained as a result of matching with the feature vector of the authorized person with a first threshold value, and if the similarity score for the authorized person does not exceed the first threshold value, matches the feature vector of the unauthorized person with the feature vector of the person to be authenticated extracted by the feature vector extraction means.

[0150] (Configuration 6) The authentication device described in configuration 5, wherein the identification means compares the similarity score for the authorized person and the similarity score for the unauthorized person obtained by matching the similarity score for the authorized person and the feature vector of the unauthorized person with a second threshold different from the first threshold, and identifies whether the person to be authenticated is the authorized person or the unauthorized person based on the result of the comparison.

[0151] (Configuration 7) 7. The authentication device according to configuration 6, wherein the first threshold value is greater than the second threshold value.

[0152] (Configuration 8) The authentication device described in any one of configurations 1 to 3, characterized in that the identification means requests the person to be authenticated to present ID information other than biometric information if the maximum value of the similarity score obtained by the matching does not exceed a predetermined threshold.

[0153] (Configuration 9) The authentication device according to configuration 8, wherein the identification means identifies whether the person to be authenticated is the authorized person or the unauthorized person based on the presented ID information and the similarity score.

[0154] (Configuration 10) The authentication device described in configuration 8 or 9, wherein when the ID information is presented and the ID information is correct, the identification means identifies whether the person to be authenticated is the authorized person or the unauthorized person by comparing a feature vector of a facial image included in the ID information with the feature vector of the person to be authenticated extracted by the feature vector extraction means.

[0155] (Configuration 11) 11. The authentication device according to any one of configurations 1 to 10, further comprising a display control means for displaying a result of the identification performed by the identification means on a display device.

[0156] (Configuration 12) 12. The authentication device according to any one of configurations 1 to 11, further comprising a notification unit that notifies the person to be authenticated of the result of identification made by the identification unit.

[0157] (Configuration 13) The authentication device described in configuration 11, characterized in that the display control means displays on the display device whether the person to be authenticated is an authorized person or an unauthorized person, and also displays a person ID identified as the ID of the person to be authenticated.

[0158] (Configuration 14) The authentication device described in configuration 11, wherein the display control means, if the similarity score obtained by the matching does not exceed a predetermined threshold, does not display the person ID, and displays a message on the display device indicating that the person is an unauthorized person.

[0159] (Configuration 15) The authentication device described in any one of configurations 1 to 14, characterized in having a feature vector management means for managing the person ID of the permitted person in association with the feature vector of the permitted person, and for managing the person ID of the unauthorized person in association with the feature vector of the unauthorized person.

[0160] (Configuration 16) An authentication device according to any one of configurations 1 to 15; and a control means for controlling the person to be authenticated from entering and leaving the predetermined area based on the identification result determined by the authentication device. 1. A management system comprising:

[0161] (Configuration 17) 17. The management system according to claim 16, further comprising a display device for displaying the result of the determination performed by the determination means.

[0162] (Configuration 18) The management system according to configuration 16 or 17, further comprising a notification means for notifying at least an administrator of the management system that the authenticated person is an unauthorized person when the identification means identifies the authenticated person as the unauthorized person.

[0163] (Configuration 19) A method for controlling an authentication device, comprising: an extraction step of extracting a feature vector of a person to be authenticated from an image captured of the person to be authenticated attempting to enter or exit a predetermined area; and a determination step of determining whether the person to be authenticated is the authorized person or the unauthorized person by comparing a feature vector of an authorized person who is authorized to enter the specified area and a feature vector of an unauthorized person who is not authorized to enter the specified area with the feature vector of the person to be authenticated extracted in the extraction step. 23. A method for controlling an authentication device comprising:

[0164] (Configuration 20) A program for causing a computer to operate as the authentication device according to any one of configurations 1 to 15. [Explanation of symbols]

[0165] 100 Entrance and Exit Management System 101 Face Recognition Device 102 Control section 103 Display section

Claims

1. a feature vector extraction means for extracting a feature vector of a person to be authenticated from an image captured of the person who is about to enter or exit a predetermined area; and a determination means for determining whether the person to be authenticated is the authorized person or the unauthorized person by comparing a feature vector of an authorized person who is authorized to enter the specified area and a feature vector of an unauthorized person who is not authorized to enter the specified area with the feature vector of the person to be authenticated extracted by the feature vector extraction means.

13. An authentication device comprising:

2. The authentication device according to claim 1, characterized in that the identification means compares a maximum value of the similarity score obtained by performing the matching with a predetermined threshold value, and identifies whether the person to be authenticated is the authorized person or the unauthorized person based on a result of the comparison.

3. When the maximum value of the similarity scores exceeds the predetermined threshold, the identification means further determines whether or not the maximum value of the similarity scores is a similarity score of a feature vector of the authorized person; 3. The authentication apparatus according to claim 2, wherein the specifying means specifies whether the person to be authenticated is the authorized person or the unauthorized person based on the result of the judgment.

4. The authentication device according to claim 1, characterized in that the identification means determines whether or not to compare the feature vector of the unauthorized person with the feature vector of the person to be authenticated extracted by the feature vector extraction means based on the result of the comparison with the feature vector of the authorized person.

5. The authentication device described in claim 1, characterized in that the identification means compares the similarity score for the authorized person obtained as a result of matching with the feature vector of the authorized person with a first threshold value, and if the similarity score for the authorized person does not exceed the first threshold value, compares the feature vector of the unauthorized person with the feature vector of the authenticated person extracted by the feature vector extraction means.

6. The authentication device according to claim 5, characterized in that the identification means compares the similarity score for the authorized person and the similarity score for the unauthorized person obtained by matching the similarity score for the authorized person and the feature vector of the unauthorized person with a second threshold different from the first threshold, and identifies whether the person to be authenticated is the authorized person or the unauthorized person based on the result of the comparison.

7. The authentication device according to claim 6 , wherein the first threshold value is greater than the second threshold value.

8. The authentication device according to claim 1, characterized in that the identification means requests the person to be authenticated to present ID information other than biometric information if the maximum value of the similarity score obtained by the matching does not exceed a predetermined threshold value.

9. 9. The authentication device according to claim 8, wherein the identification means identifies whether the person to be authenticated is the authorized person or the unauthorized person based on the presented ID information and the similarity score.

10. The authentication device according to claim 8, characterized in that, when the ID information is presented and the ID information is correct, the identification means identifies whether the person to be authenticated is the authorized person or the unauthorized person by comparing a feature vector of a facial image contained in the ID information with the feature vector of the person to be authenticated extracted by the feature vector extraction means.

11. 2. The authentication device according to claim 1, further comprising a display control means for displaying the result of the identification performed by the identification means on a display device.

12. 2. The authentication apparatus according to claim 1, further comprising a notification unit that notifies the person to be authenticated of the result of identification made by the identification unit.

13. 12. The authentication device according to claim 11, wherein the display control means causes the display device to display whether the person to be authenticated is an authorized person or an unauthorized person, and to display a person ID specified as the ID of the person to be authenticated.

14. The authentication device according to claim 11, characterized in that, when the similarity score obtained by the matching does not exceed a predetermined threshold, the display control means does not display the person ID, and instead displays a message on the display device indicating that the person is an unauthorized person.

15. The authentication device according to claim 1, further comprising a feature vector management means for managing the person ID of the authorized person in association with the feature vector of the authorized person, and for managing the person ID of the unauthorized person in association with the feature vector of the unauthorized person.

16. An authentication device according to any one of claims 1 to 15, and a control means for controlling the person to be authenticated from entering and leaving the predetermined area based on the identification result determined by the authentication device.

1. A management system comprising:

17. 17. The management system according to claim 16, further comprising a display device for displaying the result of the determination performed by the determination means.

18. 17. The management system according to claim 16, further comprising a notification means for, when the person to be authenticated is identified as the unauthorized person by the identification means, notifying at least an administrator of the management system that the person to be authenticated is the unauthorized person.

19. A method for controlling an authentication device, comprising: an extraction step of extracting a feature vector of a person to be authenticated from an image captured of the person to be authenticated attempting to enter or exit a predetermined area; and a determination step of determining whether the person to be authenticated is the authorized person or the unauthorized person by comparing a feature vector of an authorized person who is authorized to enter the specified area and a feature vector of an unauthorized person who is not authorized to enter the specified area with the feature vector of the person to be authenticated extracted in the extraction step.

23. A method for controlling an authentication device comprising:

20. A program for causing a computer to operate as the authentication device according to any one of claims 1 to 15.

Images