Authentication system, authentication device and authentication method
By using information code recording medium and a single shooting unit in the face authentication system, unconscious face authentication is achieved, and the problem that the existing system requires two shooting units is solved, the authentication efficiency and accuracy are improved, and user operations are simplified.
Patent Information
- Application Number
- CN202080009218.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Priority Date
- 2019-12-19
- Filing Date
- 2020-01-16
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2040-01-16
AI Technical Summary
The existing face authentication system requires two shooting units, which makes it difficult to set up and use, and may not be able to successfully capture the user's face, affecting the authentication efficiency and accuracy.
Design an authentication system to record facial feature points extracted from user's face images using information code recording media, and shoot user's face and information code through a single shooting unit to realize unconscious face authentication and simplify the shooting process. And the identity verification is performed through the determination unit, the extraction unit, the reading unit and the comparison unit.
It shortens the authentication time, improves the authentication accuracy, avoids users' need to shoot still, reduces redundant shooting and error recognition, and enhances the security and user experience of the system.
Smart Images

Figure CN113316781B_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to an authentication system, an authentication device and an authentication method using an information code. Background Art
[0002] In the past, there were cases where a system using face recognition was used as an authentication system requiring high security. Such a system using face recognition requires a large-scale system structure and is therefore installed in a place requiring high security. On the other hand, a qualification exam held at a university or a concert or event held in an auditorium requires a simple structure to achieve high security as an authentication system. As a system for achieving high security with such a simple structure, for example, the authentication system disclosed in the following patent document 1 is known.
[0003] In this authentication system, an authentication code recording information related to the user's facial feature points is generated in a server, and the authentication code is stored in the user's mobile terminal. Then, in the authentication device, when the authentication code displayed on the mobile terminal is photographed by the information code reading unit and the user's face is photographed by the photographing unit, the facial feature points read from the photographed authentication code are compared with the facial feature points extracted from the photographed user's facial image, and the user is authenticated based on the comparison result. In this way, the user can perform authentication using facial feature points simply by having the authentication device photograph the authentication code obtained in advance and his or her face.
[0004] Prior Art Literature
[0005] Patent Literature
[0006] Patent Document 1: Japanese Patent Application Publication No. 2014-222445 Summary of the invention
[0007] Technical Problems to be Solved by the Invention
[0008] However, in the above-mentioned Patent Document 1, the information code reading unit photographs the authentication code, and the photographing unit photographs the user's face, thereby obtaining the authentication information. This requires two photographing units, so there is a situation where the installation location of each photographing unit is limited. In addition, since there are two photographing units, there is a technical problem that the user may not be successfully photographed depending on the installation conditions of the authentication device having two photographing units.
[0009] The present invention is made to solve the above-mentioned technical problems, and its object is to provide a structure that can capture the face of a user without the user's awareness and realize authentication using facial feature points extracted from the facial image.
[0010] Solutions for solving technical problems
[0011] In order to achieve the above-mentioned object, an exemplary embodiment of a first mode of the present invention is an authentication system (10), comprising: an information code recording medium (30) carried by a user; and an authentication device (20) for authenticating the user; the authentication system is characterized in that:
[0012] The information code recording medium includes an information code (C) recording facial feature points extracted from the facial image of the user,
[0013] The authentication device comprises:
[0014] Photographing department (23);
[0015] A determination unit (21) determines whether the face is in a recognizable state when the face is photographed by the photographing unit in a recognizable manner;
[0016] An extraction unit (21) extracts facial feature points from a captured image obtained by capturing the face by the capturing unit after the face is determined to be in a recognizable state by the determination unit;
[0017] a reading unit (21) for reading information related to the facial feature points recorded in the information code from a captured image obtained by the capturing unit capturing the information code displayed on the information code recording medium;
[0018] a comparing unit (21) for comparing the facial feature points extracted by the extracting unit with the facial feature points read by the reading unit; and
[0019] An authentication unit (21) authenticates the user based on the comparison result of the comparison unit.
[0020] In addition, an exemplary embodiment of a second aspect of the present invention is an authentication device (20), wherein the authentication device (20) authenticates a user who presents an information code (C), wherein facial feature points extracted from a facial image of the user are recorded in the information code, and the authentication device (20) comprises:
[0021] Photographing department (23);
[0022] A determination unit (21) determines whether the face is in a face recognizable state in which the photographing unit photographs the face of a person in a recognizable manner;
[0023] An extraction unit (21) extracts facial feature points from a captured image obtained by capturing the face by the capturing unit after the judging unit judges that the face is in the recognizable state;
[0024] A reading unit (21) reads information related to the facial feature points recorded in the information code from a captured image obtained by the capturing unit capturing the information code;
[0025] a comparing unit (21) for comparing the facial feature points extracted by the extracting unit with the facial feature points read by the reading unit; and
[0026] An authentication unit (21) authenticates the user based on the comparison result of the comparison unit.
[0027] In addition, an exemplary embodiment of the third scheme of the present invention is an authentication method, which performs authentication of a user who presents an information code (C), characterized in that facial feature points extracted from a facial image of the user are recorded in the information code,
[0028] The authentication method comprises:
[0029] A photographing step, in which a photographing unit (23) generates a photographed image;
[0030] a determination step of determining whether the face is photographed in a recognizable manner in the photographing step in a face recognizable state;
[0031] an extraction step of extracting facial feature points from a captured image obtained by capturing the face by the capturing unit after the face is determined to be in a recognizable state in the determination step;
[0032] A reading step of reading information related to the facial feature points recorded in the information code from a captured image obtained by the capturing unit capturing the information code;
[0033] a comparing step of comparing the facial feature points extracted in the extracting step with the facial feature points read in the reading step; and
[0034] An authentication step is performed to authenticate the user based on the comparison result of the comparison step.
[0035] In addition, the reference numerals in the above-mentioned parentheses indicate the corresponding relationship with the specific units described in the embodiments to be described later.
[0036] Effects of the Invention
[0037] In an exemplary embodiment of the first mode of the present invention, the information code recording medium is provided with an information code recording facial feature points extracted from a facial image of a user. Furthermore, in the authentication device, the facial feature points are extracted by the extraction unit from a captured image obtained by the camera unit capturing the face after the determination unit determines that the face is in a recognizable state, and the information related to the facial feature points recorded in the information code is read by the reading unit from a captured image obtained by the camera unit capturing the information code displayed on the information code recording medium. Then, the comparison unit compares the facial feature points extracted by the extraction unit with the facial feature points read by the reading unit, and the authentication unit authenticates the user based on the comparison result.
[0038] Thus, the user can perform authentication using facial feature points simply by having the camera unit of the authentication device capture his or her own face and the information code displayed on the information code recording medium. In particular, one camera unit serves as both a camera unit for capturing the user's face and a camera unit for capturing the information code displayed on the information code recording medium, so the user can present the information code to the camera unit while looking at the camera unit. In this case, the camera unit can capture the face of the user who wants to present the information code in a recognizable manner without the user's awareness before capturing the information code. In addition, the user does not need to move still relative to the camera unit in order to capture the user's face. Therefore, the user's face can be captured and the process for extracting the facial feature points can be started before reading the information code, so the time from presenting the information code to the camera unit to obtaining the comparison result of the comparison unit is shortened, and the authentication time for identifying the user can be shortened.
[0039] For example, a display unit is provided that indicates a state in which the information code can be read in a predetermined display state, and the camera unit is arranged at a position within the field of vision of a user viewing the display unit. Thus, when the user wants to show the information code to the camera unit, the user visually confirms that the display unit has changed to the predetermined display state, and thus the camera unit within the field of vision of the user can capture the user's face in a state that is easier to identify, thereby improving the accuracy of authentication.
[0040] In addition, for example, the facial feature points extracted by the extraction unit are temporarily stored in the storage unit and deleted after a fixed time. Thus, for example, even if the information code cannot be read due to an incorrect presentation method of the information code recording medium, the extracted facial feature points are retained for a fixed time, so by correctly reading the information code within the fixed time, the time and labor of photographing the user's face again in a recognizable manner can be saved. On the other hand, the extracted facial feature points are no longer stored after a fixed time, so even if the face of another person different from the authentication object is accidentally photographed, it can be appropriately deleted.
[0041] Furthermore, for example, the control unit controls the lighting unit so that the illumination state when the camera unit captures a face under specified conditions is different from the illumination state when the camera unit captures an information code. Thus, for example, under specified conditions such as ambient brightness, when the same camera unit is used, it is possible to switch between a capture state suitable for capturing a face and a capture state suitable for capturing an information code, thereby improving the accuracy of extracting facial feature points or reading information codes, and as a result, it is possible to improve the authentication accuracy.
[0042] Furthermore, preferably, the determination unit determines that the face is in a recognizable state when the face captured in the continuous images captured by the camera unit gradually becomes larger in a recognizable manner. In the case where a third party causes the camera unit to capture a face photo that is displayed on the screen of the mobile terminal, even if it is recognized as a face, it will not be determined to be in a recognizable state as long as the face does not gradually become larger. Therefore, even if a face photo is captured for an improper purpose, it is difficult to mistakenly determine that the face is in a recognizable state, so impersonation can be prevented.
[0043] Furthermore, the determination unit may determine that the face is in the recognizable state when the inclination of the center line of the human face captured in a recognizable manner relative to the reference line in the image captured by the imaging unit is within a predetermined angle range.
[0044] When a third party causes the camera unit to take a photo of a face that is displayed on the screen of a mobile terminal, if the face photo is tilted, the center line of the face in the captured image is tilted relative to the vertical line (the line along the gravity direction). On the other hand, when photographing a person walking toward the camera unit, the inclination of the center line of the face in the captured image relative to the vertical line is sufficiently reduced. That is, in the case where the inclination of the center line of the face photographed in a recognizable state relative to the reference line such as the vertical line in the captured image of the camera unit is not within the prescribed angle range, it is possible that a face photo was taken. Therefore, in the case where the inclination of the center line of the face photographed in a recognizable manner is within the prescribed angle range, it is determined that the face is in a recognizable state, and in the case where the center line is not within the prescribed angle range, it is determined that the face is not in a recognizable state. Therefore, even if the face photo is taken for an improper purpose, when the face photo is greatly tilted, it is difficult to mistakenly determine that it is in a recognizable state, which can prevent impersonation.
[0045] Furthermore, preferably, the determination unit determines that the face is in a recognizable state when it can be recognized that the face photographed in the continuous images photographed by the photographing unit has blinked. In the case where a third party causes the photographing unit to take a face photo displayed on the screen of the mobile terminal, etc., it is recognized as a face but cannot blink, so it will not be determined that the face is in a recognizable state. Therefore, even if a face photo is taken for an improper purpose, it will not be erroneously determined that the face is in a recognizable state, so impersonation can be prevented.
[0046] In addition, when the range of the shadow applied to the face of a person captured in a recognizable manner changes in the continuous images captured by the camera unit, it can be determined that the face is in a recognizable state. In the case where a third party causes the camera unit to capture a face photo displayed on the screen of the mobile terminal, even if it is recognized as a face, it will not be determined that the face is in a recognizable state because the range of the shadow applied to the face has not changed. Therefore, even if a face photo is captured for an improper purpose, it will not be erroneously determined that the face is in a recognizable state, so impersonation can be prevented.
[0047] Furthermore, the determination unit may determine that the face is in a recognizable state when the depth is recognized for the face photographed by the camera unit in a recognizable manner. In the case where a third party causes the camera unit to take a face photo displayed on the screen of the mobile terminal, even if it is recognized as a face, it will not be determined that the face is in a recognizable state because the depth is not recognized for the face. Therefore, even if a face photo is taken for an improper purpose, it will not be mistakenly determined that the face is in a recognizable state, so impersonation can be prevented.
[0048] In the exemplary embodiment of the second aspect of the present invention, it is possible to realize an authentication device having the same effect as the first aspect.
[0049] In addition, in the exemplary embodiment of the third aspect of the present invention, it is possible to implement an authentication method having the same effect as the first aspect. BRIEF DESCRIPTION OF THE DRAWINGS
[0050] Figure 1A This is an explanatory diagram schematically showing the configuration of the authentication system according to the first embodiment.
[0051] Figure 1B This is an explanatory diagram schematically showing a method of using the authentication system according to the first embodiment.
[0052] Figure 2 This is a diagram for explaining a part of an example of extracting positions of facial feature points from captured facial image data.
[0053] Figure 3This is a block diagram schematically showing the electrical structure of the authentication device.
[0054] Figure 4 It is a block diagram schematically showing the electrical structure of a mobile terminal carried by a user.
[0055] Figure 5 This is a flowchart illustrating the flow of authentication processing by the authentication device.
[0056] Figure 6 This is an explanatory diagram for explaining a state in which a facial photo displayed on a screen of a mobile terminal is photographed by a photographing unit of an authentication device.
[0057] Figure 7 This is an explanatory diagram for explaining a state in which a facial photo displayed on a screen of a mobile terminal is tilted and photographed by a photographing unit of an authentication device.
[0058] Figure 8 This is an explanatory diagram for explaining a state in which a facial photograph in which a shadow is cast on a part of the user's face is displayed on the screen. DETAILED DESCRIPTION
[0059] Hereinafter, an embodiment of an authentication system, an authentication device, and an authentication method realizing the present invention will be described with reference to the accompanying drawings.
[0060] Figure 1A as well as Figure 1B The authentication system 10 shown mainly includes an authentication device 20, and is configured to perform personal authentication of a user using an information code (hereinafter also referred to as an authentication code C) recording facial feature points extracted from a facial image of the user or personal information, etc. The authentication system 10 also includes a mobile terminal 30 as an information code recording medium carried by the user as a system in a broad sense.
[0061] also, Figure 1A shows the hardware structure of the authentication system 10, Figure 1B A case where a user performs authentication using a mobile terminal, which will be described later, is described.
[0062] In this embodiment, the authentication system 10 is used, for example, for an access management system that allows a user who has successfully undergone facial authentication to enter a specific room. The user carries a mobile terminal 30 as an information code recording medium having an authentication code C, and causes the authentication device 20 provided at the entrance to the specific room to capture the authentication code C displayed on the screen of the mobile terminal 30, and is allowed to enter the specific room when the authentication is successful. Specifically, authentication success information is sent from the authentication device 20 to the opening and closing lock control device 40 that controls the opening and closing of the electronic lock provided at the door of the entrance, thereby causing the opening and closing lock control device 40 to change the electronic lock to an unlocked state, thereby allowing entry to the specific room.
[0063] The authentication code C is generated by a predetermined code generating device (not shown), which obtains facial image data from a user who applies for personal authentication. The code generating device extracts facial feature points from the facial image data obtained from the user, and encodes information related to the facial feature points or personal information to generate the authentication code C. In the present embodiment, items such as the size or shape of each facial organ such as the eyes, eyebrows, nose, ears, and mouth, and the contours or the relative positions of the facial organs are used as facial feature points. Then, according to the well-known facial feature point extraction process for extracting facial feature points implemented by the code generating device, for example, for a facial feature point such as Figure 2 For the facial image data D obtained as an example, features for each item, for example, data such as “D1: narrow eyes” or “D2: big mouth” are calculated and extracted as facial feature points.
[0064] Then, according to the known information code generation process performed by the code generation device, based on the information related to the facial feature points extracted as described above or the personal information, etc., the authentication code C is encrypted in a decodable manner using a decoding key (encryption key) to generate an authentication code C. In the present embodiment, as the authentication code C, for example, a QR code (registered trademark) whose security is enhanced by encryption is used.
[0065] Next, use Figure 1A as well as Figure 1B as well as Figure 3 , the authentication device 20 is described.
[0066] The authentication device 20 is configured to authenticate the user using the authentication code C displayed on the screen of the mobile terminal 30, and is installed at a position at the user's eye level on the wall near the entrance and exit of the specific room, specifically, the door installed at the entrance and exit (see Figure 1B ).like Figure 3As shown, the authentication device 20 includes a control unit 21, a storage unit 22, a camera unit 23, a display unit 24, a lighting unit 25, an operation unit 26, and a communication unit 27. The control unit 21, the storage unit 22, and the communication unit 27 constitute a processor 20A.
[0067] The control unit 21 is mainly composed of a computer, and is configured to perform authentication processing, various operations, and overall system control, which will be described later. The computer includes a CPU (central processing unit) 21A serving as a computing center and a main memory 21B serving as a work area. The storage unit 22 is configured to include a known storage medium such as a HDD (hard disk drive) not shown in the figure and a non-volatile memory as needed in addition to a ROM (read-only memory (EEPROM, etc.)) 22A and a RAM (random access memory) 22B, and is pre-stored with an application program or a predetermined database for executing the authentication processing in a manner that can be used by the control unit 21 (i.e., CPU 21A). In addition, a decryption key for decrypting the authentication code C is pre-stored in the storage unit 22.
[0068] In the present embodiment, ROM 22A functions as a non-transitory computer-readable recording medium and stores the above-mentioned application program and other control / processing program steps as source code. The non-transitory computer-readable recording medium may also be composed of a type of RAM that does not delete stored information.
[0069] The program is read by the CPU 21A into the main memory 21B, which is a preset work area, and executed. The RAM 22B is configured to temporarily store data being processed by the CPU 21A. The main memory 21B is configured by a RAM.
[0070] Of course, the structure of the processor 20A is an example, and other structures may be used as long as the structure can execute the programs related to the required authentication or control. For example, multiple CPUs may be provided for distributed control, or a redundant system may be formed. CPU 21A is an element that serves as the computing center of the computer system, and as long as it has the same function, it may be called differently (e.g., computing device, processor, microprocessor).
[0071] The imaging unit 23 is a camera having a light receiving sensor (e.g., a C-MOS (complementary metal oxide semiconductor) area sensor, a CCD (charge coupled device) area sensor, etc.), and is configured to output image data captured through a reading window (imaging window) to the control unit 21. Based on the facial image data captured by the imaging unit 23, the control unit 21, which functions as an extraction unit, performs facial feature point extraction processing in the same manner as the above-mentioned code generation device, thereby extracting facial feature points. In addition, based on the authentication code C captured by the imaging unit 23, the control unit 21, which functions as a reading unit, performs information code reading processing (decoding processing) in a manner of decoding the encrypted authentication code C using a decryption key stored in the storage unit 22, thereby reading facial feature points recorded in the authentication code C.
[0072] The display unit 24 is, for example, an LED. Figure 1A As shown, the image capturing unit 23 is arranged in a substantially square ring shape so as to surround the reading window of the image capturing unit 23, and is configured to be controlled by the control unit 21 so as to change its lighting / flashing state according to whether the extraction of facial feature points is successful or the result of the authentication process, etc. Due to such an arrangement structure, the image capturing unit 23 is arranged at a position within the field of vision of the user viewing the display unit 24.
[0073] The lighting unit 25 includes a lighting light source or a lighting lens, etc., and is configured to be controlled by the control unit 21 so as to irradiate lighting light toward the imaging range of the imaging unit 23. The operation unit 26 is configured to output an operation signal corresponding to the input operation to the control unit 21, and the control unit 21 receives the operation signal and performs processing corresponding to the input operation. The communication unit 27 is configured as a communication interface for performing data communication with the above-mentioned opening and closing lock control device 40 or an external device 41 such as a server for access management, and is configured to cooperate with the control unit 21 to perform communication processing.
[0074] Next, use Figure 1A as well as Figure 4 The mobile terminal 30 will be described.
[0075] Figure 1A as well as Figure 4 The mobile terminal 30 shown is, for example, a smart phone carried by a user, and comprises: a control unit 31 composed of a CPU 31A, etc., a memory 32, a shooting unit 33 configured as a camera, a touch panel 34 whose display content is controlled by the control unit 31, an operation unit 35 that outputs a signal to the control unit 31 when the user touches the touch panel 34 or operates a key, and a communication unit 36 for communicating with external devices, etc.
[0076] The memory 32 of the mobile terminal 30 thus configured stores the authentication code C generated by the code generating device in advance. Then, the control unit 31 (CPU 31A) causes the authentication code C read from the memory 32 to be displayed on the touch panel 34 in response to a predetermined operation of the operation unit 35 by the user (see FIG. 1 ). Figure 1A ).
[0077] Next, as an authentication method, the authentication process performed by the authentication device 20 is referred to as Figure 5 The flowchart shown is explained in detail.
[0078] In the authentication device 20, when the control unit 21 starts the authentication process, the Figure 5 The photographing process shown in step S101 is performed. In this process, photographing by the photographing unit 23 is performed in a state where the illumination light is irradiated by the illumination unit 25 through the reading window. Next, in the determination process of step S103, it is determined whether the photographing unit 23 is in a state where a face is photographed in a recognizable manner (hereinafter also referred to as a face recognizable state). When a face is not photographed and it is determined that the face is not in a recognizable state (S103 is "No"), photographing by the photographing unit 23 is continued. In addition, the above-mentioned photographing process may be equivalent to an example of a "photographing step" in which the photographing unit 23 generates a photographed image. In addition, the determination process of the above-mentioned step S103 may be equivalent to an example of a "determination step", and the control unit 21 that executes the determination process may be equivalent to an example of a "determination unit".
[0079] The user who wants to enter the specific room wants to present the authentication code C displayed on the screen of the mobile terminal 30 to the reading window of the authentication device 20 (see Figure 1B ). At this time, when the face of the user viewing the reading window is photographed by the photographing unit 23 and the face is in a recognizable state, it is determined as "yes" in step S103, and the facial image data generation process shown in step S105 is executed. In this process, the photographing unit 23 re-photographs an image for extracting facial feature points, and generates facial image data. In addition, in the facial image data generation process, multiple images for extracting facial feature points can be re-photographed, and one image data from which facial feature points can be easily extracted can be set as facial image data.
[0080] Next, the facial feature point extraction process shown in step S107 is executed to extract facial feature points from the facial image data generated as described above, similarly to the above code generation device, and temporarily store them in the storage unit 22. The above facial feature point extraction process may correspond to an example of the "extraction step".
[0081] When the facial feature points are extracted in this way, the extraction of facial feature points is completed, indicating that the authentication code C can be read, so the display unit 24 changes to the first lighting state (for example, the blue lighting state) (S109). Next, the information code reading process shown in step S111 is performed. In this process, the decoding key stored in the storage unit 22 is used to perform a decoding process for reading the information code photographed by the imaging unit 23. In addition, the above-mentioned information code reading process can be equivalent to an example of the "reading step".
[0082] Next, in the determination process of step S113, it is determined whether the reading of the authentication code C is successful. Here, if the authentication code C is not photographed or an information code different from the authentication code C is read, it is determined as "No" in step S113, and if a fixed time has not passed after the first lighting state (S115 is "No"), the information code reading process of the above step S111 is executed, and the state in which the information code can be read is maintained.
[0083] Then, when the fixed time has not passed since the first lighting state (S115 is "No"), the user whose facial image is captured as described above presents the authentication code C displayed on the screen of the mobile terminal 30 to the reading window of the authentication device 20, so that the authentication code C is captured, and the reading of information related to facial feature points or personal information is successful (S113 is "Yes"), the comparison process shown in step S117 is performed. In this process, based on the facial feature points extracted from the facial image data and the facial feature points read from the authentication code C, all items in the facial feature points are compared to see whether they match. In addition, the above-mentioned comparison process can be equivalent to an example of a "comparison step", and the control unit 21 that performs the comparison process can be equivalent to an example of a "comparison unit".
[0084] Then, when all items in the facial feature points match, it is determined as "yes" in the determination process of step S119, indicating that the authentication is successful, and the display unit 24 changes to the second lighting state for a fixed time (for example, green lighting state), (S123). Next, the sending process shown in step S125 is performed, and the authentication success information indicating that the authentication is successful is sent to the lock and unlock control device 40 through the communication unit 27, and the authentication success information or the read personal information is sent to the server 41 (external device) for access management through the communication unit 27. Then, the process is performed from the above step S101, and it becomes a state in which the face of the next user who is about to enter the room can be photographed. In addition, in the determination process of the above step S119, it is not limited to being determined as "yes" when all items in the facial feature points match, and it can also be determined as "yes" when a specified number of items in the facial feature points match. In addition, the determination process of the above step S119 can be equivalent to an example of an "authentication step", and the control unit 21 that performs the determination process can be equivalent to an example of an "authentication unit".
[0085] As described above, the authentication success information is transmitted from the authentication device 20, and the unlocking / locking control device 40 which has received the authentication success information turns the electronic lock into the unlocked state, thereby enabling the user to enter the specific room.
[0086] On the other hand, when a third party presents the authentication code C displayed on the screen of another person's mobile terminal 30 to the reading window of the authentication device 20, the items in the facial feature points do not match, so the judgment in step S119 is "No", indicating that the authentication has failed, so the display unit 24 changes to the third lighting state for a fixed time (for example, the red lighting state) (S127). In this case, the authentication success information is not sent from the authentication device 20, so the lock control device 40 does not change the electronic lock to the unlocked state. In addition, in this case, the authentication failure information or the read personal information can also be sent to the server 41 for access management through the communication unit 27.
[0087] When a predetermined time has passed since the first lighting state without the authentication code C being captured (YES in S115), the facial feature points extracted and stored as described above are deleted from the storage unit 22, and the process is executed again from step S101.
[0088] As described above, in the authentication system 10 of the present embodiment, in the mobile terminal 30, the authentication code C, in which the facial feature points extracted from the facial image of the user are recorded, is stored in the memory 32 in a manner that can be displayed on the screen. Then, in the authentication device 20, the facial feature points are extracted from the captured image obtained by capturing the face of the person by the camera 23 after the face is determined to be in a recognizable state, and the information related to the facial feature points recorded in the authentication code C is read from the captured image captured by the camera 23 and displayed on the screen of the mobile terminal 30. Then, the extracted facial feature points are compared with the read facial feature points, and based on the comparison result, the user is authenticated (that is, it is confirmed that the user who carries the mobile terminal 30 and shows the display screen of the mobile terminal 30 to the authentication device 20 is a genuine user and is allowed to enter the room).
[0089] Thus, the user can perform authentication using facial feature points by only causing the camera 23 of the authentication device 20 to capture his / her own face and the authentication code C displayed on the screen of the mobile terminal 30. In particular, since the single camera 23 serves as both a camera unit for capturing the user's face and a camera unit for capturing the authentication code C displayed on the screen of the mobile terminal 30, the user can perform an action of presenting the authentication code C to the reading window while looking at the reading window of the camera 23. In this case, the face of the user who is to present the authentication code C can be captured by the camera 23 in a recognizable manner without the user's awareness before capturing the authentication code C. In addition, there is no need for the user to perform an action of standing still relative to the camera 23 in order to capture the user's face. Therefore, the user's face can be captured and the process of extracting the facial feature points can be started before reading the authentication code C, so that the time from presenting the authentication code C to the camera 23 to obtaining the comparison result of the comparison process is shortened, and the authentication time for identifying the user can be shortened.
[0090] In particular, the facial feature points extracted by the facial feature point extraction process are temporarily stored in the storage unit 22 until a fixed time has passed after the first lighting state (until it is determined as "yes" in S115), and are deleted after the fixed time has passed. Thus, for example, even if the authentication code C displayed on the screen of the mobile terminal 30 is not presented in the correct manner so that the authentication code C cannot be read, the extracted facial feature points are kept for a fixed time, so by properly reading the authentication code C within the fixed time, the time and labor of photographing the user's face again in a recognizable manner can be saved. On the other hand, the extracted facial feature points are deleted and no longer stored after a fixed time has passed (S115 is "yes"), so even if the face of another person different from the authentication object is accidentally photographed, it can be appropriately deleted.
[0091] As a first variation of the present embodiment, the display unit 24 may be controlled by the control unit 21, and in the process of repeating the determination of "No" in the above step S103, that is, before the face recognition state is reached, it indicates that the authentication code C can be read, and thus becomes a prescribed display state (for example, a blue flashing state) different from the first to third lighting states. Therefore, the display unit 24 that indicates the state in which the authentication code C can be read in the above prescribed display state is provided, and the imaging unit 23 is arranged at a position within the field of vision of the user viewing the display unit 24.
[0092] Thus, when the user presents the authentication code C to the camera unit 23, the visual confirmation display unit 24 is in the above-mentioned specified display state, and the user's face can be photographed in a more easily recognizable state by using the camera unit 23 within the user's field of view, which can improve the confirmation accuracy. In addition, the first lighting state to the third lighting state or the above-mentioned specified display state is an example, and color changes or flashing at a specified interval, changes in the lighting / extinguishing position, etc. can also be used. In addition, the intention can be notified by sound, etc. when the first lighting state is changed to the third lighting state or the above-mentioned specified display state.
[0093] As a second variation of the present embodiment, the control unit 21 may control the lighting unit 25 in such a manner that the lighting state when the camera unit 23 captures a human face under a predetermined condition is different from the lighting state when the camera unit 23 captures the authentication code C. For example, on the premise that the lighting unit 25 is configured to be able to irradiate infrared light in addition to visible light, infrared light may be irradiated when capturing a human face, and visible light adjusted to suppress brightness may be irradiated when capturing the authentication code C under a predetermined condition such as at night when the surroundings are dim. This is because when bright visible light is irradiated in a dim surrounding state, the user may feel dizzy, and when the authentication code C is captured using the infrared light, depending on the type of information code recording medium displaying the authentication code C, the authentication code C may not be captured in a readable manner.
[0094] In this way, under specified conditions such as the surrounding brightness, even when using the same shooting unit 23, it is possible to switch between a shooting state (irradiation state) suitable for shooting a human face and a shooting state (irradiation state) suitable for shooting the authentication code C, thereby improving the extraction accuracy of facial feature points or the reading accuracy of the authentication code C, and as a result, the authentication accuracy can be improved.
[0095] In addition, in order to further improve the confidentiality of information related to facial feature points, a partial non-public code can be used as the above-mentioned authentication code C. The partial non-public code is constructed as an information code having a public area and a non-public area, and the non-public area records information that is encrypted in a manner that can be decrypted using a decoding key. Therefore, by recording information related to facial feature points in the non-public area, a code reader that does not have a decoding key can only read the information recorded in the public area, and even cannot recognize that other information is recorded, thereby improving the confidentiality of information related to facial feature points. On the other hand, when the present authentication system 10 is used in an environment with low security requirements, the authentication code C can be generated without encryption based on the information related to the facial feature points extracted as described above.
[0096] [Second embodiment]
[0097] Next, an authentication system, an authentication device, and an authentication method according to a second embodiment will be described with reference to the drawings.
[0098] The second embodiment is different from the first embodiment in that the determination condition for determining a face recognizable state is made stricter. Therefore, the same reference numerals are given to components substantially the same as those in the first embodiment, and their description is omitted.
[0099] In the determination process of step S103 in the above authentication process (processing performed by the control unit 21 functioning as a determination unit), it is determined whether the state of the face being photographed in a recognizable manner by the photographing unit 23 (face recognizable state). Therefore, in the case where a third party causes the photographing unit 23 to photograph a face photo displayed on the screen of the mobile terminal for improper purposes, it may be erroneously determined that the face is in the recognizable state. In particular, in a system that reads an information code such as an authentication code C displayed on the screen of the mobile terminal and performs authentication as in the present invention, it is easy to assume that the face photo is displayed on the screen of the mobile terminal and is used improperly.
[0100] Therefore, in this embodiment, by making the determination conditions for determining that the face is in a recognizable state more stringent, even if the camera 23 takes a facial photo for improper purposes, it is difficult to mistakenly determine that the face is in a recognizable state, thereby suppressing the act of impersonating a legitimate authorized person (a permitted user) and performing acts that are permitted only to legitimate authorized persons.
[0101] Specifically, in the determination process of step S103 , when the size of a human face captured in a recognizable manner gradually increases in the continuous images captured by the imaging unit 23 , it is determined that the face is in the recognizable state.
[0102] Therefore, when the authorized person enters the specific room and approaches the camera 23, the authorized person's face is recognized and then captured in a gradually larger size, and thus it is determined that the face is in a recognizable state. Figure 6 As shown in the example, when a third party uses the camera 23 to capture a facial photo P displayed on the screen of the mobile terminal, even if it is recognized as a human face, it will not be determined to be in a face recognizable state unless the face gradually becomes larger.
[0103] Thus, in the present embodiment, in the continuous images captured by the shooting unit 23, when the human face captured in a recognizable manner gradually becomes larger, it is determined that the face is in a recognizable state. Thus, even if a face photo is taken for an improper purpose, it is difficult to mistakenly determine that the face is in a recognizable state, thereby preventing impersonation.
[0104] In addition, as a first variation of the present embodiment, in the determination process of the above-mentioned step S103, when the inclination of the center line Lf of the human face captured in a recognizable manner relative to the baseline Lo in the captured image of the capturing unit 23 is within a prescribed angle range, it can be determined that the face is in a recognizable state.
[0105] In the first modification, the reference line Lo is set to a vertical line (a line along the direction of gravity). When a third party causes the camera unit 23 to take a face photo displayed on the mobile terminal, for example, Figure 7 As shown in the example, when the face photo is tilted, the center line Lf of the face in the captured image is tilted relative to the vertical line serving as the reference line Lo. On the other hand, when a person walking toward the imaging unit 23 is captured, the inclination of the center line Lf of the face in the captured image relative to the reference line Lo becomes sufficiently small. That is, when the inclination θ of the center line Lf relative to the reference line Lo in the captured image of the imaging unit 23 is not within the prescribed angle range θth (e.g., 30°), there is a possibility that the face photo is being captured.
[0106] Therefore, in the first modified example, when the inclination θ of the center line Lf of the human face photographed in a recognizable manner relative to the reference line Lo is within the prescribed angle range θth, it is determined that the face is in a recognizable state, and when the inclination θ of the center line Lf relative to the reference line Lo is not within the prescribed angle range θth, it is determined that the face is not in a recognizable state. Thus, even if a face photo is taken for an improper purpose, it is difficult to mistakenly determine that the face is in a recognizable state when the face photo is greatly tilted, so impersonation can be prevented.
[0107] In addition, as a second variation of the present embodiment, in the determination process of the above step S103, when it can be identified that the face photographed in a recognizable manner has blinked in the continuous images photographed by the photographing unit 23, it can be determined that the face is in a recognizable state. In the case where a third party causes the photographing unit to photograph a face photo displayed on the screen of the mobile terminal, even if it is recognized as a human face, it will not be determined that the face is in a recognizable state because it cannot blink. Therefore, even if a face photo is taken for an improper purpose, it will not be erroneously determined that the face is in a recognizable state, so impersonation can be prevented.
[0108] In addition, as a third variation of the present embodiment, in the determination process of the above-mentioned step S103, when the range of the shadow applied to the face captured in a recognizable manner in the continuous images captured by the imaging unit 23 changes, it can be determined that the face is in a recognizable state.
[0109] For example, Figure 8 In the example shown, in a facial photograph P in which a person's face is photographed, a portion of the face is shadowed due to the surrounding brightness at the time of the photograph (see FIG. Figure 8 Then, when the person who is the subject of the photograph moves, the way the light is irradiated to the face changes, so the range of the shadow changes. On the other hand, even if a shadow is cast on the face in the face photo P, the range of the shadow does not change. That is, if the range of the shadow cast on the face does not change in the continuous images, it is very likely that the photo is being taken.
[0110] Therefore, when the range of the shadow applied to the face of a person photographed in a recognizable manner in the continuous images photographed by the photographing unit 23 changes, it is determined that the face is in a recognizable state. Therefore, when the photographing unit 23 takes a face photo, even if it is recognized as a face, it will not be determined that the face is in a recognizable state because the range of the shadow applied to the face does not change. Therefore, even if a face photo is taken for an improper purpose, it will not be erroneously determined that the face is in a recognizable state, so impersonation can be prevented.
[0111] In addition, as a fourth variation of the present embodiment, in the determination process of the above step S103, when the depth is recognized for the face photographed by the camera unit 23 in a recognizable manner, it can be determined that the face is in a recognizable state. In the case where a third party causes the camera unit to take a face photo displayed on the screen of the mobile terminal, even if it is recognized as a face, it will not be determined that the face is in a recognizable state because the depth is not recognized for the face. Therefore, even if a face photo is taken for an improper purpose, it will not be erroneously determined that the face is in a recognizable state, so impersonation can be prevented.
[0112] In addition, the present invention is not limited to the above-mentioned embodiments and modifications, and can be implemented as follows, for example.
[0113] (1) The authentication code C is not limited to being stored in the memory 32 in a manner that can be displayed on the touch panel 34 of the mobile terminal 30, and may be provided by being printed on one surface of an information code recording medium such as a paper medium in a photographable manner.
[0114] (2) The authentication system 10 of this embodiment is not limited to an access management system for allowing entry to a specific room, and can also be used, for example, as a management system for managing the entrance to a concert venue, or as a system used when a store or the like provides services that require authentication.
[0115] (3) In the authentication process executed by the control unit 21 of the authentication device 20, the process for reading the authentication code C is not limited to starting after the facial feature points are extracted, and the authentication code C may be read before the face is in a recognizable state. That is, in the case where the authentication code C is read before the face is in a recognizable state, the facial feature points may be extracted from the facial image data captured after the face is in a recognizable state, and a comparison process may be performed based on the extracted facial feature points and the facial feature points previously read from the authentication code C.
[0116] (4) In the authentication process performed by the control unit 21 of the authentication device 20, it is not limited to the comparison process based on the facial feature points extracted from the captured facial image data and the facial feature points read from the authentication code C as described above. The captured facial image data or the extracted facial feature points and the captured image data of the authentication code C or the read facial feature points may be sent to a server, etc., thereby performing a comparison process on the server, etc. to obtain the comparison result, etc.
[0117] Description of Reference Numerals
[0118] 10: Authentication system
[0119] 20: Authentication device
[0120] 21: Control unit (determination unit, extraction unit, reading unit, comparison unit, authentication unit)
[0121] 22: Storage
[0122] 23: Photography Department
[0123] 24: Display unit
[0124] 25: Lighting Department
[0125] 30: Mobile terminal (information code recording medium)
[0126] C: Authentication code (information code)
Claims
1. An authentication system comprising: Information code recording medium carried by the user; and an authentication device for authenticating the user; The authentication system is characterized in that The information code recording medium has information codes recording facial feature points extracted from the user's facial image. The authentication device comprises: a photographing unit for photographing the information code and the user's face when the user presents the information code displayed on the display unit of the information code recording medium for photographing, wherein one photographing unit serves as both a photographing unit for photographing the user's face and a photographing unit for photographing the information code displayed on the information code recording medium; a determination unit for determining whether the user's face is in a recognizable state as captured by the capturing unit in a recognizable manner; an extraction unit that extracts facial feature points of the user from the captured image of the user captured by the capturing unit after the determination unit determines that the face is in the recognizable state; a reading unit that reads information related to the facial feature points recorded in the information code from a captured image obtained by the capturing unit capturing the information code displayed on the information code recording medium; a comparing unit, configured to compare the facial feature points extracted by the extracting unit with the facial feature points read by the reading unit; an authentication unit, authenticating the user based on the comparison result of the comparison unit, an illumination unit configured to irradiate illumination light toward a photographing range of the photographing unit; as well as a control unit that controls an irradiation state of the illumination light in the illumination unit, The control unit controls the lighting unit so that the illumination state when the imaging unit captures the user's face is different from the illumination state when the imaging unit captures the information code under predetermined conditions. The authentication system is configured such that the extraction unit starts a process of extracting the facial feature points from the captured image before the reading unit reads the information related to the facial feature points.
2. The authentication system according to claim 1, wherein: A display unit is provided, wherein the display unit indicates a state in which the information code can be read in a predetermined display state, The imaging unit is arranged at a position within the field of view of the user viewing the display unit.
3. The authentication system according to claim 1 or 2, characterized in that: A storage unit is provided in which the facial feature points extracted by the extraction unit are temporarily stored and deleted after a fixed time has elapsed.
4. The authentication system according to any one of claims 1 to 3, characterized in that: The determination unit determines that the face is in the face recognizable state when the user's face captured in a recognizable manner gradually becomes larger in the continuous images captured by the imaging unit.
5. The authentication system according to any one of claims 1 to 3, characterized in that: The determination unit determines that the face is in the face recognizable state when an inclination of a center line of a face of a user captured in a recognizable manner relative to a reference line in an image captured by the imaging unit is within a predetermined angle range.
6. The authentication system according to any one of claims 1 to 3, characterized in that: The determination unit determines that the face is in the face recognizable state when it is recognizable that the face of the user captured in a recognizable manner has blinked in the continuous images captured by the imaging unit.
7. The authentication system according to any one of claims 1 to 3, characterized in that: The determination unit determines that the face is in the face recognizable state when a range of a shadow cast on the face of the user captured in a recognizable manner in the continuous images captured by the imaging unit changes.
8. The authentication system according to any one of claims 1 to 3, characterized in that: The determination unit determines that the face is in the face recognizable state when a depth is recognized with respect to the face of the user captured in a recognizable manner by the imaging unit.
9. An authentication device for authenticating a user who presents an information code, characterized in that: The information code records the facial feature points extracted from the user's facial image. The authentication device comprises: a photographing unit for photographing the information code and the user's face when the user presents the information code displayed on the display unit of the information code recording medium for photographing, wherein one photographing unit serves as both a photographing unit for photographing the user's face and a photographing unit for photographing the information code displayed on the information code recording medium; a determination unit for determining whether the user's face is in a recognizable state as captured by the capturing unit in a recognizable manner; an extraction unit that extracts facial feature points of the user from the captured image of the user captured by the capturing unit after the determination unit determines that the face is in the recognizable state; a reading unit that reads information related to the facial feature points recorded in the information code from a captured image obtained by the capturing unit capturing the information code; a comparing unit, configured to compare the facial feature points extracted by the extracting unit with the facial feature points read by the reading unit; an authentication unit, configured to authenticate the user based on the comparison result of the comparison unit; an illumination unit configured to irradiate illumination light toward a photographing range of the photographing unit; as well as a control unit that controls an irradiation state of the illumination light in the illumination unit, The control unit controls the lighting unit so that the illumination state when the imaging unit captures the user's face is different from the illumination state when the imaging unit captures the information code under predetermined conditions. The authentication device is configured such that the extraction unit starts a process of extracting the facial feature points from the captured image before the reading unit reads the information related to the facial feature points.
10. The authentication device according to claim 9, characterized in that: A display unit is provided, wherein the display unit indicates a state in which the information code can be read in a predetermined display state, The imaging unit is arranged at a position within the field of view of the user viewing the display unit.
11. The authentication device according to claim 9 or 10, characterized in that: A storage unit is provided in which the facial feature points extracted by the extraction unit are temporarily stored and deleted after a fixed time has elapsed.
12. The authentication device according to any one of claims 9 to 11, characterized in that: The determination unit determines that the face is in the face recognizable state when the user's face captured in a recognizable manner gradually becomes larger in the continuous images captured by the imaging unit.
13. The authentication device according to any one of claims 9 to 11, characterized in that: The determination unit determines that the face is in the face recognizable state when an inclination of a center line of a face of a user captured in a recognizable manner relative to a reference line in an image captured by the imaging unit is within a predetermined angle range.
14. The authentication device according to any one of claims 9 to 11, characterized in that: The determination unit determines that the face is in the face recognizable state when it is recognizable that the face of the user captured in a recognizable manner has blinked in the continuous images captured by the imaging unit.
15. The authentication device according to any one of claims 9 to 11, characterized in that: The determination unit determines that the face is in the face recognizable state when a range of a shadow cast on the face of the user captured in a recognizable manner in the continuous images captured by the imaging unit changes.
16. The authentication device according to any one of claims 9 to 11, characterized in that: The determination unit determines that the face is in the face recognizable state when a depth is recognized with respect to the face of the user captured in a recognizable manner by the imaging unit.
17. An authentication method for authenticating a user who presents an information code recorded in an information code recording medium, characterized in that: The information code records the facial feature points extracted from the user's facial image. The authentication method comprises: a photographing step, wherein a photographic image is generated by a photographing unit, wherein the photographing unit photographs the information code and the user's face when the user presents the information code displayed on the display unit of the information code recording medium for photographing, wherein one photographing unit serves as both a photographing unit for photographing the user's face and a photographing unit for photographing the information code displayed on the information code recording medium; a determination step of determining whether the user's face is photographed in a recognizable manner in the photographing step; an extraction step of extracting facial feature points of the user from an image of the user captured by the imaging unit after the face is determined to be in the recognizable state in the determination step; a reading step of reading information related to the facial feature points recorded in the information code from a captured image obtained by the capturing unit capturing the information code; a comparing step of comparing the facial feature points extracted in the extracting step with the facial feature points read in the reading step; as well as an authentication step of authenticating the user based on the comparison result of the comparison step, The authentication method is controlled so that, under predetermined conditions, the illumination state when the user's face is photographed by the photographing step is different from the illumination state when the information code is photographed by the photographing step. The extraction step starts a process of extracting the facial feature points from the captured image before the reading step reads the information related to the facial feature points.
18. A non-transitory computer-readable recording medium having a program pre-recorded in a computer-readable manner, wherein the program performs user authentication when an information code recording medium containing facial feature points extracted from a facial image of the user is presented, wherein: By executing the program read from the recording medium by the computer, the following steps are functionally provided: a step of determining whether the face of the user is in a recognizable state by being photographed in a recognizable manner by a photographing unit, the photographing unit photographing the information code and the user's face when the user presents the information code displayed on the display unit of the information code recording medium for photographing, the one photographing unit serving as both a photographing unit for photographing the user's face and a photographing unit for photographing the information code displayed on the information code recording medium; an extraction step of extracting facial feature points of the user from an image of the user captured by the imaging unit after the face is determined to be in the recognizable state in the determination step; a reading step of reading information related to the facial feature points recorded in the information code from a captured image obtained by the capturing unit capturing the information code; a comparing step of comparing the facial feature points extracted in the extracting step with the facial feature points read in the reading step; and an authentication step of authenticating the user based on the comparison result of the comparison step, Control is performed so that the illumination state when the imaging unit captures the user's face is different from the illumination state when the imaging unit captures the information code under predetermined conditions, The extraction step starts a process of extracting the facial feature points from the captured image before the reading step reads the information related to the facial feature points.
