Identity verification device, identity verification method, and computer program

Abstract

To efficiently perform identity verification using an identity verification document with sufficient reliability.SOLUTION: An identity verification device, which determines whether or not a target person is a document person indicated in an identity verification document, photographs a face of the target person to generate a target person face image, photographs a face of the identity verification document to generate a document photographed image, performs face collation based on face image data read from an IC chip of the identity verification document and the target person face image, and determines whether or not the target person is the document person based on a result of the face collation.SELECTED DRAWING: Figure 1

Description

Technical Field

[0001] The present disclosure relates to a personal identification device, a personal identification method, and a computer program.

Background Art

[0002] A person who holds a personal identification document such as a driver's license (hereinafter referred to as the target person) checks whether the person shown in the personal identification document (hereinafter referred to as the document person) is the same as the target person by visually comparing the face of the target person with the face photograph printed on the personal identification document.

[0003] Alternatively, a reader of a personal identification document requests the target person to input a PIN code, and checks whether the target person is the document person based on whether the target person can input the correct PIN code (for example, Patent Document 1). Hereinafter, checking whether the target person is the document person may be referred to as personal identification.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the method of visually comparing the face of the target person with the face photograph printed on the personal identification document for personal identification, since the printed face photograph may be forged, the reliability that the target person is the document person is not necessarily high.

[0006] On the other hand, in the method of having the target person input the correct PIN code, inputting the PIN code is troublesome, and in many cases, the target person forgets the PIN code, resulting in poor efficiency of personal identification.

[0007] The purpose of this disclosure is to provide an identity verification device and method that can perform identity verification using identity verification documents with sufficient reliability and efficiency. [Means for solving the problem]

[0008] An identity verification device according to one embodiment is an identity verification device that determines whether or not a target person is the person shown on an identity verification document, comprising: a face-capturing camera that photographs the face of the target person and generates a face image of the target person; a document-capturing camera that photographs the surface of the identity verification document and generates a document image; and an IC reader that accesses the IC chip of the identity verification document. The system includes a processor, the processor performs face matching based on the face image data read from the IC chip and the target person's face image, and determines whether the target person is the person in the document based on the result of the face matching.

[0009] One embodiment of the identity verification method is an identity verification method for determining whether a subject person is the person shown on an identity verification document, the method comprising: photographing the subject person's face to generate a subject person face image; photographing the face of the identity verification document to generate a document image; performing face matching based on the face image data read from the IC chip of the identity verification document and the subject person face image; and determining whether the subject person is the person shown on the document based on the result of the face matching.

[0010] One embodiment of a computer program causes the computer program to execute the above-described identity verification method.

[0011] These comprehensive or specific embodiments may be implemented as systems, devices, methods, integrated circuits, computer programs, or recording media, or as any combination of systems, devices, methods, integrated circuits, computer programs, and recording media. [Effects of the Invention]

[0012] This disclosure provides an identity verification device and an identity verification method that can perform identity verification using identity verification documents with sufficient reliability and efficiency. [Brief explanation of the drawing]

[0013] [Figure 1] Block diagram showing an example configuration of the identity verification device according to Embodiment 1. [Figure 2] A table showing the information contained in each type of identification document. [Figure 3] Diagram showing examples of the front and back surfaces of a My Number Card. [Figure 4] Diagram showing an example of the front side of a driver's license. [Figure 5] Diagram showing an example of the front side of a residence card. [Figure 6] A diagram showing an example of the personal information page in a passport. [Figure 7] A flowchart showing an example of the operation of the identity verification device according to Embodiment 1. [Figure 8] A flowchart showing an example of the My Number Card information acquisition process according to Embodiment 1. [Figure 9] A flowchart showing an example of the passport information acquisition process according to Embodiment 1. [Figure 10] A flowchart showing an example of the residence card information acquisition process according to Embodiment 1. [Figure 11] A flowchart showing an example of the driver's license information acquisition process according to Embodiment 1. [Figure 12] A flowchart illustrating an example of the process when the identity verification device according to Embodiment 1 is linked with a self-checkout system to perform identity verification and age verification. [Modes for carrying out the invention]

[0014] Hereinafter, embodiments of the present disclosure will be described in detail with appropriate reference to the drawings. However, a more detailed description than necessary may be omitted. For example, detailed descriptions of well-known matters and redundant descriptions of substantially the same configurations may be omitted. This is to avoid making the following description unnecessarily redundant and to facilitate the understanding of those skilled in the art. Note that the accompanying drawings and the following description are provided to enable those skilled in the art to fully understand the present disclosure, and are not intended to limit the subject matter described in the claims.

[0015] (Embodiment 1) <Configuration of the device> FIG. 1 is a block diagram showing a configuration example of the personal identification device 10 according to Embodiment 1.

[0016] The personal identification device 10 includes a processor 11, a memory 12, a face camera 13, a document camera 14, an IC reader 15, an operation device 16, a display device 17, and a placement unit 18.

[0017] The processor 11 realizes the functions of the personal identification device 10 by executing a computer program stored in the memory 12.

[0018] The memory 12 stores a computer program and data for realizing the functions of the personal identification device 10. The memory 12 may include a ROM (Read Only Memory) and a RAM (Random Access Memory). Further, the memory 12 may include a volatile storage medium and a non-volatile storage medium.

[0019] The placement unit 18 is a part on which the personal identification document 20 is placed. In the present embodiment, the personal identification document 20 will be described as being any one of a my number card 31, a passport 32, a residence card 33, and a driver's license. However, the personal identification document 20 is not limited thereto. Note that "my number card" may be read as "personal number card".

[0020] The identification document 20 is contactless readable and has a secure IC chip 21. The IC chip 21 stores the personal information of the owner of the identification document 20. Details of the personal information will be described later (see Figure 2). The IC chip 21 also stores facial image data of the owner of the identification document 20. However, if the owner of the residence card 33 is under 16 years of age, facial image data does not need to be stored in the IC chip 21 of the residence card 33.

[0021] Furthermore, the identity verification document 20 has text information 23 indicating the personal information of the owner of the identity verification document 20, and a photograph 22 of the owner's face printed on it. However, if the owner of the residence card 33 is under 16 years of age, the residence card 33 does not need to have a photograph 22 printed on it.

[0022] The IC reader 15 reads the data stored in the IC chip 21 of the identification document 20 placed on the mounting unit 18 using a contactless method (for example, NFC (Near Field Communication)).

[0023] The document camera 14 photographs the surface of the identification document 20 placed on the mounting unit 18 and generates a document image.

[0024] The face-capturing camera 13 photographs the face of a person (hereinafter referred to as the "subject person") 1 who has placed their identification document 20 on the mounting unit 18, and generates a face image of the subject person.

[0025] The operating device 16 is a device that receives input from the target person 1. Examples of the operating device 16 include a touch panel, physical buttons, or a combination thereof.

[0026] The display device 17 is a device that displays information or images, etc. Examples of the display device 17 include liquid crystal displays or organic EL displays. The operating device 16 and the display device 17 may also be configured as an integrated device (for example, a touch panel display).

[0027] When person 1 places their identification document 20 on the placement section 18 of the identification device 10, the identification device 10 uses the face-capturing camera 13 to photograph person 1's face and generate a face image of the person, obtains a document face image for identity verification from the placed identification document 20, and performs face matching between the person's face image and the document face image. This face matching may be performed by directly comparing the document face image and the person's face image, or by generating facial feature quantities for both the document face image and the person's face image and comparing those facial feature quantities. Note that this face matching is not limited to a specific face matching method, and only needs to be able to determine whether the document face image and the person's face image belong to the same person. If the face matching is successful, the identification device 10 determines that person 1 is the person indicated by the identification document 20 (hereinafter referred to as the document person) (i.e., it determines that the person is the real person), and if the face matching fails, it determines that person 1 is not the document person (i.e., it determines that the person is not the real person). Further details will be provided later, but the document face image may be either face image data stored in the IC chip 21 or a face photograph 22 printed on the identity verification document 20 (hereinafter referred to as the face photograph image).

[0028] As described above, there are various types of identity verification documents 20, such as My Number Card 31 (Individual Number Card), passport 32 (travel document), residence card 33, and driver's license 34, and mainly refer to documents issued by government agencies. The identity verification device 10 according to this embodiment determines the type of identity verification document 20 placed on the placement unit 18 and performs appropriate processing based on the determination result, thereby enabling reliable and efficient identity verification using various types of identity verification documents 20. A detailed explanation follows below.

[0029] <Information contained in identity verification documents> Figure 2 is a table showing the main information contained in each type of identity verification document 20. Figure 3 is a diagram showing examples of the front and back surfaces of a My Number Card 31. Figure 4 is a diagram showing examples of the personal information pages of a passport 32. Figure 5 is a diagram showing examples of the front surface of a residence card 33. Figure 6 is a diagram showing examples of the front surface of a driver's license 34.

[0030] As shown in Figure 3, the front surface of the My Number Card 31 has at least the name, address, date of birth, gender, security code, and expiration date printed as text information 23. The information including name, address, date of birth, and gender is also called 4 information. In this embodiment, 4 information may also be referred to as personal information. In addition, a photograph 22 is printed on the front surface of the My Number Card 31.

[0031] The IC chip 21 of the My Number Card 31 stores at least the name, address, date of birth, gender, facial image data, and personal number. The facial image data is an image of the face of the My Number Card 31 holder.

[0032] The My Number Card 31 includes, at a minimum, a verification number A, a verification number B, a signature electronic certificate PIN, a user certificate PIN, a card surface information input assistance PIN, and a resident basic register PIN as information used for access control to the IC chip 21.

[0033] The verification number B is a number printed on the front of the My Number Card 31, combining the 6-digit date of birth, the 4-digit expiration date (in Western calendar year), and the 4-digit security code. By using the verification number B, it is possible to access the four pieces of information and facial image data stored in the IC chip 21 of the My Number Card 31. In other words, by obtaining the verification number B printed on the front of the My Number Card 31, it is possible to access the four pieces of information and facial image data stored in the IC chip 21 of the My Number Card 31.

[0034] As shown in Figure 4, the personal information page of the passport 32 has at least the name, date of birth, gender, passport number, expiration date, and MRZ (Machine Readable Zone) printed as text information 23. Additionally, the personal information page of the passport 32 also has a photograph 22 printed on it.

[0035] The IC chip 21 of the passport 32 stores at least the name, date of birth, gender, facial image data, and passport number.

[0036] The passport 32 has at least an MRZ as information used for access control to the IC chip 21. By using the MRZ, it is possible to access the 4 pieces of information and facial image data stored in the IC chip 21 of the passport 32. In other words, by obtaining the MRZ from the text information 23 printed on the identity information page of the passport 32, it is possible to access the 4 pieces of information and facial image data stored in the IC chip 21 of the passport 32.

[0037] As shown in Figure 5, the front of the residence card 33 has at least the name, address, date of birth, gender, residence card number, and expiration date printed as text information 23. Additionally, a photograph 22 (if the applicant is 16 years of age or older) is printed on the front of the residence card 33.

[0038] The IC chip 21 of the residence card 33 stores at least the address, facial image data, and card surface image data. The card surface image data is an image of the front surface of the residence card.

[0039] The residence card 33 has at least a residence card number as information used for access control to the IC chip 21. By using the residence card number, it is possible to access the four pieces of information, facial image data, and card surface image data stored in the IC chip 21 of the residence card 33. In other words, by obtaining the residence card number printed on the surface of the residence card 33, it is possible to access the four pieces of information, facial image data, and card surface image data stored in the IC chip 21 of the residence card 33.

[0040] As shown in Figure 6, the front of the driver's license 34 has at least the name, address, date of birth, license number, and expiration date printed as text information 23. Additionally, a photograph 22 is printed on the front of the driver's license 34.

[0041] The IC chip 21 of the driver's license 34 stores at least the name, address, date of birth, facial image data, and license number.

[0042] The driver's license 34 has at least PIN1 and PIN2 as information used for access control to the IC chip 21. PIN1 and PIN2 are four-digit numbers pre-set by the owner of the driver's license 34. By using PIN1, it is possible to access the four pieces of information stored in the IC chip 21. Furthermore, by using PIN1 and PIN2, it is possible to access the facial image data stored in the IC chip 21 of the driver's license 34.

[0043] The IC chip 21 of the identity verification document 20 is tamper-resistant, making it extremely difficult to physically alter the data stored in the IC chip 21. Furthermore, since the data stored in the IC chip 21 is electronically signed, any alteration of the data stored in the IC chip 21 can be detected. Therefore, when the identity verification device 10 reads facial image data from the IC chip 21 and performs facial matching with the subject person's facial image using the read facial image data as the document's facial image, and the facial matching is successful, the reliability of the identity verification result, which determines that subject person 1 is the person on the document, is very high.

[0044] Therefore, it is preferable for the identity verification device 10 to use the facial image data stored in the IC chip 21 for facial matching, but there may be situations in which the IC chip 21 cannot be accessed. In such cases, if the inability to access the IC chip 21 is considered a failure of identity verification, the efficiency of identity verification will deteriorate. Therefore, if the identity verification device 10 cannot access the IC chip 21, it may use the facial photograph 22 printed on the identity verification document 20 for facial matching. However, the facial photograph 22 printed on the identity verification document 20 may be physically tampered with. Therefore, even if the identity verification device 10 performs facial matching with the facial image of the target person using the facial photograph 22 printed on the identity verification document 20 as the document face image, and the facial matching is successful, the identity verification result in which the target person 1 is determined to be the document person is less reliable compared to the identity verification result determined using the facial image data stored in the IC chip 21 as described above.

[0045] The identity verification device 10 according to this embodiment, upon successful identity verification, also displays the confidence level of that verification success, thereby informing the person verifying whether the subject person 1 is the person on the document (hereinafter referred to as the verifier) ​​or another device how much confidence the verification success should have. This allows the verifier or other device to take various actions according to the confidence level of the identity verification success.

[0046] The operation of the identity verification device 10, which can display a level of trustworthiness, will be explained in detail below.

[0047] <Overall processing> Figure 7 is a flowchart showing an example of the operation of the identity verification device 10 according to Embodiment 1.

[0048] Person 1 places their identification document 20 on the placement unit 18. At this time, the identity verification device 10 detects that the identification document 20 has been placed on the placement unit 18 (S101). For example, the identity verification device 10 may determine whether or not the identification document 20 has been placed on the placement unit 18 by either of the following methods: (A1) The identity verification device 10 keeps the document camera 14 in a shooting state at all times, and determines that the identification document 20 has been placed on the placement unit 18 if the identification document 20 is photographed. (A2) The identity verification device 10 keeps the IC reader 15 in a reading state at all times, and determines that the identification document 20 has been placed on the placement unit 18 if it is able to read the IC chip 21 of the identification document 20.

[0049] The face-capturing camera 13 of the identity verification device 10 captures the face of the subject person 1 and generates a face image of the subject person (S102).

[0050] The processor 11 accesses the AID and common area of ​​the IC chip 21 of the identity verification document 20 through the IC reader 15, and determines the type of identity verification document 20 (hereinafter referred to as the document type) based on the identification information obtained through this access (S103). Alternatively, the processor 11 may determine the type of identity verification document 20 based on an image of the identity verification document 20.

[0051] Based on the determination result in step S103, the processor 11 determines whether the document type is "My Number Card," "Passport," "Residence Card," or "Driver's License" (S104).

[0052] If the result of the determination in step S104 is "My Number Card" (S104: My Number Card), the processor 11 executes the My Number Card information acquisition process (S105). Details of the My Number Card information acquisition process will be described later (see Figure 8).

[0053] If the result of step S104 is "passport" (S104: passport), the processor 11 executes the passport information acquisition process (S106). Details of the passport information acquisition process will be described later (see Figure 9).

[0054] If the result of the determination in step S104 is "Residence Card" (S104: Residence Card), the processor 11 executes the residence card information acquisition process (S107). Details of the residence card information acquisition process will be described later (see Figure 10).

[0055] If the result of the determination in step S104 is "driver's license" (S104: driver's license), the processor 11 executes the driver's license information acquisition process (S108). Details of the driver's license information acquisition process will be described later (see Figure 11).

[0056] In other words, based on the determination result of step S104 (i.e., the document type), the processor 11 switches to one of the access methods for the IC chip 21 among steps S105, S106, S107, and S108.

[0057] In each of the above steps S105, S106, S107, and S108, the processor 11 obtains a document facial image and personal information from the identity verification document 20, and also obtains a confidence level regarding the information obtained. Hereinafter, the document facial image and personal information obtained from the identity verification document 20 may be collectively referred to as document acquisition information.

[0058] Next, the processor 11 performs a face matching operation on the target person's face image against the document's face image (S110). This face matching operation may be performed using known face image matching techniques.

[0059] Processor 11 determines whether the face matching in step S110 was successful (S111). In other words, processor 11 determines whether the target person 1 is the person on the document.

[0060] If facial recognition fails (S111: NO), the processor 11 outputs information indicating that identity verification failed (S121) and terminates the process.

[0061] If facial recognition is successful (S111: YES), the processor 11 determines whether the reliability of the document acquisition information is "high" (S112).

[0062] If the reliability of the document acquisition information is not "high" (for example, if the reliability is "medium" or "low") (S112: NO), the processor 11 outputs information indicating successful identity verification and information indicating a reliability of "medium" or "low" (S122), and terminates this process. The reliability output here is the reliability determined in the processing of step S105, step S106, step S107, or step S108.

[0063] If the reliability of the document acquisition information is "high" (S112:YES), the processor 11 outputs information indicating successful identity verification and information indicating a "high" reliability (S123), and terminates this process.

[0064] The information output in step S121, step S122, or step 123 may be displayed on the display device 17. This allows the verifier to know not only whether or not the subject person 1 is the person on the document, but also the reliability of the verification if the identity verification is successful.

[0065] Furthermore, the information output in step S121, step S122, or step 123 may be transmitted to other devices. This allows other devices to utilize the determination result of whether or not the subject person 1 is the person on the document, as well as the confidence level in the case of successful identity verification. An example of how other devices can use the identity verification device 10 will be described later (see Figure 12).

[0066] In the process shown in Figure 7, if the face matching in step S110 is successful (S111:YES), identity verification is considered successful regardless of the reliability of the document information obtained. However, the success or failure of identity verification may be determined according to the reliability. For example, even if the face matching in step S110 is successful (S111:YES), the processor 11 may consider identity verification to have failed if the reliability of the document information obtained is not "high" (for example, if the reliability is "medium" or "low") (S112:NO), and may only consider identity verification successful if the reliability of the document information obtained is "high" (S112:YES).

[0067] Furthermore, while this embodiment uses "high," "medium," and "low" to express the level of reliability, other expressions may be used. For example, the reliability may be expressed numerically, with higher numbers indicating higher reliability and lower numbers indicating lower reliability.

[0068] <My Number Card Information Acquisition Process> Figure 8 is a flowchart showing an example of the My Number Card information acquisition process according to Embodiment 1. This process corresponds to step S105 in Figure 7.

[0069] The document camera 14 of the identity verification device 10 photographs the front surface of the My Number Card 31 and generates a document image (S201).

[0070] The processor 11 of the identity verification device 10 performs OCR (Optical Character Recognition) on the document image generated in step S201 to obtain character information 23, and obtains the verification number B from the character information 23 (S202). Specifically, the processor 11 obtains the 6-digit date of birth, the 4-digit expiration date (in Western calendar year), and the 4-digit security code printed on the surface of the My Number Card 31, and obtains the verification number B by combining them.

[0071] The processor 11 determines in step S202 whether or not it was possible to obtain the verification number B (S203). For example, if the ticket surface is dirty, or if the ticket surface photographed in step S201 is the opposite side (i.e., the back) from the side on which the verification number B is printed, it may not be possible to obtain the verification number B.

[0072] If the processor 11 can obtain the matching number B (S203: YES), it proceeds to step S204; if it cannot obtain the matching number B (S203: NO), it proceeds to step S210.

[0073] First, let's explain the case where we were able to obtain matching number B (S203:YES).

[0074] The processor 11 accesses the IC chip 21 using the matching number B as the key (S204). Then, the processor 11 determines whether or not it was able to access the IC chip 21 (S205).

[0075] If the processor 11 cannot access the IC chip 21 (S205: NO), it proceeds to step S210.

[0076] If the processor 11 can access the IC chip 21 (S205:YES), it reads the facial image data stored in the IC chip 21 as a document facial image and also reads the personal information stored in the IC chip 21 (S206). Then, the processor 11 sets the reliability of the document acquisition information, including the read facial image data (document facial image) and personal information, to "high" (S207) and terminates this process.

[0077] Next, we will explain what happens if the matching number B cannot be obtained (S203:NO) or if the IC chip 21 cannot be accessed (S205:NO).

[0078] The processor 11 acquires the facial photograph image contained in the document image as the document facial image, and also applies OCR to the document image to acquire the text information 23 printed on the document as personal information (S210). Then, the processor 11 sets the confidence level of the acquired facial photograph image (document facial image) and the acquired document information containing personal information to "low" (S211), and terminates this process (i.e., the process in step S105).

[0079] <Passport Information Acquisition Process> Figure 9 is a flowchart showing an example of the passport information acquisition process according to Embodiment 1. This process corresponds to step S106 in Figure 7.

[0080] The document camera 14 of the identity verification device 10 photographs the identity information page of the passport 32 and generates a document image (S301).

[0081] The processor 11 of the identity verification device 10 performs OCR on the document image generated in step S301 to obtain the MRZ (S302). Specifically, the processor 11 obtains the character information 23 printed on the lower part of the identity information page of the passport 32 by OCR, and obtains the MRZ from that character information 23.

[0082] The processor 11 determines in step S302 whether or not the MRZ was acquired (S303). For example, if the personal information page is dirty, or if the page photographed in step 301 is a different page from the personal information page, the MRZ may not be acquired.

[0083] If processor 11 is able to obtain MRZ (S303:YES), it proceeds to step S304; if it is unable to obtain MRZ (S303:NO), it proceeds to step S310.

[0084] First, let's explain what happens if you obtain MRZ (S303:YES).

[0085] The processor 11 accesses the IC chip 21 using MRZ as the key (S304). Then, the processor 11 determines whether or not it was able to access the IC chip 21 (S305).

[0086] If the processor 11 is unable to access the IC chip 21 (S305: NO), it proceeds to step S310.

[0087] If the processor 11 can access the IC chip 21 (S305: YES), it reads the facial image data stored in the IC chip 21 as a document facial image and also reads the personal information stored in the IC chip 21 (S306). Then, the processor 11 sets the confidence level of the document acquisition information, including the read facial image data (document facial image) and personal information, to "high" (S307) and terminates this process.

[0088] Next, we will explain what happens if MRZ cannot be obtained (S303:NO) or if IC chip 21 cannot be accessed (S305:NO).

[0089] The processor 11 acquires the facial photograph image contained in the document image as the document facial image, and also performs OCR on the document image to acquire the text information 23 printed on the page as personal information (S310). Then, the processor 11 sets the confidence level of the acquired facial photograph image (document facial image) and the acquired document information containing personal information to "low" (S311), and terminates this process (i.e., the process in step S106).

[0090] <Residence card information acquisition process> Figure 10 is a flowchart showing an example of the residence card information acquisition process according to Embodiment 1. This process corresponds to step S107 in Figure 7.

[0091] The document camera 14 of the identity verification device 10 photographs the front surface of the residence card 33 and generates a document image (S401).

[0092] The processor 11 of the identity verification device 10 performs OCR on the document image generated in step S401 to obtain character information 23, and obtains the residence card number from the character information 23 (S402).

[0093] Processor 11 determines whether or not the residence card number was obtained in step S402 (S403). For example, if the card surface is dirty, or if the side of the card photographed in step S401 is the opposite side (i.e., the back) from the side where the residence card number is printed, the residence card number may not be obtainable.

[0094] If the processor 11 is able to obtain the residence card number (S403:YES), it proceeds to step S404; if it is unable to obtain the residence card number (S403:NO), it proceeds to step S420.

[0095] First, let's explain the case where you have obtained a residence card number (S403:YES).

[0096] The processor 11 accesses the IC chip 21 using the residence card number as a key (S404). Then, the processor 11 determines whether or not it was able to access the IC chip 21 (S405).

[0097] If the processor 11 cannot access the IC chip 21 (S405: NO), it proceeds to step S420.

[0098] If the processor 11 can access the IC chip 21 (S405: YES), it retrieves the card image data stored in the IC chip 21 (S406). The card image data is image data of the front surface of the residence card 33.

[0099] The processor 11 performs OCR on the acquired ticket image data to obtain character information 23, and then obtains personal information from that character information 23 (S407).

[0100] Processor 11 determines whether the age included in the personal information obtained in step S407 indicates that the person is 16 years of age or older (S408).

[0101] If the personal information indicates that the person is 16 years of age or older (S408: YES), the processor 11 reads the facial image data from the IC chip 21 as a document facial image (S409). Then, the processor 11 sets the confidence level of the document acquisition information, including the acquired personal information and facial image data (document facial image), to "high" (S410) and terminates this process.

[0102] If the personal information indicates that the person is under 16 years old (S408:NO), the processor 11 does not acquire facial image data, sets the confidence level of the document information containing the personal information to "medium" (S411), and terminates the process. The reason for not acquiring facial image data is that the residence card 33 of a person under 16 years old does not have facial image data stored in the IC chip 21.

[0103] Next, we will explain what happens if you are unable to obtain a residence card number (S403:NO) or if you are unable to access IC chip 21 (S405:NO).

[0104] The processor 11 performs OCR on the captured document image to obtain character information 23, and then obtains personal information from that character information 23 (S420).

[0105] The processor 11 determines whether the age included in the personal information obtained in step S420 indicates that the person is 16 years of age or older (S421).

[0106] If the personal information indicates that the person is 16 years of age or older (S421: YES), the processor 11 acquires the facial photograph image included in the document image as the document facial image (S422). Then, the processor 11 sets the confidence level of the document acquisition information, including the acquired personal information and facial photograph image (document facial image), to "low" (S423), and terminates this process (i.e., the process in step S107).

[0107] If the personal information indicates that the person is under 16 years old (S421:NO), the processor 11 does not acquire a facial photograph image, sets the confidence level of the document information containing the personal information to "low" (S424), and terminates the process. The reason for not acquiring a facial photograph image is that residence cards 33 for persons under 16 years old do not have a facial photograph printed on the card.

[0108] <Driver's License Information Acquisition Process> Figure 11 is a flowchart showing an example of the driver's license information acquisition process according to Embodiment 1. This process corresponds to step S108 in Figure 7.

[0109] The document camera 14 of the identity verification device 10 photographs the front surface of the driver's license 34 and generates a document image (S501).

[0110] The processor 11 of the identity verification device 10 prompts the user to enter PIN1 (S502).

[0111] The processor 11 accesses the personal information on the IC chip 21 using the PIN 1 entered in step S502 as a key (S503).

[0112] The processor 11 determines whether or not it was able to access the personal information on the IC chip 21 (S504). For example, if the entered PIN1 is incorrect, or if access is locked, access is not possible.

[0113] If the processor 11 is able to access the personal information on the IC chip 21 (S504: YES), it proceeds to step S505; otherwise, it proceeds to step S520.

[0114] First, let's explain what happens if we can access the personal information on IC chip 21 (S504: YES).

[0115] The processor 11 reads personal information from the IC chip 21 (S505).

[0116] Processor 11 prompts the user to enter PIN2 (S506).

[0117] The processor 11 accesses the facial image data of the IC chip 21 using the PIN2 entered in step S506 as a key (S507).

[0118] The processor 11 determines whether or not it was able to access the facial image data on the IC chip 21 (S508). For example, if the entered PIN2 is incorrect, or if access is locked, access is not possible.

[0119] If the processor 11 can access the facial image data on the IC chip 21 (S508: YES), it reads the facial image data from the IC chip 21 and uses it as the document facial image (S509). Then, the processor 11 sets the confidence level of the document acquisition information, including the acquired personal information and facial image data (document facial image), to "high" (S510) and terminates this process.

[0120] If the processor 11 is unable to access the facial image data on the IC chip 21 (S508: NO), it obtains a facial photograph image from the document image and uses it as the document facial image (S511). Then, the processor 11 sets the confidence level of the document acquisition information, including the acquired personal information and facial photograph image (document facial image), to "medium" (S512) and terminates this process.

[0121] Next, we will explain what happens if you are unable to access the personal information on IC chip 21 (S504:NO).

[0122] The processor 11 performs OCR on the document image generated in step S501 to obtain character information 23, and then obtains personal information from that character information 23 (S520).

[0123] The processor 11 obtains a facial photograph image from the document image and uses it as the document facial image (S521). Then, the processor 11 sets the confidence level of the document acquisition information, including the acquired personal information and facial photograph image (document facial image), to "low" (S522), and terminates this process (i.e., the process in step S108).

[0124] <Integration with identity verification devices> The identity verification device 10 can be linked with various devices or systems. For example, examples of devices or systems that can be linked with the identity verification device 10 include self-checkout systems, vending machines, access control systems for amusement parks, identity verification of health insurance cards in medical facilities, or various other devices that require identity verification. Below, as an example of linkage, we will describe the case in which the identity verification device 10 is linked with a self-checkout system.

[0125] Figure 12 is a flowchart showing an example of the process when the identity verification device 10 according to Embodiment 1 is linked with a self-checkout register to perform identity verification and age verification.

[0126] The identity verification device 10 performs the process shown in Figure 7 (S601). Specifically, the person 1 using the self-checkout machine places their identity verification document 20 on the placement section 18 of the identity verification device 10. The identity verification device 10 performs the process shown in Figure 7 on the identity verification document 20 and outputs the identity verification result (success or failure), personal information, and trust level to the self-checkout machine.

[0127] Self-checkout machines scan the barcodes of the products (S602).

[0128] The self-checkout machine determines in step 602 whether or not any of the scanned items require age verification (S603).

[0129] If the scanned item does not require age verification (S603: NO), the self-checkout machine adds the item's price to the total (S610) and returns to step S602.

[0130] If the scanned product requires age verification (S603: YES), the self-checkout system determines whether the identity verification of person 1 was successful based on the identity verification result output from the identity verification device 10 (S604). In addition to whether the identity verification was successful or not, the identity verification result may also include information indicating the date of birth of person 1.

[0131] If the identity verification of person 1 is successful (S604: YES), the self-checkout machine refers to the personal information output from the identity verification device 10 and determines whether or not person 1 is of an age or older that is permitted to purchase goods (S605).

[0132] If the age of person 1 is above the age at which the product can be sold (S605: YES), the self-checkout system determines whether the confidence level output from the identity verification device 10 is "high" (S606).

[0133] If the confidence level is "high" (S606: YES), the self-checkout machine adds the price of the items to the total (S610) and returns to step S602.

[0134] If the identity verification of person 1 fails (S604: NO), if person 1 is under the age to which the product can be sold (S605: NO), or if the confidence level output from the identity verification device 10 is "medium" or "low" (S606: NO), the self-checkout system will call a store employee (S611) and temporarily suspend processing. At this time, the self-checkout system's monitor may display information indicating whether identity verification failed, whether person 1 is under the age to which the product can be sold, or whether the confidence level is "medium" or "low". If the called store employee finds that person 1 is under the age to which the product can be sold, they may, for example, collect the product and resume processing at the self-checkout system, ask the person to line up again at a manned checkout, or visually confirm the person's age on the spot and have the person approve the age verification at the self-checkout system. Furthermore, if the identity verification fails, or if the reliability is "medium" or "low," the summoned employee may, for example, ask person 1 to place another form of identification 20 on the placement section 18 of the identity verification device 10 and undergo identity verification again.

[0135] In this way, by linking the identity verification device 10 with other devices, the other devices can use the identity verification results, personal information, and confidence level output from the identity verification device 10 to perform processing according to the required level of strictness of identity verification. For example, in situations where strict identity verification is required, even if the identity verification output from the identity verification device 10 is successful, the other device may determine that the identity verification has failed if the confidence level output along with it is lower than "high". On the other hand, in situations where such strict identity verification is not required, the other device may determine that the identity verification has succeeded if the identity verification output from the identity verification device 10 is successful and the confidence level output along with it is "high" or "medium", and may determine that the identity verification has failed if the confidence level output along with it is "low".

[0136] The above example describes a self-checkout system, where if identity verification fails, the reliability is low, or identity verification cannot be performed, human intervention or suspension of sales is performed. However, when using the identity verification device 10 for entry and exit in locations with low security risks, entry and exit may be permitted even if the reliability is low. In this case, the identity verification device 10 may notify the user (administrator) that the reliability was low. This allows the user (administrator) to visually confirm the actual verification image of the person 1 with low reliability, or to monitor that person 1. This makes it possible to realize a system including the identity verification device 10 that can address security risks while maintaining the convenience of entry and exit. In this way, other devices or systems can take various actions depending on the reliability level output by the identity verification device 10.

[0137] (Summary of this disclosure) Based on the above description of embodiments, the following technologies are disclosed.

[0138] <Technology 1> The identity verification device 10, which determines whether or not the subject person 1 is the person shown on the identity verification document 20, comprises a face-capturing camera 13 that photographs the face of the subject person 1 and generates a face image of the subject person, a document-capturing camera 14 that photographs the surface of the identity verification document 20 and generates a document-capturing image, an IC reader 15 that accesses the IC chip 21 of the identity verification document 20, and a processor 11. The processor 11 performs face matching based on the face image data read from the IC chip 21 and the face image of the subject person, and determines whether or not the subject person 1 is the person shown on the document based on the result of face matching. In this way, the identity verification device 10 can perform identity verification using the identity verification document 20 with sufficient reliability and efficiency by comparing the subject person's face image, which is a photograph of the subject person's face, with the face image data stored in the IC chip 21 of the identity verification document 20, or with the face photograph printed on the face of the identity verification document 20, and by determining whether the subject person 1 is the person on the document.

[0139] <Technology 2> In the identity verification device 10 described in Technology 1, the processor 11 identifies the type of identity verification document 20, switches the access method to the IC chip 21 based on the identified type, and reads facial image data from the IC chip 21 using the switched access method. This makes it possible to provide an identity verification device 10 that can use various types of identity verification documents 20 that have different methods of accessing the IC chip 21.

[0140] <Technology 3> In the identity verification device 10 described in Technology 2, the types of identity verification documents 20 include at least one of the following: My Number Card 31, Passport 32, Residence Card 33, and Driver's License 34. This makes it possible to provide an identity verification device 10 that can use My Number Card 31, passport 32, residence card 33, and driver's license 34 for identity verification.

[0141] <Technology 4> In the identity verification device 10 described in Technology 2 or 3, if the processor 11 identifies that the identity verification document 20 is of a type that can access the IC chip 21 based on the information on the face of the identity verification document 20, it switches to an access method that reads character information from the captured image of the document and reads facial image data from the IC chip 21 using the read character information. As a result, when a My Number Card 31, passport 32, or residence card 33 is placed on the identity verification device 10, the device can automatically acquire facial image data from the IC chip 21 and perform facial matching without requiring the person 1 to enter a PIN or the like. Therefore, the identity verification device 10 can perform identity verification efficiently and with high reliability.

[0142] <Technology 5> In the identity verification device 10 described in any one of technologies 2 to 4, if the processor 11 determines that the type of identity verification document 20 is one that can be accessed from the IC chip 21 based on a PIN, it prompts the subject person 1 to enter a PIN and switches to an access method that reads facial image data from the IC chip 21 using the entered PIN. As a result, when a driver's license 34 is placed on the device, the identity verification device 10 can automatically acquire facial image data from the IC chip 21 using the entered PIN and perform verification. Therefore, the identity verification device 10 can perform identity verification efficiently and with high reliability.

[0143] <Technology 6> In the identity verification device 10 described in any one of technologies 1 to 5, the identity verification document 20 has a facial photograph on the document surface, and if the processor 11 is able to read facial image data from the IC chip 21, it performs facial matching based on the facial image data read from the IC chip 21 and the facial image of the target person. If the processor 11 is unable to read facial image data from the IC chip 21, it performs facial matching based on the facial photograph image obtained from the document image and the facial image of the target person. In this way, the identity verification device 10 can perform identity verification with higher reliability by preferentially using facial image data acquired from the IC chip 21 for facial matching.

[0144] <Technology 7> In the identity verification device 10 described in any one of technologies 1 to 6, the processor 11 determines the reliability of the face matching result based on whether face image data or a photographic image is used for face matching. In this way, the identity verification device 10 determines the reliability of the facial recognition result, enabling various subsequent actions and processes based not only on the facial recognition result but also on that reliability.

[0145] <Technology 8> In the identity verification device 10 described in Technology 7, the processor 11 determines a higher confidence level when using facial image data for facial matching than when using a facial photograph image for facial matching. In this way, by determining a high level of confidence when using facial image data read from the IC chip 21 for facial matching, the identity verification device 10 can be used for various responses and processes that require a higher level of confidence.

[0146] <Technology 9> The identity verification method for determining whether or not person 1 is the person shown on the identity verification document 20 involves taking a photograph of person 1's face to generate a face image of person 1, taking a photograph of the face of the identity verification document 20 to generate a document image, performing a face comparison based on the face image data read from the IC chip 21 of the identity verification document 20 and the face image of person 1, and determining whether or not person 1 is the person shown on the document based on the result of the face comparison. In this way, the identity verification method allows for reliable and efficient identity verification using the identity verification document 20 by comparing the facial image of the subject person, which is a photograph of the subject person's face, with the facial image data stored in the IC chip 21 of the identity verification document 20, or with the facial photograph printed on the face of the identity verification document, and determining whether the subject person 1 is the person on the document.

[0147] <Technology 10> The computer program causes the computer to perform the actions described in Technique 9. In this way, by comparing the facial image of the subject person, which was captured by photographing the subject person's face, with the facial image data stored in the IC chip 21 of the identity verification document 20, or the facial photograph printed on the face of the identity verification document, and determining whether the subject person 1 is the person on the document, the computer running the computer program can perform identity verification using the identity verification document 20 with sufficient reliability and efficiency.

[0148] While embodiments have been described above with reference to the attached drawings, this disclosure is not limited to such examples. It is clear to those skilled in the art that various modifications, alterations, substitutions, additions, deletions, and equivalents can be conceived within the scope of the claims, and these are also understood to fall within the technical scope of this disclosure. Furthermore, the components of the embodiments described above can be combined in any way without departing from the spirit of the invention. [Industrial applicability]

[0149] The technology disclosed herein is useful for devices, methods, and computer programs, etc., that determine, confirm, or prove that a person is the person indicated in identification documents. [Explanation of Symbols]

[0150] 1. Target Person 10. Identity Verification Device 11 processors 12 memory 13 Face-capturing camera 14. Document photography camera 15 IC Reader 16 Operating device 17 Display device 18 Mounting section 20. Identity verification documents 21 IC chips 22 Face photo 23 characters 31 My Number Card 32 Passport 33 Residence Card 34. Driver's license

Claims

1. An identity verification device that determines whether or not the person in question is the person shown on the identification document, A face-capturing camera that captures the face of the aforementioned person and generates a face image of the person, A document camera that photographs the surface of the aforementioned identification document and generates a document image, An IC reader that accesses the IC chip of the aforementioned identity verification document, Equipped with a processor, The aforementioned processor, Identify the type of the aforementioned identity verification document, If the identity verification document is identified as a type that can access the IC chip based on the information on the document, the access method switches to reading text information from the document image and using the read text information to read facial image data from the IC chip. The face image data is read from the IC chip using the switched access method. Face matching is performed based on the face image data read from the IC chip and the target person's face image. Based on the results of the facial matching, it is determined whether the subject person is the person described in the document. Identity verification device.

2. The aforementioned identity verification documents must include at least one of the following: My Number Card, passport, residence card, and driver's license. The identity verification device according to claim 1.

3. The aforementioned processor, If the type of the aforementioned identity verification document is identified as a type that can access the IC chip based on the PIN, The access method involves having the person to input a PIN and then using the input PIN to read the facial image data from the IC chip. The identity verification device according to claim 1.

4. The aforementioned identification document has a photograph of the face on it, The aforementioned processor, If the facial image data can be read from the IC chip, a facial matching is performed based on the facial image data read from the IC chip and the target person's facial image. If the facial image data cannot be read from the IC chip, a facial matching is performed based on the facial photograph image obtained from the document image and the facial image of the target person. The identity verification device according to claim 1.

5. The aforementioned processor, The reliability of the facial matching result is determined based on whether the facial image data or the facial photograph image is used for facial matching. The identity verification device according to claim 4.

6. The aforementioned processor, The confidence level when using the aforementioned facial image data for facial matching is determined to be higher than the confidence level when using the aforementioned facial photograph image for facial matching. The identity verification device according to claim 5.

7. A method for verifying identity in which a computer determines whether or not a person is the person shown on an identity verification document, A facial camera is used to capture the face of the subject person and generate a facial image of the subject person. Using a document-taking camera, the surface of the aforementioned identity verification document is photographed to generate a document image. Identify the type of the aforementioned identity verification document, If the identity verification document is identified as a type that allows access to the IC chip of the identity verification document based on the information on the document's surface, the access method switches to reading text information from the document's image and using the read text information to read facial image data from the IC chip. The face image data is read from the IC chip using the switched access method. Face matching is performed based on the face image data read from the IC chip and the target person's face image. Based on the results of the facial matching, it is determined whether the subject person is the person described in the document. Methods of identity verification.

8. A computer program for causing the computer to perform the method according to claim 7.

9. An identity verification device for determining whether or not a target person is the person shown on an identity verification document, A face-capturing camera that captures the face of the aforementioned person and generates a face image of the person, A document camera that photographs the surface of the aforementioned identification document and generates a document image, An IC reader that accesses the IC chip of the aforementioned identity verification document, Equipped with a processor, The aforementioned identification document has a photograph of the face on it, The aforementioned processor, If facial image data can be read from the IC chip, a facial matching is performed based on the facial image data read from the IC chip and the target person's facial image. If the facial image data cannot be read from the IC chip, a facial matching is performed based on the facial photograph image obtained from the document image and the facial image of the person concerned. Based on the results of the facial matching, it is determined whether the subject person is the person described in the document. Identity verification device.

10. The processor is The reliability of the facial matching result is determined based on whether the facial image data or the facial photograph image is used for facial matching. The identity verification device according to claim 9.

11. The processor is The confidence level when using the aforementioned facial image data for facial matching is determined to be higher than the confidence level when using the aforementioned facial photograph image for facial matching. The identity verification device according to claim 10.

Images