PHOTOGRAPHING APPARATUS, ITS CONTROL METHOD, AND PROGRAM

Abstract

A photographing device, a control method and a program for the device, which enables electronic application for administrative procedures etc. by strict identity verification, are provided. [Solution] This is a photography device that uses a reading device, camera, IC card reader, etc. to read the face information and data in the IC chip of the applicant's ID card (personal identification medium), and uses the camera to take an application image, which is a photograph to be used in the application procedure. The same person determination unit B408 of the control unit 101 compares the face information read from the ID card or the person image contained in the IC chip with the taken application image to determine whether they are the same person. The authenticity determination unit 402 determines the authenticity of the ID card based on the information read from the ID card.

Description

[Technical field]

[0001] The present invention relates to a photographing device, a control method thereof, and a program, and more particularly to electronic applications for administrative procedures and the like. [Background technology]

[0002] Electronic application schemes using smartphones and other devices are beginning to penetrate the market. For example, there are plans to digitize application procedures and realize them online for new passport / renewal applications, My Number card renewal applications, driver's license renewal applications, and other applications. Applications require identity verification using a medium of identification (identification card; ID card) such as a driver's license, but this is not done face-to-face, and electronic applications using personal devices raise concerns about fraud whereby ID cards are forged or altered to impersonate another person.

[0003] Incidentally, Patent Document 1 discloses a technology for determining whether an ID card such as a driver's license is counterfeit. The authenticity determination device in Patent Document 1 is composed of a reader that reads an ID card and a computer that performs authenticity determination, and determines whether there is a contradiction in the contents indicated by the character information obtained from the card face image, thereby determining the authenticity of the ID card. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-209603 A [Non-patent literature]

[0005] [Patent Document 1] Ministry of Foreign Affairs, "Roadmap for digitalization of administrative procedures", [online], searched on December 15, 2020, Internet,<URL: http: / / www.moj.go.jp / content / 001318660.pdf> Summary of the Invention [Problem to be solved by the invention]

[0006] However, as mentioned above, electronic applications have a higher risk of data tampering than face-to-face application procedures, making it difficult to strictly verify the identity of the person. In addition, it is difficult to require proper photographs when electronic applications are made using the consumer's own smartphone or PC. In fact, according to Non-Patent Document 1, "Roadmap for the Digitalization of Administrative Procedures," it is not possible to use smartphones or home PCs to make electronic applications for next-generation passports.

[0007] The present invention has been made in consideration of the above problems, and has an object to provide a photographing device etc. that can realize electronic application by performing strict identity verification. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, the first invention is a photographing device equipped with a camera for photographing an applicant as a subject, an identity verification means for receiving an input of an authentication code that has been given to the applicant by a predetermined server; A reading means for reading information on the applicant's identity verification medium; an action instruction means for instructing the applicant to take an action; and a liveness determination means for capturing a video of the applicant's movements responding to the action instruction given by the action instruction means with the camera and analyzing the captured video to determine whether the applicant is a live person or not; an application image capturing means for capturing an application image, which is a photograph used in an application procedure, by the camera; a same person determination means for comparing a person image included in the information read by the reading means with the application image and determining whether they are the same person; an authenticity determination means for determining the authenticity of the personal identification medium based on the information read by the reading means; and a processing means for processing the application image when the same person determination means determines that they are the same person and the personal identification medium is determined to be authentic by the authenticity determination means. The above and an output means for outputting the information to a predetermined server. The identity verification means accepts input of an authentication code before the reading means reads information from the identity verification medium of the applicant, and the same person determination means further compares a front face image of the applicant cut out from a video captured in a liveness determination process by the liveness determination means with the application image to determine whether they are the same person. The photographing device is characterized in that

[0009] According to the first invention, the photographing device reads information on the applicant's identification medium by the reading means, takes an application image, which is a photograph used in the application procedure, by the camera, and compares the person image included in the information read by the reading means with the application image to determine whether they are the same person. Also, the authenticity of the identification medium is determined based on the information read by the reading means. If it is determined that the person image included in the information read by the reading means and the person in the application image are the same person and the identification medium is determined to be authentic, the application image is output to a specified server.

[0010] This makes it possible to use the photographing device to determine the authenticity of the identity verification medium and to strictly confirm whether the person written on the identity verification medium is the same as the photographed person (applicant). This makes it possible to prevent fraud in electronic applications and realize electronic applications for administrative procedures, etc. By using an image taken with the photographing device for an application, a proper image for application can be provided. It is also preferable that the system further includes a registration information acquisition means for acquiring registration information that the applicant has registered in advance in the server and that is stored in the server in association with the authentication code after the identity verification means accepts input of the authentication code, and the identity verification means performs identity verification of the applicant by comparing the registration information acquired by the registration information acquisition means with the information read by the reading means. This allows the applicant to efficiently apply on the photographing device using information that the applicant has registered in advance from a PC or the like at home. In this case, the identity of the applicant is verified by comparing the contents of the identity verification medium with the contents of the registration information, thereby preventing fraudulent applications. It is also preferable that the liveness determination means performs liveness determination by checking whether or not there is a characteristic movement of a living organism while detecting a response of the applicant to an action instruction by the action instruction means.

[0011] In the first aspect of the present invention, the reading means includes a scanner for reading face information in the personal identification medium; Identity verification medium The personal identification medium may include at least one of a camera for taking an image of the face of the personal identification medium and a reader for reading data recorded on a recording medium in the personal identification medium, and the authenticity determination means determines the authenticity of the personal identification medium based on at least one of the face information read by the scanner, the image of the face of the personal identification medium taken by the camera, and the data read by the reader. The personal identification medium may further include an input means for inputting a secret code for the personal identification medium, the reader reads data stored in a memory area protected by the secret code using the secret code input by the input means, and the authenticity determination means may further determine the authenticity of the personal identification medium based on the data stored in the memory area protected by the secret code. This makes it possible to determine the authenticity of the personal identification medium using the face information of the personal identification medium and the data recorded therein.

[0016] Also ,before The system may further include a biometric authentication device that reads biometric information of the applicant and performs authentication processing based on the read biometric information. The biometric authentication device may be, for example, a fingerprint authentication device that reads fingerprint information of the applicant and performs authentication processing based on the read fingerprint information, a vein authentication device that reads vein information of the applicant and performs authentication processing based on the read vein information, an iris authentication device that reads iris information of the applicant and performs authentication processing based on the read iris information, or the like, and any one of these may be used, or a combination of two or more may be used.

[0017] It is preferable that the application procedure further includes a document scanner for reading documents used in the application procedure, and the output means outputs the document data read by the document scanner together with the application image to the predetermined server. This makes it possible to set the result of identity verification and the images and documents required for the application and perform an electronic application.

[0018] It is preferable that the application procedure is performed by a setting means for setting a secret code required for the application procedure, and the output means outputs the secret code set by the setting means to the predetermined server together with the application image. This makes it possible to input a secret code set at the time of application, such as a secret code for a driver's license, into the photographing device.

[0020] A second aspect of the present invention is a method for controlling a photographing device equipped with a camera that photographs an applicant as a subject, comprising: a code value acquisition step of accepting an input of an authentication code previously assigned to the applicant by a predetermined server; and after the code value acquisition step, a reading step of reading information on the applicant's identity verification medium; an action instruction step of instructing the applicant to perform an action; and a liveness determination step of capturing a video of the applicant's movements responding to the action instruction with the camera and analyzing the captured video to determine whether the applicant is a live person or not. an application image capturing step of capturing an application image, which is a photograph to be used in the application procedure, by the camera; and a first same person determination step of comparing a person image included in the information read by the reading step with the application image to determine whether or not they are the same person; a second same person determination step of comparing a front face image of the applicant cut out from the video captured in the liveness determination process in the liveness determination step with the application image to determine whether or not they are the same person; an authenticity determination step for determining the authenticity of the personal identification medium based on the information read in the reading step; and and the second same person determination stepWhen it is determined that the person is the same person by the above-mentioned authenticity determination step and the identity verification medium is determined to be authentic by the above-mentioned authenticity determination step, The above and an output step of outputting the image data to a predetermined server.

[0021] The second invention makes it possible to use a photographing device to determine the authenticity of a personal identification medium and to strictly confirm whether the person described on the personal identification medium is the same as the photographed person (applicant). This makes it possible to prevent fraud in electronic applications and realize electronic applications for administrative procedures, etc. By using an image taken by a photographing device for an application, a proper image for application can be provided.

[0022] In the second invention, The code value received in the code value acquisition step The method may further include a step of acquiring a unique ID that is information linked to the authentication code, and the output step may link the unique ID to the application image and transmit the image to the server. The output step may further link auxiliary information to the application image and the unique ID and transmit the auxiliary information to the server.

[0023] In addition, in the second invention, it is desirable to further include a second same person determination step of comparing a frontal facial image of the applicant cut out from a video captured for the biometric determination process with the application image to determine whether the photographed person matches.

[0024] In addition, in the first same person determination process, it is desirable to further compare a frontal facial image of the applicant extracted from the video captured in the biometric determination process with the person image contained in the information read in the reading step and the application image, and determine whether they are all the same person.

[0025] The third invention relates to a control device for a photographing device having a camera for photographing an applicant as a photographed subject, an identity verification means for receiving an input of an authentication code previously assigned to the applicant by a predetermined server; A reading means for reading information on the applicant's identity verification medium; an action instruction means for instructing the applicant to take an action; a liveness determination means for capturing a video of the applicant's movements responding to the action instruction given by the action instruction means with the camera and analyzing the captured video to determine whether the applicant is a live person;an application image capturing means for capturing an application image, which is a photograph used in application procedures, by the camera; a same person determination means for comparing a person image included in the information read by the reading means with the application image and determining whether they are the same person; an authenticity determination means for determining the authenticity of the personal identification medium based on the information read by the reading means; and when the same person determination means determines that they are the same person and the authenticity determination means determines that the personal identification medium is authentic, The above A program for functioning as an output means for outputting to a specified server the personal identification means accepts input of an authentication code before the reading means reads information from the personal identification medium of the applicant, and the same person determination means further compares a front face image of the applicant cut out from a video captured in a liveness determination process by the liveness determination means with the application image to determine whether they are the same person. It is.

[0026] According to the third invention, it is possible to cause a computer to function as a control device for the photographic device of the first invention. Effect of the Invention

[0027] According to the present invention, it is possible to perform strict identity verification using a photographing device, and it is possible to realize electronic applications for administrative procedures, etc. [Brief description of the drawings]

[0028] [Figure 1] FIG. 1 is a diagram showing the external configuration of a photographing device 1. [Diagram 2] FIG. 1 is a diagram showing a hardware configuration of a photographing device 1. [Diagram 3] FIG. 1 is a diagram showing an example of a communication connection between a photographing device 1 and a server 5. [Figure 4] FIG. 1 shows a functional configuration of a control unit 101 of a photographing device 1. [Diagram 5] A flowchart showing the flow of a menu selection process executed by the photographing device 1. [Figure 6] Flowchart showing the flow of application menu processing [Figure 7] FIG. 13 is a diagram showing an example of an application content selection screen 51. [Figure 8] FIG. 13 is a diagram showing an example of a code reading guide screen 52. [Figure 9] FIG. 13 is a diagram showing an example of a code input screen 53. [Figure 10] FIG. 13 shows an example of a code confirmation screen 54. [Figure 11] FIG. 13 is a diagram showing an example of an ID card insertion guide screen 55. [Figure 12] FIG. 13 shows an example of a screen 56 for inputting personal identification information on an ID card. [Figure 13] Flowchart showing the flow of identity verification processing [Figure 14] FIG. 13 is a diagram showing an example of an error notification screen 57. [Figure 15] Flowchart showing the flow of authenticity determination processing [Figure 16] Flowchart showing the flow of a liveness determination process [Figure 17] FIG. 13 shows an example of an operation instruction screen 58. [Figure 18] Flowchart showing the flow of processing following FIG. 6 (application menu processing) [Figure 19] FIG. 13 is a diagram showing an example of an application image capture screen 61. [Figure 20] FIG. 13 shows an example of operation screens 62 and 63 for aligning photographs. [Figure 21] Flowchart showing the flow of object detection processing [Figure 22] Image of a fraudster covering his face and a diagram explaining the object detection process [Diagram 23] Flowchart showing the flow of the same person determination process A [Figure 24] Flowchart showing the flow of the same person determination process B [Diagram 25] Flowchart showing the flow of processing following FIG. 18 (application menu processing) [Figure 26] FIG. 7 shows an example of a visual acuity test screen 71 and a measurement result screen 72. [Figure 27] FIG. 7 shows an example of a password setting screen 73. [Figure 28] FIG. 7 shows an example of (a) an application document reading guidance screen 74 and (b) a confirmation screen 75 for the read document data. [Figure 29] FIG. 76 is a diagram showing an example of an application content confirmation screen 76. [Diagram 30] FIG. 7 shows an example of an end screen 77. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0029] A preferred embodiment of the present invention will be described in detail below with reference to the drawings. The photographing device 1 according to the present invention includes a stationary type photographing device for ID photography installed at a storefront, a station, on the street, etc., and a stationary type photographing device used for taking photographs for purposes other than ID photography.

[0030] FIG. 1 is a diagram showing the external configuration of the photographing device 1, and FIG. 2 is a diagram showing the hardware configuration of the photographing device 1. As shown in FIG. 1, the photographing device 1 is a box-type device that allows a person to enter the inside of a housing that is a photographing room and take pictures and perform operations. The form of the photographing device 1 is not limited to a box type, and may be an open type or a stand type. The photographing device 1 has a function of taking a picture of a person in a photographing room as a subject, generating an identification photo according to a purpose from the image and printing it (identification photo menu), and a function of printing an image received from a terminal 6 such as a user's smartphone, or downloading and printing an image that the user has previously uploaded from the terminal 6 to the server 5 (image print menu). In the present invention, the photographing device 1 has an application menu for carrying out application procedures for a driver's license, a My Number card, a residence card, a special permanent resident card, a passport, etc. A user can select and use one of the identification photo menu, the image print menu, the application menu, and other menus.

[0031] The photographing device 1 shown in the example of Fig. 1 is a box type with an open side, and the outside is separated from the inside (photography room) by a curtain 28. On the front panel of the photography room, an operation panel 21, a coin mechanism 22, a camera 23, a lighting unit 24, an audio output unit 27, a subject sensor 29, a reader 31 for an identity verification medium (hereinafter referred to as an ID card), a document scanner 34, a code reader 35, etc. are provided. In addition, inside the housing of the photographing device 1, a control device 10, a printer 25, etc. are stored, and printed materials printed by the printer 25 are discharged from a printed material outlet 26 provided on an outer wall panel of the photographing device 1.

[0032] Fig. 2 is a block diagram showing the hardware configuration of the photographing device 1. As shown in Fig. 2, the control device 10 of the photographing device 1 is connected to the lighting unit 24, the camera 23, the object sensor 29, the audio output unit 27, the operation panel 21, the coin mechanism 22, the printer 25, the reading device 31, the document scanner 34, the code reader 35, etc. via a bus, and the control device 10 controls each unit.

[0033] The control device 10 includes a control unit (CPU; Central Processing Unit) 101, an I / O (Input / Output) control 102, an I / F (Interface) 103, and a ROM (Read Only Memory). The control device 10 has an I / F 103, a memory 104, a RAM (Random Access Memory) 105, a storage unit 106, and a communication unit 107. The control device 10 is connected to an operation panel 21 and a voice output unit 27 via an I / O control 102. The control device 10 is also connected to a coin mechanism 22, a printer 25, a reading device 31, a document scanner 34, and a code reader 35 via an I / F 103.

[0034] The control unit 101 is a CPU and centrally controls each unit of the photography device 1. The control unit 101 loads programs stored in the storage unit 106, ROM 104, etc. into a work area on the RAM 105 and executes them. The ROM 104 is a non-volatile memory and permanently stores programs such as a boot program and BIOS, data, etc. The RAM 105 is a volatile memory and temporarily stores programs and data loaded from the storage unit 106, ROM 104, etc., and also includes a work area used by the control unit 101 to perform various processes.

[0035] The storage unit 106 is a hard disk drive, a solid state drive, a flash memory, or the like, and stores programs executed by the control unit 101 during processing described below, data necessary for executing the programs, an OS, etc. These programs and data are read and executed by the control unit 101 as necessary.

[0036] The I / O control 102 is an interface for inputting and outputting various types of information to and from the operation panel 21, the audio output unit 27, and the like. The I / F 103 is an interface for inputting and outputting various types of information between the coin mechanism 22, the printer 25, the reading device 31, the document scanner 34, and the code reader 35.

[0037] The communication unit 107 includes a communication control device and a communication interface that control communication of the photography device 1. Fig. 3 is a diagram for explaining a communication connection between the photography device 1 and the server 5. The photography device 1 is equipped with a wireless LAN such as Wi-Fi (registered trademark) or a short-range wireless communication such as Bluetooth (registered trademark), and is communicatively connected to the server 5 via a wide area network 7 such as the Internet.

[0038] The server 5 is a server that manages the photographing device 1. When information required for identity verification or application is input from the user's terminal 6 via a web browser or an app, the server 5 issues a predetermined code and registers the input information in association with the code. The server 5 has a storage area for storing various data (photographic images and other data, etc.) uploaded from the photographing device 1.

[0039] A user can download image data of a photograph taken by the photographing device 1 from the server 5 to a terminal 6 such as a PC or a smartphone via the network 7. In addition, an image uploaded from the user's terminal 6 to the server 5 via the network 7 can be downloaded to the photographing device 1 and printed (image print menu).

[0040] 1 and 2. Illumination unit 24 is a surface light source located in front of or behind the subject. Illumination unit 24 is connected to light source (strobe) 241, and the timing and amount of light emission are controlled according to a control signal input from control unit 101.

[0041] Camera 23 is a digital camera that is composed of an optical lens, an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), an A / D (Analog / Digital) converter, etc. Camera 23 captures moving images or still images in accordance with a control signal input from control unit 101. Camera 23 inputs image data obtained by imaging to control unit 101. Control unit 101 displays and outputs the image data input from camera 23 on the display of operation panel 21. La 23 also functions as an ID card reader by photographing the face of the ID card.

[0042] The operation panel 21 is an operation means of the photography device 1, and has a display for displaying an operation screen showing an operation procedure, a preview screen, etc., as well as a touch panel for a user to input operation instructions. The operation panel 21 performs display processing according to a display signal input from the control unit 101. Also, the operation panel 21 sends a control signal to the control unit 101 according to a touch operation on the touch panel.

[0043] The object sensor 29 is a sensor that detects a person in the photography room. When a person approaches the front panel of the photography device 1, the object sensor 29 outputs a detection signal to the control unit 101. As the object sensor 29, an optical sensor consisting of a light emitter and a light receiver is used.

[0044] The audio output unit 27 acquires predetermined audio data (operation guidance audio, etc.) from the ROM 104 or the storage unit 106 in accordance with a control signal input from the control unit 101, and outputs the audio from a speaker.

[0045] The coin mech 22 is a cash register equipped with a deposit port for coins and bills, a refund port for change, etc., and performs processing for settling the usage fee for the photography device 1. The coin mech 22 outputs information on the amount deposited from the deposit port to the control unit 101, and discharges coins and bills according to the change amount to the refund port in accordance with a control signal input from the control unit 101. Note that in addition to the coin mech 22, a payment processing unit for IC card payment or smartphone payment may be provided.

[0046] Printer 25 prints the image data of the photographed photo on a predetermined sheet of paper in accordance with a control signal input from control unit 101. Printer 25 is a high-resolution color printer, and any type of printer, such as dye-sublimation type or melt type, is acceptable.

[0047] The reading device 31 is a device that reads information from an ID card. Specifically, the reading device 31 includes a face scanner 32 that performs an image scan to read information on the face (front and back) of an ID card, and an IC chip reader 33 (such as a non-contact / contact IC reader) that reads data recorded in an IC chip inside the ID card. The face scanner 32 may include a UV / IR scanner that uses ultraviolet (UV) or infrared (IR) as a light source, in addition to a typical image scanner that uses white light as a light source. The IC chip of the ID card has a free area that is a storage area that is not protected by a secret code, and a locked area that is a storage area that is protected by a secret code. The reading device 31 can read data from the free area and the locked area. The reading device 31 inputs image data (hereinafter referred to as face information) of the face of the ID card read by the face scanner 32 to the control unit 101. The reading device 31 also inputs data read by the IC chip reader 33 to the control unit 101.

[0048] Also, instead of reading the ID card with the above-mentioned reader 31, the camera 23 or an IC card reader may be used to read information on the face (front and back) of the ID card. Specifically, (1) the front of the ID card is photographed with the camera 23, the photographed image of the front is subjected to OCR processing to read the character information on the front, and the back of the ID card is read with the IC card reader. Alternatively, (2) the front and back of the ID card are photographed with the camera 23, and the photographed images of the front and back are subjected to OCR processing to read the information on the front and back.

[0049] The document scanner 34 is a scanner that optically reads paper of about A4 size (210 mm×297 mm). The document scanner 34 reads documents used in application procedures, and inputs the read document data to the control unit 101.

[0050] The code reader 35 is a device that reads two-dimensional codes such as QR codes (registered trademark) and bar codes, and inputs the read code information to the control unit 101. The code reader 35 is used, for example, when reading a QR code (authentication code) used in identity verification processing. The identity verification processing will be described later.

[0051] Next, a functional configuration of the control unit 101 of the photographing device 1 will be described. Fig. 4 is a block diagram showing the functional configuration of the control unit 101 of the photographing device 1 in the application mode. As shown in the figure, the control unit 101 includes, as means for realizing the present invention, an identity verification unit 401, an authenticity determination unit 402, an operation instruction unit 403, a biometric determination unit 404, an application image photographing unit 405, an object detection unit 406, a same person determination unit A407, a same person determination unit B408, an image determination unit 409, a visual acuity test unit 411, a PIN setting value input unit 412, a document scan unit 413, a payment processing unit 414, a data output unit 415, and the like.

[0052] The identity verification unit 401 executes identity verification processing using an authentication code. The authentication code is identification information for authentication given by the server 5 to an applicant who is going to carry out an application procedure using the photographing device 1, and is linked to information (unique ID) that uniquely identifies the applicant and includes numerical or character information. The unique ID is converted into a QR code or a barcode and notified to the applicant as an authentication code. The unique ID is not limited to a QR code or a barcode, and may be recorded on an IC card or a magnetic card and used as an authentication code. When the applicant accesses the server 5 from his / her own terminal 6 and inputs predetermined information (personal information such as name and date of birth, application details (what to apply for), information required for the application, etc.), the server 5 issues a unique ID (authentication code) and transmits it to the terminal 6. The server 5 stores the information input from the terminal 6 as registration information in association with the unique ID (authentication code). When the applicant uses the photographing device 1, the applicant manually inputs the value of the authentication code (unique ID) using a soft keyboard, or displays the QR code (authentication code) on the terminal 6 and holds it over the code reader 35. Alternatively, if the authentication code (unique ID) is recorded on an IC card or a magnetic card, the IC card or magnetic card is read by an IC card reader or a magnetic card reader (not shown) provided in the photographing device 1, and the photographing device 1 acquires the value of the authentication code (unique ID). The photographing device 1 accesses the server 5, acquires the registration information stored in association with the authentication code (unique ID) from the server 5, and stores it in a predetermined folder (storage unit 106). The applicant also inserts his / her ID card into the photographing device 1 and has the reader 31 read the information on the ID card. Alternatively, the camera 23, an IC card reader, or the like reads the information on the ID card. The identity verification unit 401 compares the registration information acquired from the server 5 with the information read from the ID card by the reader 31, the camera 23, an IC card reader, or the like, and verifies the identity of the applicant by determining whether the predetermined information, such as the name and date of birth, matches.

[0053] An ID card is a medium for identity verification used by government agencies, financial institutions, mobile phone carriers, etc., and includes, for example, a driver's license, My Number card, residence card, special permanent resident card, etc. In the case of a driver's license, the name of the person, date of birth, address, date of issue, expiration date, name of the ID card issuer, etc. are printed on the face of the card. In addition, some or all of this information is stored as data in the IC chip of the ID card. The memory area of ​​the IC chip has a locked area protected by a secret code such as a PIN, and a free area that is not protected by a secret code.

[0054] The authenticity determination unit 402 performs authenticity determination of the ID card. The authenticity determination unit 402 determines whether the ID card is counterfeit or altered based on information read by the reading device 31 or information acquired by the camera 23 or IC card reader. In this embodiment, the reading device 31 includes a face scanner 32 and an IC chip reader 33, and therefore can scan the face information of the ID card and read data recorded in a recording medium (IC chip) in the ID card. The authenticity determination unit 402 determines the authenticity of the ID card based on the read face information, determines the authenticity of the ID card based on data recorded in the IC chip, determines the authenticity of the ID card based on an image of the face photographed by the camera 23 or information read by the IC card reader, or both of these pieces of information. When data in the lock area of ​​the ID card can be read, the authenticity of the ID card may be determined based on the data stored in the lock area. A specific method of authenticity determination will be described later. The authenticity determination unit 402 stores the determination result in a predetermined folder (storage unit 106). The authenticity determination unit 402 is a function related to the gist of the present invention.

[0055] The operation instructing unit 403 instructs the applicant to perform an operation when the biometric determination unit 404 described later determines whether the applicant is biometric. The operation instructing unit 403 instructs the content of the operation by voice through the audio output unit 27, or displays it on the operation panel 21 as characters, moving images, animations, etc. It is desirable that the operation instructing unit 403 presents a plurality of instructions in a random order from a plurality of operation instructions pre-stored in the storage unit 106. This makes it possible to prevent fraud such as playing a video in which a movement corresponding to an operation instruction is pre-recorded during the biometric determination process. Examples of the operations to be instructed are shown in (1) to (5) below.

[0056] (1) "Please blink slowly." (2) "Please turn slowly to the right (left)" → "Please turn slowly to the front" (3) "Please look down (up) slowly." → "Please look forward slowly." (4) "Please tilt your face to the right (left)." → "Please keep your face straight." (5) "Please open your mouth wide" → "Please close your mouth"

[0057] The liveness determination unit 404 uses the camera 23 to capture a video of the applicant's movements responding to the operation instructions given by the operation instruction unit 403, and analyzes the captured video to determine whether the applicant is a live person. The liveness determination unit 404 performs liveness determination by checking whether there are characteristic live person movements, such as gaze fluctuation or eye opening and closing (blinking), using face recognition technology while detecting each response. The liveness determination unit 404 stores the determination result in a specified folder (storage unit 106).

[0058] The application image photographing unit 405 photographs an application image, which is a photograph used in the application procedure, with the camera 23. When photographing, the application image photographing unit 405 photographs an image (still image) of the applicant's face at a predetermined timing while displaying operation guidance on the operation panel 21 and outputting guidance audio. The application image photographing unit 405 performs processing such as size adjustment, positioning, trimming, and background color setting on the photographed face image according to the purpose to generate an application image. The application image photographing unit 405 stores the photographed application image in a predetermined folder (storage unit 106).

[0059] The object detection unit 406 detects whether or not there is an object covering the face of the person being photographed (applicant) in the application image photographed by the application image photographing unit 405. This detects fraudulent attempts to impersonate another person by holding up a photo of another person's face (a printed piece of paper or a printed object) in front of the face or a tablet or the like displaying another person's face image in front of the face. The object detection unit 406 detects objects by, for example, subtracting the background color from an image photographed at a wide angle before cropping, and performing image processing such as edge detection to detect unnatural corners, frames, and edges. The object detection unit 406 stores the detection results in a specified folder (storage unit 106).

[0060] The same person determination unit A407 compares the applicant's front face image cut out from the video captured in the biometric determination process by the biometric determination unit 404 with the application image captured by the application image capture unit 405 to determine whether the captured person matches. This detects fraudulent switching of the photographed person in the momentary gap between the biometric determination process and the application image capture. The same person determination unit A407 stores the determination result in a specified folder (storage unit 106).

[0061] The same person determination unit B408 compares the image of a person read from the face of the ID card by the reading device 31 or the camera 23, and the image of a person contained in the information read from the IC chip if the image of a person is stored in the IC chip, with the application image captured by the application image capturing unit 405, and determines whether the person in these images is the same person. Furthermore, the biometrics determination unit 404 also compares the applicant's front face images cut out from the video captured during the biometrics determination process, and determines whether the person in all images is the same person. This ensures that the person image contained on the face of the ID card and in the IC chip used for identity verification is that of the applicant. The same person determination unit B408 stores the determination result in a specified folder (storage unit 106). The same person determination unit B408 is a function related to the gist of the present invention.

[0062] The image determination unit 409 determines whether the application image taken by the application image photographing unit 405 is properly taken for the application procedure. This is a function to prevent defects in the photograph. For example, the specifications and precautions regarding passport application photographs are set in international standards, so the photographed image is determined to be in accordance with these. If the result of the determination is that the application image taken is not appropriate, the image is retaken or attention is given to how it is taken. The image determination unit 409 stores the determination result in a specified folder (storage unit 106).

[0063] The visual acuity test section 411 displays the visual acuity test screen 71 on the operation panel 21 and performs a visual acuity test on the applicant. The visual acuity test section 411 displays the measurement results and also stores them in a prescribed folder (storage section 106).

[0064] The password setting value input unit 412 displays an input screen for setting a password (such as a PIN) required for the application procedure. The password input by the applicant via the input screen is stored in a predetermined folder (storage unit 106) together with the application image and the like, and is output to the server 5 by the data output unit 415.

[0065] The document scanning unit 413 reads documents used in the application procedure using the document scanner 34, and stores them in a predetermined folder (storage unit 106). The read document data is output to the server 5 together with application images and the like.

[0066] The payment processing unit 414 calculates the usage fee for the photography device 1 and accepts the payment from the coin mechanism 22. It also calculates the change and dispenses the change from the coin mechanism 22. In the case of payment by IC card or smartphone payment, the payment of the usage fee is made in cooperation with a payment server.

[0067] The data output unit 415 outputs the application image and the attached information (set password information, scanned document data, and other information required for application) to the server 5 when the personal identification unit 401 determines that the person image on the ID card matches the applicant, the authenticity determination unit 402 determines that the ID card is authentic, the biometric determination unit 404 determines that the applicant (photographed person) is a biometric person, the same person determination unit A407 and the same person determination unit B408 determine that the photographed video, still image, and ID card person images match, the object detection unit 406 determines that there is no object covering the face, and the image determination unit 409 determines that the application image is proper. Note that the data output unit 415 preferably outputs the application image and the attached information to the server 5 in association with the unique ID associated with the authentication code. This causes the information required for the application procedure to be sent from the photographing device 1 to the server 5. The server 5 transmits these pieces of information to a computer (server) of an agency that processes electronic applications.

[0068] The data output unit 415 also transmits the result of the determination by the identity verification unit 401, the information read from the ID card, the result of the determination by the authenticity determination unit 402, the result of the determination by the liveness determination unit 404, the detection result by the object detection unit 406, the results of the determination by the same person determination units A407 and B408, the result of the determination by the image determination unit 409, and the measurement result by the eyesight test unit 411 as auxiliary information, linked to the unique ID, together with the application image, etc. This makes it possible to transmit to the server 5 information such as the fact that identity verification was performed using the ID card, that the ID card is not counterfeit or altered, that the applicant is a live person, that the people photographed on the ID card and in the application image match, and also match the person being determined to be live, and that the application image was properly photographed, etc.

[0069] Next, a description will be given of the flow of processing executed by the photographing device 1 of this embodiment. Fig. 5 is a flow chart showing the flow of menu selection processing in the photographing device 1, and Fig. 6 is a flow chart showing the flow of application menu processing.

[0070] When the control unit 101 of the photographing device 1 detects the entry of a user (person to be photographed / applicant) by the object sensor 29 (step S101; Yes), it displays a top screen on the operation panel 21 (step S102). The top screen has buttons for selecting various services, such as an ID photographing service, an application service, and an image printing service.

[0071] When the user operates any of the buttons to select a service (menu) to be executed (step S103; Yes), the control unit 101 proceeds to processing of the selected service (menu) (step S104).

[0072] If the user has an authentication code (QR code, etc.) in advance, the user may input the authentication code (step S105) instead of selecting a service (menu) on the top screen (step S103; No). The authentication code (QR code, barcode, or code information (numeric or text information) recorded on an IC card, magnetic card, etc.) is linked to a unique ID issued by the server 5, and is issued when the user accesses the server 5 in advance from his / her own terminal 6 to select a menu to be used on the photographing device 1, input information required for identity verification, and input information required for processing, and this information is registered in the server 5.

[0073] If the authentication code is issued as a QR code, when the user holds the QR code over the code reader 35 (step S105; Yes), the code reader 35 reads the QR code (step S106). If the reading of the QR code fails (step S107; Yes), the control unit 101 displays a screen notifying an error (step S108) and returns to the top screen (step S102). In the process of step S106, instead of reading the QR code, input of the authentication code value by a soft keyboard or input of the authentication code by an IC card reader or a magnetic card reader may be accepted. When the reading of the authentication code / input of the authentication code is completed (step S107; No), the control unit 101 accesses the server 5, acquires from the server 5 the registration information stored in association with the read QR code (authentication code), and proceeds to a processing menu specified in the registration information (step S109).

[0074] In this embodiment, the application menu will be described. When the application menu is selected by the process in Fig. 5, the control unit 101 executes the application menu process shown in Fig. 6.

[0075] FIG. 6 is a flowchart showing the flow of application menu processing. When moving to the application menu, first, the control unit 101 displays the application content selection screen 51 on the operation panel 21 and waits for an operation. FIG. 7 is a diagram showing an example of the application content selection screen 51. As shown in FIG. 7, the application content selection screen 51 displays selectable application contents. In the example of FIG. 7, each button of "passport application", "driver's license application", and "my number card application" is displayed. Note that the application contents are not limited to these, and may include applications for other identification documents (residence card, special permanent resident card, etc.). In addition, the application may be either a renewal application or a new application. However, only those for which electronic application is officially permitted can be selected.

[0076] When the user selects any application content on the application content selection screen 51, the control unit 101 acquires the value of the authentication code (step S201). When the reading of the QR code / input of the authentication code is completed in step S106 of FIG. 5, the control unit 101 acquires the read code value (unique ID) or the input code value (unique ID). When the reading of the QR code or the input of the authentication code is not performed, the control unit 101 displays the code reading guide screen 52 shown in FIG. 8 or the code input screen 53 shown in FIG. 9 on the operation panel 21 to accept the reading of the QR code or the input of the authentication code value. When the code value (unique ID) is read or input, the control unit 101 displays the code confirmation screen 54 shown in FIG. 10. When the "Yes" button 54b is selected on the code confirmation screen 54, the process proceeds to step S202 of FIG. 6. Note that when the "Back" button 52a, 53a or the cancel button 54a is operated on each screen of FIG. 8 to FIG. 10, the process returns to the application content selection screen 51.

[0077] Next, the photographing device 1 reads the information of the user's ID card. Specifically, the photographing device 1 scans the face of the ID card with the face scanner 32 of the reading device 31 (step S202), and reads the data recorded in the IC chip of the ID card with the IC chip reader 33 (step S204). In addition to the normal image scan using white light, the scan in step S202 may also be an image scan using ultraviolet light / infrared light (UV / IR scan). Instead of scanning the face with the reading device 31, the face (front and back) of the ID card may be photographed with the camera 23, or the data recorded in the IC chip of the ID card may be read with the IC card reader. The information on the ID card may also be read by combining the photographing of the face with the camera 23 and the reading of the IC chip with the IC card reader. If the IC chip has a storage area (locked area) protected by a secret code (PIN, etc.) like a driver's license, input of the secret code is accepted (step S203), and data stored in the free area and the locked area is read (step S204). If the IC chip does not have a storage area (locked area) protected by a secret code or if the user does not input a secret code, only data stored in the free area is read in step S204.

[0078] If the information cannot be read from the IC chip (step S205; Yes), the ID card may be counterfeit or altered, so an error notification is issued (step S206), and then the process returns to the application content selection screen 51. If the information is read from the IC chip (step S205; No), the process proceeds to step S207.

[0079] In step S202, the control unit 101 may display an ID card insertion guide screen 55 shown in Fig. 11 on the operation panel 21. In step S203, the control unit 101 displays a password input screen 56 shown in Fig. 12 on the operation panel 21.

[0080] When the information on the ID card is read by the processes of steps S202 to S204 (step S205; No), the control unit 101 executes an identity verification process using the code value (unique ID) acquired in step S201 (step S207).

[0081] Fig. 13 is a flow chart showing the flow of the identity verification process in step S207. As shown in Fig. 13, first, the control unit 101 downloads the registration information linked to the code value (unique ID) acquired in step S201 from the server 5 (step S301). Note that if the download of the registration information has already been completed by the code reading process in step S106 in Fig. 5 and the registration information has been stored in the RAM 105 or the storage unit 106 of the photography device 1, the control unit 101 acquires the registration information from the RAM 105 or the storage unit 106.

[0082] The control unit 101 compares the acquired registration information with the information read from the ID card (step S302) and judges whether there is a contradiction in the information (step S303). For example, the control unit 101 performs OCR (Optical Character Recognition) processing on the face information read from the ID card to acquire character information and numerical information such as the name and date of birth. If the name and date of birth written on the face of the ID card match the name and date of birth information included in the registration information, it is judged that there is no contradiction (step S303; Yes), and the authentication is successful. If the name and date of birth information on the face of the ID card do not match the name and date of birth information included in the registration information (step S303; No), it is judged that the authentication is unsuccessful. Note that the items compared in step S302 are not limited to the name and date of birth. The comparison may be made based on information stored in the IC chip, such as the date of issue and expiration date of the ID card. The comparison may also be made based on a face image on the ID card. In this case, the same information (issue date, expiration date, facial image, etc.) must be registered in advance in the registration information.

[0083] The control unit 101 acquires the result of the identity verification process, and if the authentication is unsuccessful (step S208 in FIG. 6; No), it displays an error notification screen 57 as shown in FIG. 14 (step S209), and then returns to the application content selection screen 51. The error notification screen 57 displays the content of the error (for example, "The registered information does not match the information on the ID card"). If the authentication is successful (step S208; Yes), the control unit 101 stores the result of the identity verification process (identity verification process "OK") in the folder in which the registration information is registered, and proceeds to step S210.

[0084] In step S210, the control unit 101 executes a process for determining whether the ID card is authentic or counterfeit (step S210).

[0085] FIG. 15 is a flowchart showing the flow of the authenticity determination process. There are various methods for determining the authenticity of an ID card, but as an example, we will explain a case in which authenticity is determined by detecting obvious discrepancies based on the card face information, card face image, and IC chip information read by a reading device 31, camera 23, IC card reader, etc.

[0086] The control unit 101 performs OCR processing on the image information of the face of the ID card read from the ID card by the face scanner 32 or the camera 23 of the reading device 31 (step S401), and judges whether there is a contradiction in the face information. The judgment of the contradiction in the face information is disclosed, for example, in JP 2019-209603 A. Specifically, when the ID card is, for example, a driver's license, the control unit 101 obtains the name, date of birth, address, date of issue of the ID card, expiration date, issuer name, and the like written on the face of the ID card by scanning and OCR, and judges whether there is a contradiction between the content of one character information and the content of another character information, or whether there is a contradiction between the content of the character information and the attribute of the character information on the ID card. For example, if the date of expiration on an ID card is legally required to be one month after the date of birth, and a person forges the card by adding a 2 using a pen to change "June 1, 1975" to "June 21, 1975," it will be determined that there is a discrepancy between the date of birth and the expiration date (the date of expiration is not one month after the date of birth). Also, if the date of birth is an impossible date, such as "1985" or "13th month," it will be determined that there is a discrepancy in the content.

[0087] If no contradiction is detected in the face information, the result is judged as "OK". In this case (step S403; Yes), the control unit 101 compares the information (e.g., expiration date, date of issue) stored in the IC chip of the ID card with the information (expiration date, date of issue) acquired from the face information (step S404). If the comparison results in a match, the result is judged as "OK" (step S405; Yes). Next, the control unit 101 compares the information (name, date of birth, address, etc.) stored in the lock area of ​​the IC chip of the ID card with the information (name, date of birth, address, etc.) acquired from the face information (step S406). If the comparison results in a match, the result is judged as "OK" (step S407; Yes). Next, the control unit 101 judges whether or not there is any fraud in the ID card based on the result of the UV / IR scan of the ID card face (step S408). Traces of replacement of the face of the card may be detected by the UV / IR scan. If no fraud is detected as a result of the UV / IR scan, the control unit 101 judges the ID card to be "genuine" (step S409). If there is a mismatch as a result of the judgment and comparison in any of steps S402, S404, and S406 ("No" in any of steps S403, S405, and S407), or if fraud is detected by the UV / IR scan ("Yes" in step S408), the control unit 101 judges the ID card to be "fake" (step S410).

[0088] In addition, in the case of an ID card that does not have a lock area in the IC chip or in the case where the information of the lock area cannot be read, the control unit 101 may skip the processes of steps S406 to S407. In addition, in the case where the reader 31 does not have a UV / IR scanning function, the process of step S408 is skipped.

[0089] Next, the control unit 101 acquires a face image recorded on the face of the ID card or on the IC chip acquired by the reader 31 or the camera 23 (step S411), and stores the acquired face image and the authenticity determination results (true or false) of steps S409 and S410 in a predetermined folder in the storage unit 106 (step S412). This ends the ID card authenticity determination process.

[0090] If the ID card is judged to be "fake" as a result of the authenticity judgment in Fig. 15 (step S410, step S211 in Fig. 6; No), the control unit 101 displays an error notification screen 57 as shown in Fig. 14 (step S212), and then returns to the application content selection screen 51. The error notification screen 57 displays the content of the error (for example, "An inconsistency has been detected in the description on the ID card" or "Fraud has been detected on the ID card", etc.). If the ID card is judged to be "authentic" (step S211; Yes), the process proceeds to step S213.

[0091] In step S213, the control unit 101 executes a biometrics determination process to determine whether or not the person to be photographed (applicant) is a biometrics person (step S213).

[0092] 16 is a flowchart showing the flow of the biometric determination process. In the biometric determination process, the control unit 101 captures a moving image of the subject (applicant) using the camera 23 (step S501). In addition, the control unit 101 sets a counter n to an initial value "1" (step S502) and issues an n-th operation instruction (step S503).

[0093] In step S503, the control unit 101 acquires operation instruction data pre-stored in the storage unit 106, and performs an operation instruction in a random order from among a plurality of operation instructions a predetermined number of times (N times). This is to prevent fraud, such as recording in advance a video of the movements responding to the operation instructions in the same order if the operation instructions are always performed in the same order, and replaying the recorded video on a tablet terminal or the like. In step S503, the control unit 101 displays an operation instruction screen 58 as shown in Fig. 17 on the operation panel 21, and displays the contents of the operation instructions as animation or text in an instruction content display area 58a on the screen, or outputs the operation instructions as audio.

[0094] The response of the subject to the operation instruction is recorded in a video and displayed in real time in the video display area 58b of the operation instruction screen 58. The control unit 101 determines the response (movement) recorded in the video, detects whether it is a movement according to the operation instruction or a characteristic movement of the living body (such as the fluctuation of the line of sight or blinking) by image analysis, and determines whether the subject is a living body (step S504). Also, when the response is confirmed, the control unit 101 cuts out the face facing the front from the video and saves it as a still image in a predetermined folder of the RAM 105 or the storage unit 106 (step S505). The saved front face image is compared with the front face image obtained during the response to the operation instruction after the next time (step S506).

[0095] The control unit 101 compares the value of the counter n with a predetermined maximum value N. If n < N (step S507; Yes), the value of the counter n is incremented (n + 1) (step S508), returns to step S503, and issues the next operation instruction. The control unit 101 repeats the processing of steps S503 to S506 N times. As a result, the living body determination results for N operation instructions and the collation results of the front face images for each time are obtained. When the N - time processing is completed (step S507; No), the determination result is output (step S509).

[0096] Based on the result of the living body determination process, if the control unit 101 cannot detect the characteristics of the living body from the subject even once, or if the front faces obtained in each operation instruction do not match, it determines "NG". In that case (step S214 in FIG. 6; No), it proceeds to step S215, displays an error notification screen 57 as shown in FIG. 14 (step S215), and then returns to the application content selection screen 51. The error content (for example, "There was a problem with the operation", etc.) is displayed on the error notification screen 57 in step S215.

[0097] Based on the result of the living body determination process, if the characteristics of the living body are detected in the subject and the front faces obtained in each operation instruction match, it determines "OK" (step S214 in FIG. 6; Yes), and proceeds to step S216 in FIG. 18.

[0098] FIG. 18 is a flowchart showing the flow of processing following FIG. 6 (application menu processing). The control unit 101 performs a photographing process of an application image in step S216. An application image is a photograph to be submitted when applying for a passport, a driver's license, a My Number card, etc. When performing a photographing process of an application image, the control unit 101 displays an application image photographing screen 61 shown in FIG. 19 and operation screens 62 and 63 shown in FIG. 20, and guides the user on instructions for photographing and face alignment operations. For example, before starting photographing, the control unit 101 displays the application image photographing screen 61 shown in FIG. 19 on the operation panel 21 and outputs a guidance voice such as "Look at the glass in front" from the voice output unit 27. Thereafter, the control unit 101 photographs a still image of the person to be photographed (applicant) with the camera 23 and stores the image in a predetermined folder in the RAM 105 or the storage unit 106. The photographing is performed with a wide angle with the face as the center, and size adjustment and alignment are performed according to the photo size in a later process. The photographing may be performed multiple times.

[0099] FIG. 20(a) shows an operation screen 62 for adjusting the vertical position of the face, and FIG. 20(b) shows an operation screen 63 for adjusting the horizontal position of the face. A preview image 62a is displayed on the operation screen 62 in FIG. 20(a), and guide lines 62b and 62c indicating the tip position of the head and the tip position of the chin are displayed on the screen. Up and down buttons 62d and 62e are provided for moving the guide lines 62b and 62c up and down, respectively. When the decision button 62f is operated, the operation screen 63 shown in FIG. 20(b) is then displayed. A preview image 63a is displayed on the operation screen 63 in FIG. 20(b), and guide lines 63b indicating the center position of the face are displayed on the screen. A left and right button 63c is provided for moving the guide line 63b left and right. When the decision button 63d is operated, the left and right center position of the face is decided. The control unit 101 then generates an application image of a size according to the application contents.

[0100] Next, the control unit 101 executes object detection processing (step S217). The object detection processing is processing for determining whether the face of the person to be photographed (applicant) is covered by an object. FIG. 21 is a flowchart showing the flow of the object detection processing, and FIG. 22 is a diagram for explaining an image of face-covering fraud and the object detection processing. As shown in FIGS. 22(a) and (b), a possible fraudulent act is to place a face image displayed on a tablet or a printed out face photo in front of the face and take a picture while impersonating another person. To prevent such fraud, the photographing device 1 performs object detection processing.

[0101] As shown in the flowchart of FIG. 21, the control unit 101 acquires an image before trimming of the application image taken in step S214 (step S601). The application image is first taken with a wide angle as shown in FIG. 22(d) and then trimmed as shown in FIG. 22(c). The control unit 101 detects unnatural frames, corners, and edges from the image before trimming acquired in step S601 (step S602). The process of step S602 can be performed, for example, by subtracting the background color from the captured image and performing edge detection processing on the difference image. If an unnatural frame, corner, or edge is detected, an object is detected in front of the face, and the result is determined as "NG". If an unnatural frame or corner is not detected, an object is not detected in front of the face, and the result is determined as "OK". The control unit 101 stores the detection result in a predetermined folder in the RAM 105 or the storage unit 106 (step S603).

[0102] Next, the control unit 101 executes the same person determination process A (step S218 in FIG. 18). FIG. 23 is a flowchart showing the flow of the same person determination process A. In the same person determination process A, the control unit 101 compares the front face image cut out from the video captured in the biometric determination process in step S211 with the application image captured in step S216, and determines whether the person captured is the same person (step S701). The control unit 101 performs face recognition processing, and determines "OK" if the person in the front face image and the application image match, and determines "NG" if they do not match. The control unit 101 stores the determination result in a predetermined folder in the RAM 105 or the storage unit 106 (step S702).

[0103] Next, the control unit 101 executes the same person determination process B (step S219 in FIG. 18). FIG. 24 is a flowchart showing the flow of the same person determination process B. In the same person determination process B, the control unit 101 compares the face image (hereinafter, the ID card face image) included in the face of the ID card and the IC chip acquired in steps S202 to S204 with the application image captured in step S214, and determines whether the person captured is the same person (step S801). Furthermore, the control unit 101 compares the front face image cut out from the video captured in the biometric determination process in step S211, the ID card face image, and the application image, and determines whether the person captured is the same person (step S802). The control unit 101 performs face recognition processing, and if the person in the ID card face image and the application image match, and the person in the front face image also matches, it determines "OK". If there is even one image in which the person does not match, it determines "NG". The control unit 101 stores the determination result in a predetermined folder in the RAM 105 or the storage unit 106 (step S803).

[0104] When the processes of steps S217 to S219 (object detection process, same person determination process A, same person determination process B) are completed, control unit 101 acquires the results of each process ("OK" / "NG") stored in a predetermined folder in RAM 105 or storage unit 106, and if any process is "NG" (step S220; No), it displays error notification screen 57 as shown in Fig. 14 (step S221), and then returns to application content selection screen 51. Error notification screen 57 in step S221 displays the content of the error (for example, "There was a problem with the captured image").

[0105] If the results of the object detection process, same person determination process A, and same person determination process B are all "OK" (step S220; Yes), the process proceeds to step S222.

[0106] The control unit 101 displays a preview of the application image captured and generated in step S216 (step S222), and performs image judgment processing (step S223). In the image judgment processing in step S223, the control unit 101 judges whether the application image is appropriate for application procedures. For example, a plurality of judgment items are set in advance in the storage unit 106, and the control unit 101 judges "OK" / "NG" for each judgment item. The judgment items are, for example, whether there is a pattern on the background, whether the color of the background is appropriate, whether the boundary between the person and the background is clear, whether the pupils are not red, whether the lighting is reflected on the glasses, whether there is a shadow on the face, whether there is a shadow on the background, whether there is out of focus, whether there is blur, whether there is shine or unevenness, etc. In addition, it may be possible to detect whether the subject is tilted to the left or right, facing sideways, being off-center, the head is hidden by a hat or a wide hair band, there is an ornament that hides the face or head organs, the outline of the face is hidden, hair is covering the eyes, color contact lenses are worn, colored glasses or sunglasses, or the frame of glasses is covering the eyes, etc. If any of the judgment items is "NG" (step S224; No), an error notification screen 57 is displayed (step S225), and then the process returns to the biometric judgment process (video shooting) of step S213 (FIG. 6). The error notification screen 57 of step S225 displays the contents of the error (for example, "There was a problem with the captured image") and the contents of the judgment NG item, so as to alert the subject to caution when shooting. If all judgment items of the image judgment process are "OK" (step S224; Yes), the process proceeds to step S226 of FIG. 25.

[0107] FIG. 25 is a flow chart showing the process following FIG. 18. The control unit 101 executes the visual acuity test process (step S226). In step S226, the control unit 101 displays the visual acuity test screen 71 as shown in FIG. 26(a) on the operation panel 21. The visual acuity test screen 71 displays an index 71a for visual acuity testing such as a Landolt ring, and operation buttons such as an up button 71b, a down button 71c, a left button 71d, and a right button 71e, and accepts the applicant's response operation. For example, in the case of FIG. 26(a), since the upward direction of the Landolt ring is open, if the up button 71b is operated, it is the correct answer. If the correct answer is obtained with a predetermined standard value or more (step S227; Yes), the measurement result screen 72 as shown in FIG. 26(b) is displayed (step S231), and the measurement result is stored in a predetermined folder in the RAM 105 or the storage unit 106. If the measurement result does not reach the predetermined standard value (step S227; No), if it is the first test (step S228; No), a guide to remeasure is displayed on the screen (step S229), and then the test is performed again (step S226). If the measurement result does not reach the predetermined standard value even after multiple repetitions (step S228; Yes), an error is notified (step S230), and the screen returns to the application content selection screen 51 (step S201 in FIG. 6).

[0108] If the vision test confirms that the vision is equal to or higher than the standard value (step S227; Yes), the control unit 101 then accepts input of a secret information setting value (step S232). The value entered here is the secret information (e.g., PIN) value to be set on the ID card being applied for. The control unit 101 displays a secret information setting screen 73 as shown in FIG. 27 on the operation panel 21. When the applicant enters a value (secret information) into the secret information setting screen 73, the control unit 101 displays a confirmation screen for the input contents (step S233), and if a confirmation response of "OK" is obtained (step S234; Yes), the control unit 101 stores the entered value in a predetermined folder in the storage unit 106 or in the RAM 105, and proceeds to step S235. If an "OK" response is not obtained on the confirmation screen of step S233 (step S234; No), the process returns to step S232 and accepts re-entry of the secret information.

[0109] After the password is entered, the control unit 101 then accepts the scanning of the application documents (step S235). In step S235, the control unit 101 displays a reading guidance screen 74 as shown in FIG. 28(a). The reading guidance screen 74 displays a guidance message such as "Please read the application documents" and the position of the document insertion slot of the document scanner 34. When a document is set in the document scanner 34, the control unit 101 starts the reading operation by the document scanner 34. When the reading of the document is completed, the control unit 101 displays a confirmation screen 75 of the scan result as shown in FIG. 28(b) (step S236). The confirmation screen 75 displays the contents 75a read by the document scanner 34, and displays an "OK" button 75b and a "Back" button 75c. When the "OK" button 75b is operated by the applicant (step S237; Yes), the control unit 101 stores the read content as document data in a specified folder in the RAM 105 or the storage unit 106, and proceeds to the process of step S239. When the "Back" button 75c is operated (step S237; No), the control unit 101 guides the applicant to rescan by displaying on the screen or outputting audio (step S238), and returns to step S235 to rescan the document.

[0110] When scanning of the application documents is completed, the control unit 101 then displays an application content confirmation screen 76 on the operation panel 21 (step S239). The example shown in Fig. 29 is an application content confirmation screen 76 for a My Number card. The application content confirmation screen 76 displays an application date 76a, an application ID 76b, an application image 76c, a request 76d for an electronic certificate or Braille notation, etc., a change button 76e for the electronic certificate or Braille notation, etc., an application button 76f, a cancel button 76g, etc. The application date 76a and application ID 76b are issued by the control unit 101. The application image 76c is the application image photographed and generated in step S216 of the above-mentioned process. The request 76d for electronic certificate, braille notation, etc. is registered information that the applicant has registered in advance in the server 5 or information included in the scanned document data. When the change button 76e is operated, the screen transitions to a setting screen for electronic certificate, braille notation, etc., and changes to the request are accepted.

[0111] When the applicant confirms the application details on the application details confirmation screen 76 and operates the Apply button 76f (step S240; Yes), the process proceeds to the payment process in step S241. When the cancel button 76g is operated (step S240; No), the process returns to the application details selection screen 51 (step S201).

[0112] In the payment process in step S241, the control unit 101 calculates the usage fee and accepts the payment into the coin mechanism 22. The control unit 101 also calculates the change and dispenses the change from the coin mechanism 22. When payment is made with a transportation IC card or the like or by smartphone, the control unit 101 cooperates with the payment server to settle the usage fee (step S241).

[0113] When the payment process is completed, the control unit 101 generates data by linking the information required for the application with the application ID 76b and the unique ID linked to the authentication code, and outputs the data to the server 5 (step S242). The information required for the application is preset depending on the application contents, and for example, in the case of an application for a driver's license, the information includes an application image, as well as ancillary information such as the results of the eyesight test, a password setting value, and scanned document data. In addition, the personal identification information (information read from the ID card) acquired in steps S202 to S204, the authenticity determination result of the ID card, and the like are also output as ancillary information to the server 5 together with the information required for the application. Thereafter, the control unit 101 displays an end screen 77 as shown in FIG. 30, and ends the application menu process.

[0114] The processing contents of the application menu are not limited to the above contents and can be changed appropriately depending on the application contents. For example, an application for a driver's license includes a vision test and a password setting, but an application for a My Number card does not require a vision test. In addition, an application for a passport does not require a vision test and a password setting, and the image judgment process is set as a judgment item that produces an image suitable for a passport photo. In addition, an electronic signature screen may be displayed, an input of an electronic signature may be accepted, and the input electronic signature may be transmitted to a server together with an application image and information required for the application. In addition, a breathalyzer may be provided in the photographing device 1 to detect the alcohol concentration in the breath of the applicant, and the result may be transmitted to a server together with an application image and information required for the application. In addition, a biometric authentication device that reads the biometric information of the applicant and performs authentication processing based on the read biometric information may be provided in the photographing device 1, and the result of the biometric authentication may be transmitted to a server together with an application image and information required for the application. Examples of biometric authentication devices that can be used include a fingerprint authentication device that reads the applicant's fingerprint information and performs authentication processing based on the read fingerprint information, a vein authentication device that reads the applicant's vein information and performs authentication processing based on the read vein information, and an iris authentication device that reads the applicant's iris information and performs authentication processing based on the read iris information.

[0115] As described above, the photographing device 1 of the present invention reads the information on the applicant's ID card using the reading device 31, the camera 23, an IC card reader, etc. Also, the camera 23 takes an application image, which is a photograph used in the application procedure. The control unit 101 of the photographing device 1 compares the photographed application image with the person image included in the information of the ID card read by the reading device 31 or the camera 23, and judges whether they are the same person (same person determination process B). The control unit 101 also judges the authenticity of the ID card (authenticity determination process). If the same person determination process judges that the person image included in the face or IC chip of the ID card and the person in the photographed application image are the same person, and if the authenticity determination process judges that the ID card is authentic, the application image is output to a predetermined server 5. This makes it possible to perform strict identity verification and prevent fraud in electronic applications. This allows electronic applications for administrative procedures, etc. to be realized by the photographing device 1. By using an image taken with the photographing device 1 rather than a home PC or smartphone for the application, an appropriate image for application can be provided.

[0116] Although the preferred embodiment of the present invention has been described above with reference to the accompanying drawings, the present invention is not limited to the above examples. It is clear that a person skilled in the art can come up with various modified or altered examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally belong to the technical scope of the present invention. [Explanation of symbols]

[0117] 1. Photography device 5. Server 6. Terminal 7. Network 10. Control device 21 Operation panel 22. Coin Mech 23. Camera 24. Lighting section 25 Printer 26...Printed matter outlet 27 Audio output section 28. Curtains 29. Object sensor 31 Reading device 32. Card scanner 33. IC chip reader 34 Document scanner 35 Code reader 51 Application content selection screen 52 Code reading guide screen 53 Code entry screen 54 Code confirmation screen 55 ID card insertion guide screen 56 PIN entry screen 57 Error notification screen 58······Operation instruction screen 61 Application image capture screen 62, 63...Operation screen 71 Vision test screen 72...Measurement result screen 73 Password setting screen 74 Reading guide screen 75 Confirmation screen of scanned document data 76... Application details confirmation screen 77....End screen 101... Control unit 401...Identification section 402 Authenticity determination unit 403...Movement instruction section 404... Biometric determination unit 405: Application image capture section 406...Object detection unit 407... Same person determination section A 408... Same person determination section B 409...Image judgment unit 411 Vision Testing Department 412: Password setting value input section 413 Document Scanner 414: Payment processing unit 415 Data output section

Claims

1. A photographing device equipped with a camera for photographing an applicant as a subject, an identity verification means for receiving an input of an authentication code that has been given to the applicant by a predetermined server; A reading means for reading information on the applicant's identity verification medium; an action instruction means for instructing the applicant to take an action; a liveness determination means for capturing a video of the applicant's movements in response to an action instruction given by the action instruction means, and analyzing the captured video to determine whether the applicant is a live person; an application image capturing means for capturing an application image, which is a photograph to be used in the application procedure, using the camera; a person identification means for comparing a person image included in the information read by the reading means with the application image and determining whether or not they are the same person; an authenticity determining means for determining the authenticity of the personal identification medium based on the information read by the reading means; an output means for outputting the application image to the predetermined server when the same person determination means determines that the two persons are the same person and the authenticity determination means determines that the personal identification medium is authentic; Equipped with the identity verification means accepts input of an authentication code before the reading means reads information from the identity verification medium of the applicant; The same person determination means further compares a frontal facial image of the applicant cut out from a video captured during the biometric determination process by the biometric determination means with the application image to determine whether they are the same person.

2. a registration information acquiring means for acquiring registration information that the applicant has registered in advance in the server and that is stored in the server in association with the authentication code after the identity verification means accepts the input of the authentication code; Equipped with 2. The photographing device according to claim 1, wherein the identity verification means performs identity verification of the applicant by comparing the registration information acquired by the registration information acquisition means with the information read by the reading means.

3. 3. The photographing device according to claim 1, wherein the biometric determination means performs biometric determination by checking whether or not there is a characteristic movement of a living body while detecting the applicant's response to an operational instruction by the operational instruction means.

4. The reading means includes at least one of a scanner that reads face information in the personal identification medium, a camera that takes an image of the face of the personal identification medium, and a reader that reads data recorded on a recording medium in the personal identification medium; A photographing device as described in any one of claims 1 to 3, characterized in that the authenticity determination means determines the authenticity of the identity verification medium based on at least one of the card face information read by the scanner, the image of the card face photographed by the camera, and the data read by the reader.

5. An input means for inputting a password for the personal identification medium, The reader reads the data stored in the storage area protected by the secret information by using the secret information input by the input means, 5. The photographing device according to claim 4, wherein said authenticity determining means further determines the authenticity of said personal identification medium based on data stored in a storage area protected by said password.

6. 6. The photographing device according to claim 1, further comprising a biometric authentication device that reads biometric information of the applicant and performs authentication processing based on the read biometric information.

7. The application process further includes a document scanner for reading documents used in the application process, 7. The photographing device according to claim 1, wherein the output means outputs the document data read by the document scanner together with the application image to the predetermined server.

8. A setting means for setting a password required for the application procedure is provided, The output means outputs the secret information set by the setting means to the predetermined server together with the application image.

8. The photographing apparatus according to claim 1, wherein the first and second photographing means are arranged to be arranged in a direction parallel to the first and second photographing directions.

9. A method for controlling a photographing device equipped with a camera that photographs an applicant as a subject, comprising the steps of: a code value acquisition step of accepting an input of an authentication code previously assigned to the applicant by a predetermined server; a reading step of reading information from the applicant's identity verification medium after the code value acquisition step; an operation instruction step of instructing the applicant to perform an operation; a liveness determination step of capturing a video of the applicant's movements in response to an operational command by the camera and analyzing the captured video to determine whether the applicant is a live person; An application image capturing step of capturing an application image, which is a photograph to be used in the application procedure, by the camera; a first same person determination step of comparing a person image included in the information read in the reading step with the application image to determine whether or not they are the same person; a second same person determination step of comparing a front face image of the applicant cut out from a moving image captured in the liveness determination process in the liveness determination step with the application image to determine whether or not they are the same person; an authenticity determining step of determining authenticity of the personal identification medium based on the information read by the reading step; an output step of outputting the application image to the predetermined server when it is determined that the person is the same person by the first same person determination step and the second same person determination step, and when it is determined that the personal identification medium is genuine by the authenticity determination step; 13. A method for controlling a photographing apparatus, comprising:

10. The method further includes a step of acquiring a unique ID, which is information associated with the authentication code received in the code value acquisition step, 10. The method of claim 9, wherein the output step includes linking the application image with the unique ID and transmitting the image to the server.

11. 11. The method according to claim 10, wherein said output step further comprises the step of linking additional information with said application image and said unique ID and transmitting the information to said server.

12. A control device for a photographing device equipped with a camera for photographing the applicant as a subject, an identity verification means for receiving an input of an authentication code previously assigned to the applicant by a predetermined server; A reading means for reading information on the applicant's identity verification medium; an operation instruction means for instructing the applicant to perform an operation; a liveness determination means for capturing a video of the applicant's movements in response to an action instruction given by the action instruction means, and analyzing the captured video to determine whether the applicant is a live person; an application image capturing means for capturing an application image, which is a photograph to be used in the application procedure, using the camera; a person identification means for comparing a person image included in the information read by the reading means with the application image and determining whether or not they are the same person; an authenticity determining means for determining the authenticity of the personal identification medium based on the information read by the reading means; an output means for outputting the application image to the predetermined server when the same person determination means determines that the two persons are the same person and the authenticity determination means determines that the identity verification medium is authentic; A program for causing the device to function as a the identity verification means accepts input of an authentication code before the reading means reads information from the identity verification medium of the applicant; The same person determination means further includes a program for comparing a frontal facial image of the applicant extracted from a video captured during the biometric determination process by the biometric determination means with the application image to determine whether they are the same person.

Images