PHOTOGRAPHING APPARATUS, ITS CONTROL METHOD, AND PROGRAM

Abstract

To provide a photographing device capable of detecting unauthorized photographing and realizing an electronic application using the photographing device.SOLUTION: A photographing device 1 issues an operation instruction to a subject to be photographed (operation instruction unit 403), captures, in a video, the subject to be photographed in response to the operation instruction, and analyzes the captured video to determines whether or not the subject to be photographed is a living body (living body determination unit 404). In addition, the photographing device 1 photographs an application image, which is a picture used for an application procedure (application image photographing unit 405). The photographing device 1 compares the image clipped from the video with the application image to determine whether or not the photographed subjects matches (same person determination unit A407). The photographing device 1 outputs the application image to a predetermined server 5 when it is determined that the subject to be photographed is a living body and that the subject to be photographed who is responding to the operation instruction matches the subject to be photographed in the application image (data outputting unit 415).SELECTED DRAWING: Figure 4

Description

[Technical field]

[0001] The present invention relates to a photographing device, a control method thereof, and a program, and more particularly to electronic applications for administrative procedures and the like. [Background technology]

[0002] Electronic application schemes using smartphones and other devices are beginning to penetrate the market. For example, there are plans to digitize application procedures and realize them online for new passport / renewal applications, My Number card renewal applications, driver's license renewal applications, and other applications. Applications require identity verification using a form of identification (identification card) such as a driver's license, but this is not done face-to-face, and electronic applications using personal devices raise concerns about fraudulent activity such as impersonation.

[0003] Incidentally, Patent Document 1 discloses a technology for determining whether an ID card (medium for personal identification) such as a driver's license is counterfeit. The authenticity determination device in Patent Document 1 is composed of a reader that reads an ID card and a computer that performs authenticity determination, and determines whether there is a contradiction in the contents indicated by the character information obtained from the card face image, thereby determining the authenticity of the ID card. [Prior art documents] [Patent documents]

[0004] [Patent Document 1] JP 2019-209603 A [Non-patent literature]

[0005] [Patent Document 1] Ministry of Foreign Affairs, "Roadmap for digitalization of administrative procedures", [online], searched on December 15, 2020, Internet,<URL: http: / / www.moj.go.jp / content / 001318660.pdf> Summary of the Invention [Problem to be solved by the invention]

[0006] However, as mentioned above, electronic applications have a higher risk of data tampering than face-to-face application procedures, making it difficult to strictly verify the identity of the person. In addition, when electronic applications are made using the consumer's own smartphone or PC, it is difficult to require a proper photograph. In fact, according to the "Roadmap for Digitization of Administrative Procedures" in Non-Patent Document 1, the use of smartphones and home PCs is not permitted for electronic applications for next-generation passports. Even when photographs are taken using a photographing device, there are cases where the person being photographed wears a highly accurate mask or displays an image of another person on a tablet and appears in the photograph.

[0007] The present invention has been made in consideration of the above problems, and has an object to provide a photographing device and the like that can detect fraudulent photography and realize electronic application. [Means for solving the problem]

[0008] In order to solve the above-mentioned problems, a first invention is a photographing device equipped with a camera that photographs an applicant as a subject, the photographing device comprising: an action instruction means for issuing action instructions to the subject; a liveness determination means for taking a video of the movements of the subject responding to the action instructions with the camera and analyzing the taken video to determine whether the subject is a live person or not; an application image photographing means for taking an application image which is a photograph used in an application procedure with the camera; an identical person determination means for comparing an image cut out from the video with the application image to determine whether the photographed person matches; and an output means for outputting the application image to a predetermined server when the liveness determination means determines that the subject is a live person and the same person determination means determines that the persons match. a reading means for reading information on the identity verification medium of the applicant, a registration information acquisition means for acquiring registration information that the applicant has registered in advance in the server, and an identity verification means for verifying the identity of the applicant by comparing the registration information acquired by the registration information acquisition means with the information read by the reading means. The photographing device is characterized in that

[0009] According to the first invention, the photographing device issues an instruction to the person to be photographed, captures a video of the person's movements responding to the instruction, and analyzes the captured video to determine whether the person is a living body. The photographing device also captures an application image, which is a photograph used in the application procedure. The photographing device compares an image cut out from the video with the application image to determine whether the person photographed matches. If it is determined that the person to be photographed is a living body and that the person responding to the instruction matches the person in the application image, the application image is output to a specified server.

[0010] This makes it possible to confirm whether the person photographed by the photographing device is a living body and whether the person being determined to be living body matches the person in the application image, thereby preventing fraud in electronic applications and realizing electronic applications for administrative procedures, etc. For example, it is possible to detect fraudulent photography in which the face is covered with a face photo printed on paper or a tablet, a skin film, a mask, etc., and the like to impersonate another person. In addition, by using an image taken with a photographing device instead of a PC or smartphone at home for the application, a strictly proper image for the application can be provided. Furthermore, the applicant can efficiently apply for the photo shooting device using the information that he / she registered in advance from his / her home PC, etc. At that time, the identity of the person who registered the information and the applicant can be verified using the identity verification medium, thereby preventing fraudulent applications.

[0011] In the first aspect of the present invention, the action instruction means randomly presents a plurality of pre-stored action instructions, thereby preventing fraudulent acts such as playing a video in which a motion corresponding to an action instruction is recorded in advance during the liveness determination process.

[0012] The system may further include a scanner for reading face information of the applicant's identity verification medium, and the same person determination means may compare a person image included in the face information read by the scanner with an image cut out from the video and the application image to determine whether they are the same person. This allows for more rigorous identity verification, since it can be confirmed whether the person image on the identity verification medium, the person being identified, and the person in the application image are the same person.

[0013] It is desirable to further provide a notification means for notifying an error when the liveness determination means determines that the person to be photographed is not a live body or when the same person determination means determines that the persons do not match.

[0014] It is preferable that the application image photographing means photographs a wide angle image and further comprises an object detection means for detecting an object covering the face of the person to be photographed from the photographed image. This makes it possible to detect an object covering the face of the person to be photographed and detect unauthorized photographing.

[0015] It is also preferable to further include an image determination unit that determines whether the application image is appropriately taken for the application procedure, thereby making it possible to reliably provide an application image suitable for the application procedure.

[0016] The apparatus may further comprise a visual acuity testing means for displaying a visual acuity test screen and conducting a visual acuity test on the applicant. This allows applications requiring a visual acuity test, such as a driver's license, to be submitted using the photographing device.

[0017] It is preferable that the application procedure further includes a document scanner for reading documents used in the application procedure, and the output means outputs the document data read by the document scanner together with the application image to the predetermined server. This makes it possible to set the result of identity verification and the images and documents required for the application and perform an electronic application.

[0018] It is preferable that the application procedure is performed by a setting means for setting a secret code required for the application procedure, and the output means outputs the secret code set by the setting means to the predetermined server together with the application image. This makes it possible to input a secret code set at the time of application, such as a secret code for a driver's license, into the photographing device.

[0020] A second aspect of the present invention is a method for controlling a photographing device equipped with a camera that photographs an applicant as a subject, comprising: a reading step of reading information from an identity verification medium of the applicant; a registration information acquisition step of acquiring registration information previously registered by the applicant in a predetermined server; and an identity verification step of verifying the identity of the applicant by comparing the registration information acquired in the registration information acquisition step with the information read in the reading step.an action instruction step of issuing action instructions to the person to be photographed; a liveness determination step of using the camera to capture a video of the movements of the person to be photographed in response to the action instructions and analyzing the captured video to determine whether or not the person to be photographed is a live body; an application image capture step of using the camera to capture an application image which is a photograph to be used in the application procedure; an identical person determination step of comparing an image cut out from the video with the application image to determine whether or not the photographed person matches; and when it is determined that the person to be photographed is a live body by the liveness determination step and that the persons match by the identical person determination step, The above and an output step of outputting the image data to a predetermined server.

[0021] The second invention makes it possible to confirm whether the person photographed by the photographing device is a living body and whether the person being determined to be living body matches the person in the application image, thereby preventing fraud in electronic applications and realizing electronic applications for administrative procedures, etc. For example, it is possible to detect fraudulent photography in which the face is covered with a facial photo output on paper, a tablet, etc., a skin film, a mask, etc., and the like to impersonate another person. In addition, by using an image taken with a photographing device instead of a PC or smartphone at home for the application, a strictly proper image for the application can be provided.

[0022] A third invention provides a control device for a photography device equipped with a camera for photographing an applicant as a subject, the control device functioning as: an action instruction means for issuing action instructions to the subject; a liveness determination means for taking a video of the movements of the subject responding to the action instructions with the camera and analyzing the taken video to determine whether the subject is a live person or not; an application image capture means for taking an application image which is a photograph used in application procedures with the camera; an identical person determination means for comparing an image cut out from the video with the application image to determine whether the photographed person matches; and an output means for outputting the application image to a predetermined server when the liveness determination means determines that the subject is a live person and the same person determination means determines that the persons match. a reading means for reading information from the applicant's identity verification medium, a registration information acquisition means for acquiring registration information that the applicant has registered in advance in the server, and an identity verification means for verifying the identity of the applicant by comparing the registration information acquired by the registration information acquisition means with the information read by the reading means. This is a program for

[0023] According to the third invention, it is possible to cause a computer to function as a control device for the photographic device of the first invention. Effect of the Invention

[0024] According to the present invention, it is possible to detect unauthorized photography and realize electronic applications using a photography device. [Brief description of the drawings]

[0025] [Figure 1] FIG. 1 is a diagram showing the external configuration of a photographing device 1. [Diagram 2] FIG. 1 is a diagram showing a hardware configuration of a photographing device 1. [Diagram 3] FIG. 1 is a diagram showing an example of a communication connection between a photographing device 1 and a server 5. [Figure 4] FIG. 1 shows a functional configuration of a control unit 101 of a photographing device 1. [Diagram 5] A flowchart showing the flow of a menu selection process executed by the photographing device 1. [Figure 6] Flowchart showing the flow of application menu processing [Figure 7] FIG. 13 is a diagram showing an example of an application content selection screen 51. [Figure 8] FIG. 13 is a diagram showing an example of a code reading guide screen 52. [Figure 9] FIG. 13 is a diagram showing an example of a code input screen 53. [Figure 10] FIG. 13 shows an example of a code confirmation screen 54. [Figure 11] FIG. 13 is a diagram showing an example of an ID card insertion guide screen 55. [Figure 12] FIG. 13 shows an example of a screen 56 for inputting personal identification information on an ID card. [Figure 13] Flowchart showing the flow of identity verification processing [Figure 14] FIG. 13 is a diagram showing an example of an error notification screen 57. [Figure 15] Flowchart showing the flow of authenticity determination processing [Figure 16]Flowchart showing the flow of a liveness determination process [Figure 17] FIG. 13 shows an example of an operation instruction screen 58. [Figure 18] Flowchart showing the flow of processing following FIG. 6 (application menu processing) [Figure 19] FIG. 13 is a diagram showing an example of an application image capture screen 61. [Figure 20] FIG. 13 shows an example of operation screens 62 and 63 for aligning photographs. [Figure 21] Flowchart showing the flow of object detection processing [Figure 22] Image of a fraudster covering his face and a diagram explaining the object detection process [Diagram 23] Flowchart showing the flow of the same person determination process A [Figure 24] Flowchart showing the flow of the same person determination process B [Diagram 25] Flowchart showing the flow of processing following FIG. 18 (application menu processing) [Figure 26] FIG. 7 shows an example of a visual acuity test screen 71 and a measurement result screen 72. [Figure 27] FIG. 7 shows an example of a password setting screen 73. [Figure 28] FIG. 7 shows an example of (a) an application document reading guidance screen 74 and (b) a confirmation screen 75 for the read document data. [Figure 29] FIG. 76 is a diagram showing an example of an application content confirmation screen 76. [Diagram 30] A diagram showing an example of an end screen 77. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0026] A preferred embodiment of the present invention will be described in detail below with reference to the drawings. The photographing device 1 according to the present invention includes a stationary type photographing device for ID photography installed at a storefront, a station, on the street, etc., and a stationary type photographing device used for taking photographs for purposes other than ID photography.

[0027] FIG. 1 is a diagram showing the external configuration of the photographing device 1, and FIG. 2 is a diagram showing the hardware configuration of the photographing device 1. As shown in FIG. 1, the photographing device 1 is a box-type device or the like that allows a person to enter the inside of a housing that is a photographing room to photograph and operate. The form of the photographing device 1 is not limited to the box type, and may be an open type or a stand type. The photographing device 1 has a function of photographing a person in a photographing room as a subject, generating and printing an ID photo according to the purpose from the image (ID photo menu), and a function of printing an image received from a terminal 6 such as a user's smartphone, or downloading and printing an image that the user has previously uploaded from the terminal 6 to the server 5 (image print menu). In the present invention, the photographing device 1 has an application menu for carrying out application procedures for a driver's license, a My Number card, a residence card, a special permanent resident card, a passport, etc. A user can select and use one of the ID photo menu, the image print menu, the application menu, and other menus.

[0028] The photographing device 1 shown in the example of Fig. 1 is a box type with an open side, and the outside is separated from the inside (photography room) by a curtain 28. On the front panel of the photography room, an operation panel 21, a coin mechanism 22, a camera 23, a lighting unit 24, an audio output unit 27, a subject sensor 29, a reader 31 for an identity verification medium (hereinafter referred to as an ID card), a document scanner 34, a code reader 35, etc. are provided. In addition, inside the housing of the photographing device 1, a control device 10, a printer 25, etc. are stored, and printed materials printed by the printer 25 are discharged from a printed material outlet 26 provided on an outer wall panel of the photographing device 1.

[0029] Fig. 2 is a block diagram showing the hardware configuration of the photographing device 1. As shown in Fig. 2, the control device 10 of the photographing device 1 is connected to the lighting unit 24, the camera 23, the object sensor 29, the audio output unit 27, the operation panel 21, the coin mechanism 22, the printer 25, the reading device 31, the document scanner 34, the code reader 35, etc. via a bus, and the control device 10 controls each unit.

[0030] The control device 10 has a control unit (CPU; Central Processing Unit) 101, an I / O (Input / Output) control 102, an I / F (Interface) 103, a ROM (Read Only Memory) 104, a RAM (Random Access Memory) 105, a storage unit 106, a communication unit 107, etc. The control device 10 is connected to an operation panel 21 and an audio output unit 27 via the I / O control 102. The control device 10 is also connected to a coin mechanism 22, a printer 25, a reading device 31, a document scanner 34, and a code reader 35 via the I / F 103.

[0031] The control unit 101 is a CPU and centrally controls each unit of the photography device 1. The control unit 101 loads programs stored in the storage unit 106, ROM 104, etc. into a work area on the RAM 105 and executes them. The ROM 104 is a non-volatile memory and permanently stores programs such as a boot program and BIOS, data, etc. The RAM 105 is a volatile memory and temporarily stores programs and data loaded from the storage unit 106, ROM 104, etc., and also includes a work area used by the control unit 101 to perform various processes.

[0032] The storage unit 106 is a hard disk drive, a solid state drive, a flash memory, or the like, and stores programs executed by the control unit 101 during processing described below, data necessary for executing the programs, an OS, etc. These programs and data are read and executed by the control unit 101 as necessary.

[0033] The I / O control 102 is an interface for inputting and outputting various types of information to and from the operation panel 21, the audio output unit 27, and the like. The I / F 103 is an interface for inputting and outputting various types of information between the coin mechanism 22, the printer 25, the reading device 31, the document scanner 34, and the code reader 35.

[0034] The communication unit 107 includes a communication control device and a communication interface that control communication of the photography device 1. Fig. 3 is a diagram for explaining a communication connection between the photography device 1 and the server 5. The photography device 1 is equipped with a wireless LAN such as Wi-Fi (registered trademark) or a short-range wireless communication such as Bluetooth (registered trademark), and is communicatively connected to the server 5 via a wide area network 7 such as the Internet.

[0035] The server 5 is a server that manages the photographing device 1. When information required for identity verification or application is input from the user's terminal 6 via a web browser or an app, the server 5 issues a predetermined code and registers the input information in association with the code. The server 5 has a storage area for storing various data (photographic images and other data, etc.) uploaded from the photographing device 1.

[0036] A user can download image data of a photograph taken by the photographing device 1 from the server 5 to a terminal 6 such as a PC or a smartphone via the network 7. In addition, an image uploaded from the user's terminal 6 to the server 5 via the network 7 can be downloaded to the photographing device 1 and printed (image print menu).

[0037] 1 and 2. Illumination unit 24 is a surface light source located in front of or behind the subject. Illumination unit 24 is connected to light source (strobe) 241, and the timing and amount of light emission are controlled according to a control signal input from control unit 101.

[0038] Camera 23 is a digital camera composed of an optical lens, an imaging element such as a CCD (Charge Coupled Device) or a CMOS (Complementary Metal-Oxide Semiconductor), an A / D (Analog / Digital) converter, etc. Camera 23 captures moving images or still images in accordance with a control signal input from control unit 101. Camera 23 inputs image data obtained by capturing images to control unit 101. Control unit 101 displays and outputs the image data input from camera 23 on the display of operation panel 21.

[0039] The operation panel 21 is an operation means of the photography device 1, and has a display for displaying an operation screen showing an operation procedure, a preview screen, etc., as well as a touch panel for a user to input operation instructions. The operation panel 21 performs display processing according to a display signal input from the control unit 101. Also, the operation panel 21 sends a control signal to the control unit 101 according to a touch operation on the touch panel.

[0040] The object sensor 29 is a sensor that detects a person in the photography room. When a person approaches the front panel of the photography device 1, the object sensor 29 outputs a detection signal to the control unit 101. As the object sensor 29, an optical sensor consisting of a light emitter and a light receiver is used.

[0041] The audio output unit 27 acquires predetermined audio data (operation guidance audio, etc.) from the ROM 104 or the storage unit 106 in accordance with a control signal input from the control unit 101, and outputs the audio from a speaker.

[0042] The coin mech 22 is a cash register equipped with a deposit port for coins and bills, a refund port for change, etc., and performs processing for settling the usage fee for the photography device 1. The coin mech 22 outputs information on the amount deposited from the deposit port to the control unit 101, and discharges coins and bills according to the change amount to the refund port in accordance with a control signal input from the control unit 101. Note that in addition to the coin mech 22, a payment processing unit for IC card payment or smartphone payment may be provided.

[0043] Printer 25 prints the image data of the photographed photo on a predetermined sheet of paper in accordance with a control signal input from control unit 101. Printer 25 is a high-resolution color printer, and any type of printer, such as dye-sublimation type or melt type, is acceptable.

[0044] The reading device 31 is a device that reads information from an ID card. Specifically, the reading device 31 includes a face scanner 32 that performs an image scan to read information on the face (front and back) of an ID card, and an IC chip reader 33 (such as a non-contact / contact IC reader) that reads data recorded in an IC chip inside the ID card. The face scanner 32 may include a UV / IR scanner that uses ultraviolet (UV) or infrared (IR) as a light source, in addition to a typical image scanner that uses white light as a light source. The IC chip of the ID card has a free area that is a storage area that is not protected by a secret code, and a locked area that is a storage area that is protected by a secret code. The reading device 31 can read data from the free area and the locked area. The reading device 31 inputs image data (hereinafter referred to as face information) of the face of the ID card read by the face scanner 32 to the control unit 101. The reading device 31 also inputs data read by the IC chip reader 33 to the control unit 101.

[0045] The document scanner 34 is a scanner that optically reads paper of about A4 size (210 mm×297 mm). The document scanner 34 reads documents used in application procedures, and inputs the read document data to the control unit 101.

[0046] The code reader 35 is a device that reads two-dimensional codes such as QR codes (registered trademark) and bar codes, and inputs the read code information to the control unit 101. The code reader 35 is used, for example, when reading a QR code (authentication code) used in identity verification processing. The identity verification processing will be described later.

[0047] Next, a functional configuration of the control unit 101 of the photographing device 1 will be described. Fig. 4 is a block diagram showing the functional configuration of the control unit 101 of the photographing device 1 in the application mode. As shown in the figure, the control unit 101 includes, as means for realizing the present invention, an identity verification unit 401, an authenticity determination unit 402, an operation instruction unit 403, a biometric determination unit 404, an application image photographing unit 405, an object detection unit 406, a same person determination unit A407, a same person determination unit B408, an image determination unit 409, a visual acuity test unit 411, a PIN setting value input unit 412, a document scan unit 413, a payment processing unit 414, a data output unit 415, and the like.

[0048] The identity verification unit 401 executes identity verification processing using the authentication code. The authentication code is identification information for authentication given by the server 5 to an applicant who is going to carry out an application procedure using the photographing device 1, and includes numerical information or character information. The authentication code may be converted into a QR code or a barcode and notified to the applicant. When the applicant accesses the server 5 from his / her own terminal 6 and inputs predetermined information (personal information such as name and date of birth, application details (what is being applied for), information necessary for the application, etc.), the server 5 issues an authentication code and transmits it to the terminal 6. The server 5 stores the information input from the terminal 6 as registration information in association with the authentication code. When the applicant uses the photographing device 1, he / she inputs the value of the authentication code or displays the QR code (authentication code) on the terminal 6 and holds it over the code reader 35. Then, the photographing device 1 acquires the value of the authentication code. The photographing device 1 accesses the server 5, acquires the registration information stored in association with the authentication code from the server 5, and stores it in a predetermined folder (storage unit 106). The applicant also inserts his / her ID card into the photographing device 1 and has the information on the ID card read by the reading device 31. The identity verification unit 401 compares the registration information acquired from the server 5 with the information read from the ID card by the reading device 31, and verifies the identity of the applicant by determining whether or not predetermined information such as the name and date of birth match.

[0049] An ID card is a medium for identity verification used by government agencies, financial institutions, mobile phone carriers, etc., and includes, for example, a driver's license, My Number card, residence card, special permanent resident card, etc. In the case of a driver's license, the name of the person, date of birth, address, date of issue, expiration date, name of the ID card issuer, etc. are printed on the face of the card. In addition, some or all of this information is stored as data in the IC chip of the ID card. The memory area of ​​the IC chip has a locked area protected by a secret code such as a PIN, and a free area that is not protected by a secret code.

[0050] The authenticity determination unit 402 performs authenticity determination of the ID card inserted into the photographing device 1. The authenticity determination unit 402 determines whether the ID card is counterfeit or altered based on the information read by the reading device 31. In this embodiment, the reading device 31 includes the face scanner 32 and the IC chip reader 33, and therefore can scan the face information of the ID card and read the data recorded in the recording medium (IC chip) in the ID card. The authenticity determination unit 402 determines the authenticity of the ID card based on the read face information, determines the authenticity of the ID card based on the data recorded in the IC chip, or determines the authenticity of the ID card based on both of these pieces of information. When the data in the lock area of ​​the ID card can be read, the authenticity of the ID card may be determined based on the data stored in the lock area. A specific method of authenticity determination will be described later. The authenticity determination unit 402 stores the determination result in a predetermined folder (storage unit 106).

[0051] The operation instructing unit 403 instructs the applicant to perform an operation when the biometric determination unit 404 described later determines whether the applicant is biometric. The operation instructing unit 403 instructs the content of the operation by voice through the audio output unit 27, or displays it on the operation panel 21 as characters, moving images, animations, etc. It is desirable that the operation instructing unit 403 presents a plurality of instructions in a random order from a plurality of operation instructions pre-stored in the storage unit 106. This makes it possible to prevent fraud such as playing a video in which a movement corresponding to an operation instruction is pre-recorded during the biometric determination process. Examples of the operations to be instructed are shown in (1) to (5) below.

[0052] (1) "Please blink slowly." (2) "Please turn slowly to the right (left)" → "Please turn slowly to the front" (3) "Please look down (up) slowly." → "Please look forward slowly." (4) "Please tilt your face to the right (left)." → "Please keep your face straight." (5) "Please open your mouth wide" → "Please close your mouth"

[0053] The liveness determination unit 404 uses the camera 23 to capture a video of the applicant's movements responding to the operation instructions given by the operation instruction unit 403, and analyzes the captured video to determine whether the applicant is a live person. The liveness determination unit 404 performs liveness determination by checking whether there are characteristic live person movements, such as gaze fluctuation or eye opening and closing (blinking), using face recognition technology while detecting each response. The liveness determination unit 404 stores the determination result in a specified folder (storage unit 106). The biometric determination unit 404 is a function related to the gist of the present invention.

[0054] The application image photographing unit 405 photographs an application image, which is a photograph used in the application procedure, with the camera 23. When photographing, the application image photographing unit 405 photographs an image (still image) of the applicant's face at a predetermined timing while displaying operation guidance on the operation panel 21 and outputting guidance audio. The application image photographing unit 405 performs processing such as size adjustment, positioning, trimming, and background color setting on the photographed face image according to the purpose to generate an application image. The application image photographing unit 405 stores the photographed application image in a predetermined folder (storage unit 106).

[0055] The object detection unit 406 detects whether or not there is an object covering the face of the person being photographed (applicant) in the application image photographed by the application image photographing unit 405. This detects fraudulent attempts to impersonate another person by holding up a photo of another person's face (a printed piece of paper or a printed object) in front of the face or a tablet or the like displaying another person's face image in front of the face. The object detection unit 406 detects objects by, for example, subtracting the background color from an image photographed at a wide angle before cropping, and performing image processing such as edge detection to detect unnatural corners, frames, and edges. The object detection unit 406 stores the detection results in a specified folder (storage unit 106).

[0056] The same person determination unit A407 compares the applicant's front face image cut out from the video captured in the biometric determination process by the biometric determination unit 404 with the application image captured by the application image capture unit 405 to determine whether the captured person matches. This detects fraudulent switching of the photographed person in the momentary gap between the biometric determination process and the application image capture. The same person determination unit A407 stores the determination result in a specified folder (storage unit 106). The same person determination unit A407 is a function related to the gist of the present invention.

[0057] The same person determination unit B408 compares the person image read from the face of the ID card by the reading device 31, and if a person image is stored in the IC chip, the person image contained in the information read from the IC chip with the application image captured by the application image capturing unit 405, and determines whether the person in these images is the same person. Furthermore, the biometrics determination unit 404 also compares the applicant's front face images cut out from the video captured during the biometrics determination process, and determines whether the person in all images is the same person. This ensures that the person image contained on the face of the ID card and in the IC chip used for identity verification is that of the applicant. The same person determination unit B408 stores the determination result in a specified folder (storage unit 106).

[0058] The image determination unit 409 determines whether the application image taken by the application image photographing unit 405 is properly taken for the application procedure. This is a function to prevent defects in the photograph. For example, the specifications and precautions regarding passport application photographs are set in international standards, so the photographed image is determined to be in accordance with these. If the result of the determination is that the application image taken is not appropriate, the image is retaken or attention is given to how it is taken. The image determination unit 409 stores the determination result in a specified folder (storage unit 106).

[0059] The visual acuity test section 411 displays the visual acuity test screen 71 on the operation panel 21 and performs a visual acuity test on the applicant. The visual acuity test section 411 displays the measurement results and also stores them in a prescribed folder (storage section 106).

[0060] The password setting value input unit 412 displays an input screen for setting a password (such as a PIN) required for the application procedure. The password input by the applicant via the input screen is stored in a predetermined folder (storage unit 106) together with the application image and the like, and is output to the server 5 by the data output unit 415.

[0061] The document scanning unit 413 reads documents used in the application procedure using the document scanner 34, and stores them in a predetermined folder (storage unit 106). The read document data is output to the server 5 together with application images and the like.

[0062] The payment processing unit 414 calculates the usage fee for the photography device 1 and accepts the payment from the coin mechanism 22. It also calculates the change and dispenses the change from the coin mechanism 22. In the case of payment by IC card or smartphone payment, the payment of the usage fee is made in cooperation with a payment server.

[0063] The data output unit 415 outputs the application image, the set password, scanned document data, etc. to the server 5 when the personal identification unit 401 judges that the person image on the ID card matches the applicant, the authenticity determination unit 402 judges that the ID card is authentic, the biometric determination unit 404 judges that the applicant (photographed person) is a biometric person, the same person determination unit A407 and the same person determination unit B408 judge that the photographed video, still image, and ID card person image match, the object detection unit 406 judges that there is no object covering the face, and the image determination unit 409 judges that the application image is proper. This causes the photographing device 1 to send information necessary for the application procedure to the server 5. The server 5 transmits this information to the computer (server) of the electronic application processing agency.

[0064] The data output unit 415 also transmits, together with the application image, the determination result by the identity verification unit 401, information read from the ID card, the determination result by the authenticity determination unit 402, the determination result by the liveness determination unit 404, the detection result by the object detection unit 406, the determination results by same person determination units A407 and B408, the determination result by the image determination unit 409, and the measurement result by the eyesight test unit 411. This makes it possible to transmit to the server 5 information such as the fact that identity verification has been performed using the ID card, that the ID card is not counterfeit or altered, that the applicant is a live person, that the people photographed on the ID card and in the application image match, and also match the person being subjected to liveness determination, and that the application image has been properly photographed.

[0065] Next, a description will be given of the flow of processing executed by the photographing device 1 of this embodiment. Fig. 5 is a flow chart showing the flow of menu selection processing in the photographing device 1, and Fig. 6 is a flow chart showing the flow of application menu processing.

[0066] When the control unit 101 of the photographing device 1 detects the entry of a user (person to be photographed / applicant) by the object sensor 29 (step S101; Yes), it displays a top screen on the operation panel 21 (step S102). The top screen has buttons for selecting various services, such as an ID photographing service, an application service, and an image printing service.

[0067] When the user operates any of the buttons to select a service (menu) to be executed (step S103; Yes), the control unit 101 proceeds to processing of the selected service (menu) (step S104).

[0068] If the user already has an authentication code (such as a QR code), the user may hold the QR code over the code reader 35 (step S105) instead of selecting a service (menu) on the top screen (step S103; No). The authentication code (QR code, barcode, or numerical or textual information, etc.) is an authentication code issued by the server 5, and is issued when the user has previously accessed the server 5 from his / her own terminal 6 to select a menu to be used in the photographing device 1, input information required for identity verification, and input information required for processing, and this information has been registered in the server 5.

[0069] When the user holds the QR code over the code reader 35 (step S105; Yes), the code reader 35 reads the QR code (step S106). When the reading of the QR code fails (step S107; Yes), the control unit 101 displays a screen notifying an error (step S108) and returns to the top screen (step S102). Note that in the process of step S106, instead of reading the QR code, input of an authentication code value may be accepted. When reading of the QR code / input of the authentication code is completed (step S107; No), the control unit 101 accesses the server 5, acquires from the server 5 the registration information stored in association with the read QR code (authentication code), and proceeds to a processing menu specified in the registration information (step S109).

[0070] In this embodiment, the application menu will be described. When the application menu is selected by the process in Fig. 5, the control unit 101 executes the application menu process shown in Fig. 6.

[0071] FIG. 6 is a flowchart showing the flow of application menu processing. When moving to the application menu, first, the control unit 101 displays the application content selection screen 51 on the operation panel 21 and waits for an operation. FIG. 7 is a diagram showing an example of the application content selection screen 51. As shown in FIG. 7, the application content selection screen 51 displays selectable application contents. In the example of FIG. 7, each button of "passport application", "driver's license application", and "my number card application" is displayed. Note that the application contents are not limited to these, and may include applications for other identification documents (residence card, special permanent resident card, etc.). In addition, the application may be either a renewal application or a new application. However, only those for which electronic application is officially permitted can be selected.

[0072] When the user selects any application content on the application content selection screen 51, the control unit 101 acquires the value of the authentication code (step S201). When the reading of the QR code / input of the authentication code is completed in step S106 of FIG. 5, the control unit 101 acquires the read code value or the input code value. When the reading of the QR code or the input of the authentication code is not performed, the control unit 101 displays the code reading guide screen 52 shown in FIG. 8 or the code input screen 53 shown in FIG. 9 on the operation panel 21 to accept the reading of the QR code or the input of the authentication code value. When the code value is read or input, the control unit 101 displays the code confirmation screen 54 shown in FIG. 10. When the "Yes" button 54b is selected on the code confirmation screen 54, the process proceeds to step S202 of FIG. 6. Note that when the "Back" button 52a, 53a or the cancel button 54a is operated on each screen of FIG. 8 to FIG. 10, the process returns to the application content selection screen 51.

[0073] Next, the photographing device 1 reads information on the user's ID card. Specifically, the photographing device 1 scans the face of the ID card with the face scanner 32 of the reading device 31 (step S202), and reads data recorded in the IC chip of the ID card with the IC chip reader 33 (step S204). In addition to the normal image scan using white light, the scan in step S202 may also be an image scan using ultraviolet light / infrared light (UV / IR scan). If the IC chip has a memory area (lock area) protected by a secret code (PIN, etc.) like a driver's license, the input of the secret code is accepted (step S203), and data stored in the free area and the lock area is read (step S204). If the IC chip does not have a memory area (lock area) protected by a secret code or if the user does not input a secret code, only data stored in the free area is read in step S204.

[0074] If the information cannot be read from the IC chip (step S205; Yes), the ID card may be counterfeit or altered, so an error notification is issued (step S206), and then the process returns to the application content selection screen 51. If the information is read from the IC chip (step S205; No), the process proceeds to step S207.

[0075] In step S202, the control unit 101 may display an ID card insertion guide screen 55 shown in Fig. 11 on the operation panel 21. In step S203, the control unit 101 displays a password input screen 56 shown in Fig. 12 on the operation panel 21.

[0076] When the information on the ID card is read by the processes of steps S202 to S204 (step S205; No), the control unit 101 executes an identity verification process using the code value acquired in step S201 (step S207).

[0077] Fig. 13 is a flow chart showing the flow of the identity verification process in step S207. As shown in Fig. 13, first, the control unit 101 downloads the registration information linked to the code value acquired in step S201 from the server 5 (step S301). Note that if the download of the registration information has already been completed by the code reading process in step S106 in Fig. 5 and the registration information has been stored in the RAM 105 or the storage unit 106 of the photography device 1, the control unit 101 acquires the registration information from the RAM 105 or the storage unit 106.

[0078] The control unit 101 compares the acquired registration information with the information read from the ID card (step S302) and judges whether there is a contradiction in the information (step S303). For example, the control unit 101 performs OCR (Optical Character Recognition) processing on the face information read from the ID card to acquire character information and numerical information such as the name and date of birth. If the name and date of birth written on the face of the ID card match the name and date of birth information included in the registration information, it is judged that there is no contradiction (step S303; Yes), and the authentication is successful. If the name and date of birth information on the face of the ID card do not match the name and date of birth information included in the registration information (step S303; No), it is judged that the authentication is unsuccessful. Note that the items compared in step S302 are not limited to the name and date of birth. The comparison may be made based on information stored in the IC chip, such as the date of issue and expiration date of the ID card. The comparison may also be made based on a face image on the ID card. In this case, the same information (issue date, expiration date, facial image, etc.) must be registered in advance in the registration information.

[0079] The control unit 101 acquires the result of the identity verification process, and if the authentication is unsuccessful (step S208 in FIG. 6; No), it displays an error notification screen 57 as shown in FIG. 14 (step S209), and then returns to the application content selection screen 51. The error notification screen 57 displays the content of the error (for example, "The registered information does not match the information on the ID card"). If the authentication is successful (step S208; Yes), the control unit 101 stores the result of the identity verification process (identity verification process "OK") in the folder in which the registration information is registered, and proceeds to step S210.

[0080] In step S210, the control unit 101 executes a process for determining whether the ID card is authentic or counterfeit (step S210).

[0081] FIG. 15 is a flowchart showing the flow of the authenticity determination process. There are various methods for determining the authenticity of an ID card, but as an example, we will explain a method of determining authenticity by detecting obvious discrepancies based on the card face information and IC chip information read by the reading device 31.

[0082] The control unit 101 performs OCR processing on the face information read from the ID card (step S401) and determines whether there is a contradiction in the face information. The determination of a contradiction in the face information is disclosed, for example, in JP 2019-209603 A. Specifically, when the ID card is, for example, a driver's license, the control unit 101 obtains the name, date of birth, address, date of issue of the ID card, expiration date, issuer name, and the like written on the face of the ID card by scanning and OCR, and determines whether there is a contradiction between the content of one character information and the content of another character information, or whether there is a contradiction between the content of the character information and the attribute of the character information on the ID card. For example, if the date of expiration on an ID card is legally required to be one month after the date of birth, and a person forges the card by adding a 2 using a pen to change "June 1, 1975" to "June 21, 1975," it will be determined that there is a discrepancy between the date of birth and the expiration date (the date of expiration is not one month after the date of birth). Also, if the date of birth is an impossible date, such as "1985" or "13th month," it will be determined that there is a discrepancy in the content.

[0083] If no contradiction is detected in the face information, the result is judged as "OK". In this case (step S403; Yes), the control unit 101 compares the information (e.g., expiration date, date of issue) stored in the IC chip of the ID card with the information (expiration date, date of issue) acquired from the face information (step S404). If the comparison results in a match, the result is judged as "OK" (step S405; Yes). Next, the control unit 101 compares the information (name, date of birth, address, etc.) stored in the lock area of ​​the IC chip of the ID card with the information (name, date of birth, address, etc.) acquired from the face information (step S406). If the comparison results in a match, the result is judged as "OK" (step S407; Yes). Next, the control unit 101 judges whether or not there is any fraud in the ID card based on the result of the UV / IR scan of the ID card face (step S408). Traces of replacement of the face of the card may be detected by the UV / IR scan. If no fraud is detected as a result of the UV / IR scan, the control unit 101 judges the ID card to be "genuine" (step S409). If there is a mismatch as a result of the judgment and comparison in any of steps S402, S404, and S406 ("No" in any of steps S403, S405, and S407), or if fraud is detected by the UV / IR scan ("Yes" in step S408), the control unit 101 judges the ID card to be "fake" (step S410).

[0084] In addition, in the case of an ID card that does not have a lock area in the IC chip or in the case where the information of the lock area cannot be read, the control unit 101 may skip the processes of steps S406 to S407. In addition, in the case where the reader 31 does not have a UV / IR scanning function, the process of step S408 is skipped.

[0085] Next, the control unit 101 acquires a face image from the face or IC chip of the ID card scanned by the reading device 31 (step S411), and stores the acquired face image and the authenticity determination results (true or false) of steps S409 and S410 in a predetermined folder in the storage unit 106 (step S412). This ends the ID card authenticity determination process.

[0086] If the ID card is judged to be "fake" as a result of the authenticity judgment in Fig. 15 (step S410, step S211 in Fig. 6; No), the control unit 101 displays an error notification screen 57 as shown in Fig. 14 (step S212), and then returns to the application content selection screen 51. The error notification screen 57 displays the content of the error (for example, "An inconsistency has been detected in the description on the ID card" or "Fraud has been detected on the ID card", etc.). If the ID card is judged to be "authentic" (step S211; Yes), the process proceeds to step S213.

[0087] In step S213, the control unit 101 executes a biometrics determination process to determine whether or not the person to be photographed (applicant) is a biometrics person (step S213).

[0088] 16 is a flowchart showing the flow of the biometric determination process. In the biometric determination process, the control unit 101 captures a moving image of the subject (applicant) using the camera 23 (step S501). In addition, the control unit 101 sets a counter n to an initial value "1" (step S502) and issues an n-th operation instruction (step S503).

[0089] In step S503, the control unit 101 acquires operation instruction data pre-stored in the storage unit 106, and performs an operation instruction in a random order from among a plurality of operation instructions a predetermined number of times (N times). This is to prevent fraud, such as recording in advance a video of the movements responding to the operation instructions in the same order if the operation instructions are always performed in the same order, and replaying the recorded video on a tablet terminal or the like. In step S503, the control unit 101 displays an operation instruction screen 58 as shown in Fig. 17 on the operation panel 21, and displays the contents of the operation instructions as animation or text in an instruction content display area 58a on the screen, or outputs the operation instructions as audio.

[0090] The response of the subject to the operation instruction is recorded in a video and displayed in real time in the video display area 58b of the operation instruction screen 58. The control unit 101 determines the response (movement) recorded in the video, detects whether it is a movement according to the operation instruction or a characteristic movement of the living body (such as the fluctuation or blink of the line of sight) by image analysis, and determines whether the subject is a living body (step S504). Also, when the response is confirmed, the control unit 101 cuts out the face facing the front from the video and saves it as a still image in a predetermined folder of the RAM 105 or the storage unit 106 (step S505). The saved front face image is compared with the front face image obtained during the response to the operation instruction after the next time (step S506).

[0091] The control unit 101 compares the value of the counter n with a predetermined maximum value N. If n < N (step S507; Yes), the value of the counter n is incremented (n + 1) (step S508), returns to step S503, and issues the next operation instruction. The control unit 101 repeats the processing of steps S503 to S506 N times. Thereby, the biometric determination results in N operation instructions and the collation results of the front face images for each time are obtained. When the N - time processing is completed (step S507; No), the determination result is output (step S509).

[0092] Based on the result of the biometric determination process, if the control unit 101 cannot detect the characteristics of the living body from the subject even once, or if the front faces obtained in each operation instruction do not match, it determines "NG". In that case (step S214 in FIG. 6; No), it proceeds to step S215, displays an error notification screen 57 as shown in FIG. 14 (step S215), and then returns to the application content selection screen 51. The error content (for example, "There was a problem with the operation", etc.) is displayed on the error notification screen 57 in step S215.

[0093] Based on the result of the biometric determination process, if the characteristics of the living body are detected in the subject and the front faces obtained in each operation instruction match, it determines "OK" (step S214 in FIG. 6; Yes), and proceeds to step S216 in FIG. 18.

[0094] FIG. 18 is a flowchart showing the flow of processing following FIG. 6 (application menu processing). The control unit 101 performs a photographing process of an application image in step S216. An application image is a photograph to be submitted when applying for a passport, a driver's license, a My Number card, etc. When performing a photographing process of an application image, the control unit 101 displays an application image photographing screen 61 shown in FIG. 19 and operation screens 62 and 63 shown in FIG. 20, and guides the user on instructions for photographing and face alignment operations. For example, before starting photographing, the control unit 101 displays the application image photographing screen 61 shown in FIG. 19 on the operation panel 21 and outputs a guidance voice such as "Look at the glass in front" from the voice output unit 27. Thereafter, the control unit 101 photographs a still image of the person to be photographed (applicant) with the camera 23 and stores the image in a predetermined folder in the RAM 105 or the storage unit 106. The photographing is performed with a wide angle with the face as the center, and size adjustment and alignment are performed according to the photo size in a later process. The photographing may be performed multiple times.

[0095] FIG. 20(a) shows an operation screen 62 for adjusting the vertical position of the face, and FIG. 20(b) shows an operation screen 63 for adjusting the horizontal position of the face. A preview image 62a is displayed on the operation screen 62 in FIG. 20(a), and guide lines 62b and 62c indicating the tip position of the head and the tip position of the chin are displayed on the screen. Up and down buttons 62d and 62e are provided for moving the guide lines 62b and 62c up and down, respectively. When the decision button 62f is operated, the operation screen 63 shown in FIG. 20(b) is then displayed. A preview image 63a is displayed on the operation screen 63 in FIG. 20(b), and guide lines 63b indicating the center position of the face are displayed on the screen. A left and right button 63c is provided for moving the guide line 63b left and right. When the decision button 63d is operated, the left and right center position of the face is decided. The control unit 101 then generates an application image of a size according to the application contents.

[0096] Next, the control unit 101 executes object detection processing (step S217). The object detection processing is processing for determining whether the face of the person to be photographed (applicant) is covered by an object. FIG. 21 is a flowchart showing the flow of the object detection processing, and FIG. 22 is a diagram for explaining an image of face-covering fraud and the object detection processing. As shown in FIGS. 22(a) and (b), a possible fraudulent act is to place a face image displayed on a tablet or a printed out face photo in front of the face and take a picture while impersonating another person. To prevent such fraud, the photographing device 1 performs object detection processing.

[0097] As shown in the flowchart of FIG. 21, the control unit 101 acquires an image before trimming of the application image taken in step S214 (step S601). The application image is first taken with a wide angle as shown in FIG. 22(d) and then trimmed as shown in FIG. 22(c). The control unit 101 detects unnatural frames and corners from the image before trimming acquired in step S601 (step S602). The process of step S602 can be performed, for example, by subtracting the background color from the captured image and performing edge detection processing on the difference image. If an unnatural frame, corner, or edge is detected, an object is detected in front of the face, and the result is determined as "NG". If an unnatural frame or corner is not detected, an object is not detected in front of the face, and the result is determined as "OK". The control unit 101 stores the detection result in a predetermined folder in the RAM 105 or the storage unit 106 (step S603).

[0098] Next, the control unit 101 executes the same person determination process A (step S218 in FIG. 18). FIG. 23 is a flowchart showing the flow of the same person determination process A. In the same person determination process A, the control unit 101 compares the front face image cut out from the video captured in the biometric determination process in step S211 with the application image captured in step S216, and determines whether the person captured is the same person (step S701). The control unit 101 performs face recognition processing, and determines "OK" if the person in the front face image and the application image match, and determines "NG" if they do not match. The control unit 101 stores the determination result in a predetermined folder in the RAM 105 or the storage unit 106 (step S702).

[0099] Next, the control unit 101 executes the same person determination process B (step S219 in FIG. 18). FIG. 24 is a flowchart showing the flow of the same person determination process B. In the same person determination process B, the control unit 101 compares the face image (hereinafter, the ID card face image) included in the face of the ID card and the IC chip acquired in steps S202 to S204 with the application image captured in step S214, and determines whether the person captured is the same person (step S801). Furthermore, the control unit 101 compares the front face image cut out from the video captured in the biometric determination process in step S211, the ID card face image, and the application image, and determines whether the person captured is the same person (step S802). The control unit 101 performs face recognition processing, and if the person in the ID card face image and the application image match, and the person in the front face image also matches, it determines "OK". If there is even one image in which the person does not match, it determines "NG". The control unit 101 stores the determination result in a predetermined folder in the RAM 105 or the storage unit 106 (step S803).

[0100] When the processes of steps S217 to S219 (object detection process, same person determination process A, same person determination process B) are completed, control unit 101 acquires the results of each process ("OK" / "NG") stored in a predetermined folder in RAM 105 or storage unit 106, and if any process is "NG" (step S220; No), it displays error notification screen 57 as shown in Fig. 14 (step S221), and then returns to application content selection screen 51. Error notification screen 57 in step S221 displays the content of the error (for example, "There was a problem with the captured image").

[0101] If the results of the object detection process, same person determination process A, and same person determination process B are all "OK" (step S220; Yes), the process proceeds to step S222.

[0102] The control unit 101 displays a preview of the application image captured and generated in step S216 (step S222), and performs image judgment processing (step S223). In the image judgment processing in step S223, the control unit 101 judges whether the application image is appropriate for application procedures. For example, a plurality of judgment items are set in advance in the storage unit 106, and the control unit 101 judges "OK" / "NG" for each judgment item. The judgment items are, for example, whether there is a pattern on the background, whether the color of the background is appropriate, whether the boundary between the person and the background is clear, whether the pupils are not red, whether the lighting is reflected on the glasses, whether there is a shadow on the face, whether there is a shadow on the background, whether there is out of focus, whether there is blur, whether there is shine or unevenness, etc. In addition, it may be possible to detect whether the subject is tilted to the left or right, facing sideways, being off-center, the head is hidden by a hat or a wide hair band, there is an ornament that hides the face or head organs, the outline of the face is hidden, hair is covering the eyes, color contact lenses are worn, colored glasses or sunglasses, or the frame of glasses is covering the eyes, etc. If any of the judgment items is "NG" (step S224; No), an error notification screen 57 is displayed (step S225), and then the process returns to the biometric judgment process (video shooting) of step S213 (FIG. 6). The error notification screen 57 of step S225 displays the contents of the error (for example, "There was a problem with the captured image") and the contents of the judgment NG item, so as to alert the subject to caution when shooting. If all judgment items of the image judgment process are "OK" (step S224; Yes), the process proceeds to step S226 of FIG. 25.

[0103] FIG. 25 is a flow chart showing the process following FIG. 18. The control unit 101 executes the visual acuity test process (step S226). In step S226, the control unit 101 displays the visual acuity test screen 71 as shown in FIG. 26(a) on the operation panel 21. The visual acuity test screen 71 displays an index 71a for visual acuity testing such as a Landolt ring, and operation buttons such as an up button 71b, a down button 71c, a left button 71d, and a right button 71e, and accepts the applicant's response operation. For example, in the case of FIG. 26(a), since the upward direction of the Landolt ring is open, if the up button 71b is operated, it is the correct answer. If the correct answer is obtained with a predetermined standard value or more (step S227; Yes), the measurement result screen 72 as shown in FIG. 26(b) is displayed (step S231), and the measurement result is stored in a predetermined folder in the RAM 105 or the storage unit 106. If the measurement result does not reach the predetermined standard value (step S227; No), if it is the first test (step S228; No), a guide to remeasure is displayed on the screen (step S229), and then the test is performed again (step S226). If the measurement result does not reach the predetermined standard value even after multiple repetitions (step S228; Yes), an error is notified (step S230), and the screen returns to the application content selection screen 51 (step S201 in FIG. 6).

[0104] If the vision test confirms that the vision is equal to or higher than the standard value (step S227; Yes), the control unit 101 then accepts input of a secret information setting value (step S232). The value entered here is the secret information (e.g., PIN) value to be set on the ID card being applied for. The control unit 101 displays a secret information setting screen 73 as shown in FIG. 27 on the operation panel 21. When the applicant enters a value (secret information) into the secret information setting screen 73, the control unit 101 displays a confirmation screen for the input contents (step S233), and if a confirmation response of "OK" is obtained (step S234; Yes), the control unit 101 stores the entered value in a predetermined folder in the storage unit 106 or in the RAM 105, and proceeds to step S235. If an "OK" response is not obtained on the confirmation screen of step S233 (step S234; No), the process returns to step S232 and accepts re-entry of the secret information.

[0105] After the password is entered, the control unit 101 then accepts the scanning of the application documents (step S235). In step S235, the control unit 101 displays a reading guidance screen 74 as shown in FIG. 28(a). The reading guidance screen 74 displays a guidance message such as "Please read the application documents" and the position of the document insertion slot of the document scanner 34. When a document is set in the document scanner 34, the control unit 101 starts the reading operation by the document scanner 34. When the reading of the document is completed, the control unit 101 displays a confirmation screen 75 of the scan result as shown in FIG. 28(b) (step S236). The confirmation screen 75 displays the contents 75a read by the document scanner 34, and displays an "OK" button 75b and a "Back" button 75c. When the "OK" button 75b is operated by the applicant (step S237; Yes), the control unit 101 stores the read content as document data in a specified folder in the RAM 105 or the storage unit 106, and proceeds to the process of step S239. When the "Back" button 75c is operated (step S237; No), the control unit 101 guides the applicant to rescan by displaying on the screen or outputting audio (step S238), and returns to step S235 to rescan the document.

[0106] When scanning of the application documents is completed, the control unit 101 then displays an application content confirmation screen 76 on the operation panel 21 (step S239). The example shown in Fig. 29 is an application content confirmation screen 76 for a My Number card. The application content confirmation screen 76 displays an application date 76a, an application ID 76b, an application image 76c, a request 76d for an electronic certificate or Braille notation, etc., a change button 76e for the electronic certificate or Braille notation, etc., an application button 76f, a cancel button 76g, etc. The application date 76a and application ID 76b are issued by the control unit 101. The application image 76c is the application image photographed and generated in step S216 of the above-mentioned process. The request 76d for electronic certificate, braille notation, etc. is registered information that the applicant has registered in advance in the server 5 or information included in the scanned document data. When the change button 76e is operated, the screen transitions to a setting screen for electronic certificate, braille notation, etc., and changes to the request are accepted.

[0107] When the applicant confirms the application details on the application details confirmation screen 76 and operates the Apply button 76f (step S240; Yes), the process proceeds to the payment process in step S241. When the cancel button 76g is operated (step S240; No), the process returns to the application details selection screen 51 (step S201).

[0108] In the payment process in step S241, the control unit 101 calculates the usage fee and accepts the payment into the coin mechanism 22. The control unit 101 also calculates the change and dispenses the change from the coin mechanism 22. When payment is made with a transportation IC card or the like or by smartphone, the control unit 101 cooperates with the payment server to settle the usage fee (step S241).

[0109] When the payment process is completed, the control unit 101 generates data by linking the information required for the application with the application ID 76b, and outputs the data to the server 5 (step S242). The information required for the application is set in advance depending on the application content, and for example, in the case of an application for a driver's license, the information includes an application image, eyesight test results, a password setting value, scanned document data, etc. In addition, the personal identification information (information read from the ID card) acquired in steps S202 to S204, the authenticity determination result of the ID card, etc. are also output to the server 5 together with the information required for the application. Thereafter, the control unit 101 displays an end screen 77 as shown in FIG. 30, and ends the application menu process.

[0110] The processing contents of the application menu are not limited to the above, and can be changed as appropriate depending on the application contents. For example, a driver's license application includes a vision test and setting of a password, but an application for a My Number card does not require a vision test. Also, a passport application does not require a vision test or setting of a password, and the image judgment process is set as a judgment item to produce an image suitable for a passport photo.

[0111] As described above, the photographing device 1 of the present invention issues an operation instruction to the person to be photographed (operation instruction unit 403), captures a video of the movement of the person to be photographed responding to the operation instruction by the camera 23, and analyzes the captured video to determine whether the person to be photographed is a living body (living body determination unit 404). The photographing device 1 also captures an application image, which is a photograph used in application procedures (application image capture unit 405). The photographing device 1 compares an image cut out from the video with the application image to determine whether the photographed person matches (identical person determination unit A407). If it is determined that the person to be photographed is a living body and that the person to be photographed responding to the operation instruction matches the person to be photographed in the application image, the application image is output to a predetermined server 5 (data output unit 415). This makes it possible to confirm whether the person to be photographed by the photographing device 1 is a living body and whether it matches the person in the application image, thereby preventing fraud in electronic applications and realizing electronic applications for administrative procedures, etc. For example, it can detect fraudulent photographs in which the face is covered with a face photo printed on paper or a tablet, or with a skin film, mask, etc., to impersonate another person. In addition, by using an image taken with the photography device 1, rather than a PC or smartphone at home, for application, a proper image for application can be provided.

[0112] Although the preferred embodiment of the present invention has been described above with reference to the accompanying drawings, the present invention is not limited to the above examples. It is clear that a person skilled in the art can come up with various modified or altered examples within the scope of the technical ideas disclosed in this application, and it is understood that these also naturally belong to the technical scope of the present invention. [Explanation of symbols]

[0113] 1. Photography device 5. Server 6. Terminal 7. Network 10. Control device 21 Operation panel 22. Coin Mech 23. Camera 24. Lighting section 25 Printer 26... Print output outlet 27 Audio output section 28. Curtains 29. Object sensor 31 Reading device 32. Card scanner 33. IC chip reader 34 Document scanner 35 Code reader 51 Application content selection screen 52 Code reading guide screen 53 Code entry screen 54 Code confirmation screen 55 ID card insertion guide screen 56 PIN entry screen 57 Error notification screen 58······Operation instruction screen 61 Application image capture screen 62, 63...Operation screen 71 Vision test screen 72...Measurement result screen 73 Password setting screen 74 Reading guide screen 75 Confirmation screen of scanned document data 76... Application details confirmation screen 77....End screen 101... Control unit 401...Identification section 402 Authenticity determination unit 403...Movement instruction section 404... Biometric determination unit 405: Application image capture section 406...Object detection unit 407... Same person determination section A 408... Same person determination section B 409...Image judgment unit 411 Vision Testing Department 412: Password setting value input section 413 Document Scanner 414: Payment processing unit 415 Data output section

Claims

1. A photographing device equipped with a camera for photographing an applicant as a subject, an action instruction means for giving an action instruction to the person to be photographed; a liveness determination means for capturing a video of the movement of the person to be photographed responding to the operation instruction by the camera and analyzing the captured video to determine whether the person to be photographed is a live person; an application image capturing means for capturing an application image, which is a photograph to be used in the application procedure, using the camera; a person identification means for comparing the image cut out from the video with the application image to determine whether the photographed person is the same person; an output means for outputting the application image to a predetermined server when the liveness determination means determines that the photographed person is a live body and the same person determination means determines that the persons match; Equipped with A reading means for reading information on the applicant's identity verification medium; a registration information acquiring means for acquiring registration information previously registered in the server by the applicant; an identity verification means for verifying the identity of the applicant by comparing the registration information acquired by the registration information acquisition means with the information read by the reading means; The photographing device further comprises:

2. 2. The photographing apparatus according to claim 1, wherein said action instruction means randomly presents a plurality of pre-stored action instructions.

3. The apparatus further includes a scanner for reading face information of the applicant's identity verification medium, The same person determination means The person image included in the card face information read by the scanner is compared with the image cut out from the video and the application image to determine whether they are the same person.

3. The photographing apparatus according to claim 1 or 2.

4. 4. The photographing device according to claim 1, further comprising a notification means for notifying an error when the biometric determination means determines that the person being photographed is not a biometric person or when the same person determination means determines that the persons do not match.

5. The application image photographing means photographs a wide angle image, and an object detecting means detects an object covering the face of the person to be photographed from the photographed image.

5. The photographing apparatus according to claim 1, further comprising:

6. 6. The photographing device according to claim 1, further comprising an image determining unit that determines whether the application image has been properly taken for use in the application procedure.

7. 7. The photographing device according to claim 1, further comprising a visual acuity testing means for displaying a visual acuity test screen and carrying out a visual acuity test on said applicant.

8. The application procedure further includes a document scanner for reading documents used in the application procedure, 7. The photographing device according to claim 1, wherein the output means outputs the document data read by the document scanner to the predetermined server together with the application image.

9. A setting means for setting a password required for the application procedure is provided, The output means outputs the secret information set by the setting means to the predetermined server together with the application image.

9. The photographing apparatus according to claim 1, wherein the first and second photographing means are arranged to be arranged in a direction parallel to the first and second photographing directions.

10. A method for controlling a photographing device equipped with a camera that photographs an applicant as a subject, comprising the steps of: a reading step of reading information on the applicant's identity verification medium; a registration information acquisition step of acquiring registration information previously registered by the applicant in a predetermined server; an identity verification step of verifying the identity of the applicant by comparing the registration information acquired in the registration information acquisition step with the information read in the reading step; an action instruction step of giving an action instruction to the person to be photographed; a liveness determination step of capturing a video of the movement of the subject responding to the operation instruction by the camera and analyzing the captured video to determine whether the subject is a live body; An application image capturing step of capturing an application image, which is a photograph to be used in the application procedure, by the camera; a same person determination step of comparing the image cut out from the video with the application image to determine whether the photographed person matches; an output step of outputting the application image to the predetermined server when the liveness determination step determines that the photographed person is a live body and the same person determination step determines that the persons match; 13. A method for controlling a photographing apparatus, comprising:

11. A control device for a photographing device equipped with a camera for photographing the applicant as a photographed person, an action instruction means for giving action instructions to the person to be photographed; a liveness determination means for capturing a video of the movement of the person to be photographed responding to the operation instruction by the camera and analyzing the captured video to determine whether the person to be photographed is a live person; an application image capturing means for capturing an application image, which is a photograph to be used in the application procedure, using the camera; a person identification means for comparing an image cut out from the video with the application image to determine whether the photographed person is the same person; an output means for outputting the application image to a predetermined server when the liveness determination means determines that the photographed person is a live body and the same person determination means determines that the persons match; Functioning as a A reading means for reading information on the applicant's identity verification medium; A registration information acquisition means for acquiring registration information previously registered in the server by the applicant; an identity verification means for verifying the identity of the applicant by comparing the registration information acquired by the registration information acquisition means with the information read by the reading means; A program to further enhance its functionality.

Images