Terminal device, method for controlling terminal device, and program

Abstract

A terminal device, a method for controlling a terminal device, and a program are provided that can facilitate personal identification while ensuring liveness. [Solution] The terminal device 4 comprises an imaging means 41 that captures an image including an identification card, a display unit 42 that displays an image of the imaging area of ​​the imaging means 41 on a screen, and a processing unit 43 that controls the capturing of the image. The display unit 42 displays a trapezoidal guide frame F on the screen in a random display mode, and the processing unit 43 has a determination unit 431 that determines whether the imaging condition is met, that the outline of the identification card, including its thickness, is in line with the guide frame F, and a shooting control unit 432 that causes the imaging means 41 to capture an image when the determination unit 431 determines that the imaging condition is met.

Description

Technical Field

[0001] The present disclosure relates to a terminal device for performing online personal verification using an identity document, a method for controlling the terminal device, and a program.

Background Art

[0002] For a contract of a service that conducts financial transactions such as opening a bank account, personal verification is required. This personal verification is generally performed by the service provider verifying a copy of the identity document sent by mail to authenticate the person.

[0003] In recent years, when performing this personal verification, a method is known in which a user takes a picture of their own identity document with a terminal device such as a smartphone, sends the image to the service provider, and completes the personal verification online (for example, Patent Document 1). By completing the personal verification online in this way, the convenience of the user can be improved.

Prior Art Documents

Patent Documents

[0004]

Patent Document 1

Summary of the Invention

Problems to be Solved by the Invention

[0005] However, in the method of completing personal verification online by transmitting an image of an identity document as described above, there is a case where an illegally obtained image of an identity document or an image of a reused identity document is transmitted, and fraud cannot be prevented.

[0006] Therefore, when taking a picture of an identity document, a method can be considered in which a guide frame is randomly displayed on the screen of the terminal device, the identity document is photographed along the guide frame, and the image is transmitted. According to this method, it can be estimated that the image is not a pre-prepared fraudulent image but an image of the user's own identity document taken on the spot, thereby ensuring liveness and preventing fraud.

[0007] In addition, fraudulent images of identity documents are often images of only one side of the identity document, such as the surface of the identity document with a face photo. Therefore, fraud can be prevented by causing the user to photograph not only the side with the face photo of the identity document but also the thickness (side surface) of the identity document.

[0008] However, if one attempts to photograph not only one side of the identity document with the face photo but also the thickness of the identity document, it is necessary to photograph the identity document obliquely. Moreover, since the guide frame is randomly displayed, the difficulty of photographing the identity document along the guide frame may increase.

[0009] The present disclosure has been made to solve the above problems, and an object thereof is to provide a terminal device, a method for controlling the terminal device, and a program that can facilitate personal identification while ensuring liveness.

Means for Solving the Problems

[0010] The terminal device of the present disclosure includes an imaging unit that photographs an image including an identity document, a display unit that displays an image of the imaging area of the imaging unit within the screen, and a processing unit that controls the photographing of the image. The display unit displays a trapezoidal guide frame in a random display mode within the screen. The processing unit includes a determination unit that determines whether a photographing condition that the contour including the thickness of the identity document is along the guide frame is satisfied, and a photographing control unit that causes the imaging unit to photograph the image when the determination unit determines that the photographing condition is satisfied.

[0011] A method for controlling a terminal device according to the present disclosure is a method for controlling a terminal device including an imaging unit that captures an image including an identity certificate and a display unit that displays an image of a shooting area of the imaging unit on a screen, the method including: a guide frame display step of displaying a trapezoidal guide frame in a random display mode within the screen of the display unit; a determination step of determining whether a shooting condition that a contour including the thickness of the identity certificate is along the guide frame is satisfied; and a shooting control step of causing the imaging unit to capture the image when it is determined in the determination step that the shooting condition is satisfied.

[0012] A program according to the present disclosure is a program executed on a terminal device including an imaging unit that captures an image including an identity certificate, a display unit that displays an image of a shooting area of the imaging unit on a screen, and a computer, the program causing the computer to execute: a guide frame display step of displaying a trapezoidal guide frame in a random display mode within the screen of the display unit; a determination step of determining whether a shooting condition that a contour including the thickness of the identity certificate is along the guide frame is satisfied; and a shooting control step of causing the imaging unit to capture the image when it is determined in the determination step that the shooting condition is satisfied.

[0013] A terminal device according to the present disclosure includes an imaging unit that captures an image including an identity certificate, a display unit that displays an image of a shooting area of the imaging unit on a screen, and a processing unit that controls shooting of the image. The display unit displays, in one display mode among a plurality of display modes, a guide frame in a shape obtained by viewing the identity certificate from a direction other than a direction perpendicular to the identity certificate within the screen. The processing unit includes a determination unit that determines whether a shooting condition that a contour including the thickness of the identity certificate is along the guide frame is satisfied, and a shooting control unit that causes the imaging unit to capture the image when it is determined by the determination unit that the shooting condition is satisfied.

[0014] A method for controlling a terminal device according to the present disclosure is a method for controlling a terminal device including an imaging unit that captures an image including an identity certificate and a display unit that displays an image of a shooting area of the imaging unit on a screen, the method including: a guide frame display step of displaying, in one of a plurality of display modes, a guide frame having a shape of the identity certificate viewed from a direction other than the vertical direction with respect to the identity certificate on the screen of the display unit; a determination step of determining whether a shooting condition that a contour including the thickness of the identity certificate is along the guide frame is satisfied; and a shooting control step of causing the imaging unit to capture the image when it is determined in the determination step that the shooting condition is satisfied.

[0015] A program according to the present disclosure is a program executed on a terminal device including an imaging unit that captures an image including an identity certificate, a display unit that displays an image of a shooting area of the imaging unit on a screen, and a computer, the program causing the computer to perform: a guide frame display step of displaying, in one of a plurality of display modes, a guide frame having a shape of the identity certificate viewed from a direction other than the vertical direction with respect to the identity certificate on the screen of the display unit; a determination step of determining whether a shooting condition that a contour including the thickness of the identity certificate is along the guide frame is satisfied; and a shooting control step of causing the imaging unit to capture the image when it is determined in the determination step that the shooting condition is satisfied.

Advantages of the Invention

[0016] According to the present disclosure, it is possible to obtain a terminal device, a method for controlling the terminal device, and a program that can facilitate personal identification while ensuring liveness.

Brief Description of the Drawings

[0017]

Figure 1

Figure 2

Figure 3

Figure 4

Figure 5

Figure 6

Figure 7

Figure 8

Figure 9

Figure 10

Figure 11

Figure 12

Figure 13

Figure 14

Figure 15

Figure 16

Figure 17

Embodiments for Carrying Out the Invention

[0018] [Overall Configuration] FIG. 1 is a diagram showing the configuration of a system to which a terminal device according to an embodiment of the present invention is applied. As shown in FIG. 1, the system 1 includes a server 2, a network 3, and a terminal device 4, and the server 2 and the terminal device 4 are connected via the network 3. As shown in FIG. 1, a plurality of terminal devices 4 may be connected to the server 2.

[0019] The server 2 is used to perform identity verification before providing a predetermined service to a user. The predetermined service is a service for conducting financial transactions and requires identity verification using an identity certificate before receiving the service. In the present embodiment, the predetermined service is the opening of a bank account and the deposit and withdrawal service using the bank account, and the server 2 is used to perform identity verification. As will be described later, the server 2 generates a random value and transmits the random value to the terminal device 4 via the network 3.

[0020] The network 3 is a communication network that provides a connection path enabling data transmission and reception between the server 2 and the terminal device 4. The network 3 is a wired or wireless network and is composed of one or a plurality of networks. Examples of the network include, but are not limited to, a local area network (LAN), a wide area network (WAN), the Internet, a mobile communication network, etc., and a known network can be used.

[0021] The terminal device 4 is an information processing device used by a user and includes a CPU, a memory (RAM), a storage device (such as a flash ROM), etc., which are connected to each other by a bus (none of which are shown in the figure). Typically, an OS and an application program are stored in the storage device, and when the terminal device 4 according to the embodiment operates, they are appropriately read into the memory and executed by the CPU, whereby various functions including the functions of the terminal device 4 according to the embodiment are executed. Examples of the terminal device 4 include, but are not limited to, a smartphone, a mobile phone, a computer equipped with a display and a camera, a wearable terminal, etc.

[0022] In this embodiment, the identity verification required when opening a bank account is performed online using the server 2 and the terminal device 4. Specifically, the terminal device 4 is used to photograph an identity document including its thickness, and the photographed image is transmitted to the server 2 via the network 3. The service provider of the server 2 performs identity verification using the image.

[0023] [Detailed Configuration] FIG. 2 is a functional block diagram of the terminal device 4 according to the embodiment. As shown in FIG. 2, the terminal device 4 includes an imaging unit 41, a display unit 42, a processing unit 43, a storage unit 44, and a communication unit 45.

[0024] The imaging unit 41 is a camera that captures an image including an image sensor such as a CMOS image sensor. The imaging unit 41 captures an image including an identity document. The identity document is, for example, an identity verification document with a face photo such as a driver's license, a passport, a resident basic register card, or a My Number card. Further, the identity document may be a health insurance card, a residence card, a special permanent resident certificate, a pension book, a disability handbook, a notification card, a copy of a resident register, or a seal registration certificate, or an identity verification document without a face photo. Further, since the identity document is an object, it has a thickness. That is, the identity document has a thickness even if it is as thin as a single sheet of paper as long as it is a physical object. The thickness is formed by the side surface of the identity document (especially the side surface without the description of the certification matter such as characters or a face photo). When the identity document is as thin as a single sheet of paper, for example, it is formed by the edge of the identity document. The identity document has a rectangular shape (rectangle) in this embodiment, but may have a quadrilateral shape such as a square or a rhombus, or a shape other than a quadrilateral such as a round shape.

[0025] The display unit 42 has a screen such as an organic EL or a liquid crystal display, and displays an image of the imaging area of the imaging means 41 on the screen. The display unit 42 displays a trapezoidal guide frame F in a random display mode within the screen. The guide frame F is a frame for enclosing an image of an identity document necessary for person authentication. This guide frame F may have a shape when the identity document is viewed from a direction other than the direction perpendicular to the identity document. In other words, the guide frame F has a shape when the identity document is viewed from an oblique direction with respect to one side of the identity document. The shape when the identity document is viewed from a direction other than the direction perpendicular to the identity document does not include the shape when the identity document is viewed from the direction perpendicular to the identity document, that is, the shape of the identity document itself. This is to enable the identity document to be photographed including its thickness. The guide frame F can be, for example, square, trapezoidal, round, or elliptical. In this embodiment, the guide frame F is trapezoidal. Also, when a rectangular identity document is enclosed and photographed along the trapezoidal guide frame F, the obtained image includes the thickness of the identity document (see FIG. 8). That is, the user uses the imaging means 41 of the terminal device 4 to image the identity document obliquely. The random display mode of the guide frame F is a mode in which at least one of the size of the guide frame F, the position of the guide frame F, and the type of the trapezoidal shape of the guide frame F is randomly determined. For example, if there are two types of sizes for the guide frame F (large and small), three types of positions (upper, central, and lower within the screen), and two types of trapezoidal shapes (a and b), one guide frame F is randomly displayed from among 12 patterns of the guide frame F. The type of the trapezoidal shape of the guide frame F indicates the trapezoidal shape of the identity document projected onto the screen of the display unit 42 when the surface (for example, the front surface) of the identity document is photographed at a predetermined photographing angle θ with respect to the perpendicular line perpendicular to the surface of the identity document. Type a is a trapezoidal shape with an interior angle of 60° projected when the photographing angle θ = α, and type b is a trapezoidal shape with an interior angle of 45° projected when the photographing angle θ = β. That is, since the photographing angle β is larger than the photographing angle α (α < β), when the length of the lower base is the same, the height of the guide frame F of type b is lower than that of type a.FIG. 3 shows the guide frame F when (size, position, type) = (large, center, a) of the guide frame F, FIG. 4 shows the guide frame F when (size, position, type) = (small, upper, b) of the guide frame F, and FIG. 5 shows the guide frame F when (size, position, type) = (large, lower, a) of the guide frame F.

[0026] In the present embodiment, the display unit 42 displays the guide frame F in a random display mode within the screen. However, the display unit 42 may display the guide frame F in one of a plurality of display modes. The display of the guide frame F in one of the plurality of display modes includes the display of the guide frame F in the random display mode of the guide frame F as described above. The selection of one of the plurality of display modes may be irregular (random) or may have regularity. The plurality of display modes may be predetermined in advance by a person other than the user of the terminal device 4, such as the administrator of the terminal device 4.

[0027] The display unit 42 may display guidance G that guides the user so that the contour including the thickness of the identification document can be made to coincide with the guide frame F. Examples of the guidance G include characters, figures, symbols, or combinations of two or more of these. For example, as shown in FIG. 7, when the image of the identification document C is shifted to the left with respect to the guide frame F, a message prompting the user to shift the terminal device 4 (imaging means 41) to the left is displayed, and when the image of the identification document is too small, a message prompting the user to move the terminal device 4 (imaging means 41) closer to the identification document is displayed. Examples of the figure and symbol include an arrow indicating the direction in which the terminal device 4 (imaging means 41) is to be moved. The guidance G may display a message so that the background of the identification document is not the same color or the same color system as the identification document. Thereby, the processing unit 43 (for example, the determination unit 431 and the image extraction unit 433 described later) can easily detect the boundary between the background and the identification document. The guidance G may display a message to rotate the orientation of the identification document. In one example, a message prompting rotation, for example, to the left or right, may be displayed so that the identification document fits into the guide frame F. In another example, a message may be displayed prompting the user to change the orientation of the identification document by rotating it so that the face photo of the identification document or the orientation of the characters described in the identification document is upside down. Thereby, since the user of the terminal device 4 needs to respond in real time, the liveness can be ensured and fraud in identity verification can be prevented.

[0028] The processing unit 43 is configured, for example, by an application program being executed by a CPU, and includes a determination unit 431, a shooting control unit 432, an image extraction unit 433, and a display control unit 434. The determination unit 431 determines whether the shooting conditions are satisfied. The shooting conditions are conditions such that the contour E including the thickness of the identity certificate is along the guide frame F (for example, see FIG. 8). For example, when the contour E of the identity certificate C coincides with the guide frame F, when the contour E of the identity certificate C is parallel to the guide frame F and is within the guide frame F, etc., but not limited thereto. For example, the contour E of the identity certificate C does not necessarily have to be parallel to the guide frame F. As shown in FIG. 6, if it is within the margin area inside and outside including the guide frame F, it may be determined that the contour of the identity certificate is along the guide frame F. The margin area is an area surrounded by a first frame F1 similar to the guide frame F that is one size larger than the guide frame F and a second frame F2 similar to the guide frame F that is one size smaller than the guide frame F, and includes the frame line of the guide frame F. In the present embodiment, the determination unit 431 determines whether the shooting conditions are satisfied based on whether the area of the identity certificate that is trapezoidal on the screen of the display unit 42 when shot obliquely is within a predetermined area, and whether the interior angle of the identity certificate that is trapezoidal is within a predetermined angle range. This area and interior angle can be obtained, for example, by extracting the contour of the identity certificate from an image of the identity certificate.

[0029] The shooting control unit 432 is configured to include, for example, a CPU, and controls the imaging means 41. Specifically, as shown in FIG. 8, when the determination unit 431 determines that the shooting conditions are satisfied, the shooting control unit 432 causes the imaging means 41 to shoot an image including the identity certificate. That is, the imaging means 41 automatically shoots the identity certificate on the condition that the shooting conditions are satisfied.

[0030] The image extraction unit 433 is configured to include, for example, a CPU, and extracts the inside of the contour E including the thickness of the identity certificate C from the image (see FIG. 8) taken by the shooting control unit 432 and the imaging means 41, and generates an extracted image (see FIG. 9). The extracted image is an image including the contour E including the thickness of the identity certificate C and the area inside thereof. The extracted image may be generated by deleting the upper and lower backgrounds in FIG. 8. The extracted image is, for example, an image (see FIG. 9) including the front surface S1 and the side surface S2 with the face photo of the identity certificate C, or an image including the back surface and the side surface of the identity certificate.

[0031] As shown in FIG. 10, the image extraction unit 433 masks a predetermined area R in the extracted image. Masking is a process of filling the predetermined area R, for example, painting the predetermined area R black. The predetermined area R is an area where information not necessarily required for identity verification is described, for example, an area where confidential information is described. When the identity certificate is a driver's license, it is the area between the location where the expiration date of the driver's license is described and the location where the driver's license number is described.

[0032] The display control unit 434 is configured to include, for example, a CPU, and controls the display unit 42. The display control unit 434 determines a guide frame F to be displayed on the display unit 42, and causes the display unit 42 to display the guide frame F. In the present embodiment, the terminal device 4 receives a random value from the server 2 which is an external device, and the display control unit 434 determines the guide frame F to be displayed on the display unit 42 based on this random value.

[0033] Specifically, the storage unit 44 stores a table in which the size, position, and type of trapezoidal shape of the guide frame F in the screen are associated. For example, if the size of the guide frame F has two types of large and small, the position has three types of upper, center, and lower in the screen, and the type of trapezoidal shape has two types of a and b, as shown in FIG. 11, the table has a degree of freedom of 3 and has a total of 12 patterns of guide frame F information (size, position, type). The display control unit 434 determines the guide frame F to be displayed on the display unit 42 from among the 12 patterns of guide frame F.

[0034] For example, when a predetermined numerical range that does not overlap with each pattern is associated with each pattern in the table, the display control unit 434 determines which numerical range of which pattern the obtained random value from the server 2 belongs to, reads out the information of the guide frame F of the corresponding pattern from the storage unit 44, and causes the display unit 42 to display the corresponding guide frame F. Alternatively, when a predetermined numerical range is associated with each type of the size, position, and trapezoidal shape type of the guide frame F in the table, by determining the numerical range to which each random value for the size, position, and trapezoidal shape type of the guide frame F obtained from the server 2 belongs, the information of the guide frame F of the corresponding pattern is read out from the storage unit 44, and the corresponding one guide frame F is caused to be displayed on the display unit 42.

[0035] When the determination unit 431 determines that the imaging conditions are not satisfied, the display control unit 434 causes the display unit 42 to display a guidance G for guiding the user so that the contour of the identity certificate can be made to coincide with the guide frame F as shown in FIG. 7.

[0036] The display control unit 434 extracts the contour of the identity certificate from the image of the identity certificate obtained by the imaging means 41, and by comparing the contour and the guide frame F within the screen, obtains an operation for eliminating the deviation of the contour of the identity certificate with respect to the guide frame F within the screen.

[0037] For example, the display control unit 434 calculates the deviation between the contour of the identity certificate and the center positions of each of the guide frames F. When the center position of the contour of the identity certificate is shifted to the left with respect to the center position of the guide frame F, a guidance G indicating that the terminal device 4 (imaging means 41) is to be shifted to the left is displayed. Further, the display control unit 434 compares the area within the contour of the identity certificate with the area within the guide frame F, and when the area within the contour of the identity certificate is larger than the area within the guide frame F, it determines that the terminal device 4 (imaging means 41) is too close to the identity certificate, and displays a guidance G prompting it to move away from the identity certificate. Conversely, when the area within the contour of the identity certificate is smaller than the area within the guide frame F, it determines that the terminal device 4 (imaging means 41) is too far from the identity certificate, and displays a guidance G prompting it to approach the identity certificate. Further, the display control unit 434 compares the inner angles of the trapezoidal contour of the identity certificate (hereinafter, also referred to as the inner angles of the contour) with the inner angles of the corresponding trapezoidal guide frame F (hereinafter, also referred to as the inner angles of the guide frame F), and when the inner angles of the contour are larger than the inner angles of the guide frame F, it determines that the identity certificate and the terminal device 4 are in a state closer to being parallel, and displays a guidance G prompting the terminal device 4 to be tilted more obliquely with respect to the identity certificate. Conversely, when the inner angles of the contour are smaller than the inner angles of the guide frame F, it determines that the terminal device 4 is tilted too much with respect to the identity certificate, and displays a guidance G prompting the terminal device 4 to be tilted more parallel to the identity certificate.

[0038] The storage unit 44 is a storage device such as an HDD, an SSD, or a flash memory, and stores in advance information necessary for causing the CPU to execute functions as the processing unit 43, such as programs and the above tables. The program is a program for causing the terminal device 4 to execute functions as the processing unit 43.

[0039] The communication unit 45 is an interface that transmits and receives various data via the network 3. The communication unit 45 transmits and receives data with the server 2, which is an external device. For example, the communication unit 45 receives a random value from the server 2. When the display control unit 434 determines the size, position, and type of the trapezoidal shape of the guide frame F, it receives random values for the size, position, and type of the trapezoidal shape of the guide frame F, that is, three random values. Further, the communication unit 45 transmits the extracted image to the server 2.

[0040] [Operation] The operations of the terminal device 4 and the server 2 will be described with reference to FIGS. 12 to 17. FIG. 12 is a flowchart showing the operation of the terminal device.

[0041] Here, as an example of identity verification, a case where an application for opening a bank account is made using the terminal device 4, which is a smartphone, will be described. An application program for account application is incorporated in the applicant's terminal device 4. In order to incorporate this application program into the terminal device 4, for example, access is made to a predetermined server via a network, and a dedicated application program is downloaded as a so-called app for account application. Note that this server may be a server different from the server 2, or the server 2 itself may provide the above application program to the terminal device 4.

[0042] As shown in FIG. 12, when an application program for account application is launched by the applicant (step S01), the CPU of the terminal device 4 causes the display unit 42 to display a top screen (step S02). FIG. 13(a) is a diagram showing the terminal device 4 in a state where the top screen is displayed. As shown in FIG. 13(a), two buttons, "Open an account" and "Resume application", are displayed on the top screen. The applicant can tap these two buttons to give an instruction. Tapping means an operation of lightly touching the screen by tapping the touch panel once. By tapping a button displayed on the screen, an instruction corresponding to that button can be given. Here, since the terminal device 4 is a smartphone, its touch panel serves as an input means for receiving taps.

[0043] The "Resume application" button is used to resume the application process when the application process has been temporarily interrupted. When the "Resume application" button is tapped by the applicant, the information stored in the storage unit 44 is read out as part or all of the information required for account opening, the corresponding screen is displayed, and the application process is resumed.

[0044] The "Open Account" button is used to start the account opening process. When the "Open Account" button is tapped by the applicant, the CPU of the terminal device 4 causes the display unit 42 to display a confirmation item consent screen (step S03). As shown in FIG. 13(b), on the confirmation item consent screen, an explanation of the confirmation items is displayed at the upper part of the screen. Also, at the uppermost part of the screen, a "Back" button for returning to the top screen shown in FIG. 13(a) is displayed. At the lowermost part, links are provided for the personal information protection policy, the app handling regulations, and the declaration / confirmation content of the statement of not being an anti-social force. Clicking on the icons on the right side of each allows the linked content to be viewed. Above that, a square checkbox for consent is provided. Although not shown in the figure, by scrolling the screen, an "Apply" button is displayed. Tapping the displayed checkbox checks it. When the "Apply" button is tapped by the applicant with the checkbox checked, the CPU of the terminal device 4 causes the display unit 42 to display a shooting caution screen (step S04). The display of the shooting caution screen is exchanged for consent to various confirmation items.

[0045] FIG. 14(a) is a diagram showing the terminal device 4 with the shooting caution screen displayed. As shown in FIG. 14(a), on the shooting caution screen, an explanation of the precautions during shooting is displayed at the upper part of the screen. Also, at the uppermost part of the screen, a "Back" button for returning to the confirmation item consent screen shown in FIG. 13(b) is displayed. At the lower part of the screen, a "Next" button is displayed. When the "Next" button is tapped by the applicant, the imaging means 41 is activated by the CPU of the terminal device 4. As a result, the screen of the display unit 42 switches to a shooting screen through the lens of the imaging means 41, and a frame line for positioning the identity certificate is displayed on the shooting screen.

[0046] FIG. 14(b) is a diagram showing a terminal device in a state where a screen for photographing the front surface of an identity certificate is displayed. As shown in FIG. 14(b), when switching to the photographing screen, the CPU of the terminal device 4 switches to a horizontally long screen format for display. This is because it is more suitable to have a horizontally long format when photographing a driver's license as an identity certificate. As shown in FIG. 14(b), at the uppermost part of the screen for photographing the front surface of the identity certificate, a "BACK" button for returning to the caution screen during photographing is displayed. And on the right side of the screen, a "PHOTOGRAPH" button is displayed. The frame line for positioning the driver's license can be in any form, but in the example of FIG. 14(b), it is formed by four dashed lines that form a rectangle. In this state, when the applicant aligns the driver's license with the position of the frame line and taps the "PHOTOGRAPH" button, the imaging means 41 performs photographing and acquires a photographed image (step S05). Specifically, the image of the front surface of the license is temporarily stored in the memory of the terminal device 4.

[0047] Then, the CPU of the terminal device 4 uses the acquired image of the front surface of the license to cause the display unit 42 to display a confirmation screen for the photographing result of the front surface of the driver's license (step S06). FIG. 15 is a diagram showing the terminal device in a state where the photographing result confirmation screen is displayed. As shown in FIG. 15, on the photographing result confirmation screen, an image of the front surface of the license is displayed approximately in the center. Also, on the left side at the uppermost part of the screen, a "BACK" button for returning to the screen for photographing the front surface of the license is displayed. Also, on the right side at the uppermost part of the screen, a "NEXT" button is displayed.

[0048] Also, the CPU of the terminal device 4 extracts the face image included in the image of the front surface of the license, which is the face image of the identity certificate (step S07), and stores the extracted face image in the memory or the like of the terminal device 4 (step S08).

[0049] On the shooting result confirmation screen, the applicant checks the license surface image which is the shooting result. When the "Next" button on the shooting result confirmation screen is tapped by the applicant, the CPU of the terminal device 4 activates the imaging means 41. Then, the screen of the display unit 42 switches to the shooting screen through the lens of the imaging means 41, and a frame line for positioning the back side of the driver's license is displayed on the shooting screen. That is, when the imaging means 41 is activated, the screen of the display unit 42 switches to the back side shooting screen of the identity certificate.

[0050] Although the back side shooting screen of the identity certificate is not shown in the figure, it is a horizontally long screen similar to the front side shooting screen of the identity certificate shown in Fig. 14(b), and a "Back" button and a "Shoot" button are displayed. Also, a frame line for positioning the driver's license is displayed in the central part. In this state, when the applicant aligns the back side of the driver's license with the position of the frame line and taps the "Shoot" button, the imaging means 41 takes a shot and acquires a shot image (step S09). The CPU of the terminal device 4 extracts the part corresponding to the inside of the frame line from the obtained image and acquires it as the back side image of the license. Specifically, the back side image of the license is temporarily stored in the memory of the terminal device 4.

[0051] The CPU of the terminal device 4 causes the display unit 42 to display the back side shooting result confirmation screen (step S10). Although the back side shooting result confirmation screen is not shown in the figure, it is a horizontally long screen similar to the front side shooting result confirmation screen shown in Fig. 15. A "Back" button for returning to the back side shooting screen is displayed on the upper left side of the screen, and a "Next" button is displayed on the upper right side of the screen. When the "Next" button is tapped by the applicant, the CPU of the terminal device 4 activates the imaging means 41 and takes a shot of the identity certificate from an oblique angle (step S11). The details of this oblique shot will be described separately later.

[0052] After photographing the identity certificate obliquely, the CPU of the terminal device 4 activates the in-camera of the terminal device 4 (step S12), and photographs the applicant's face with this in-camera (step S13). Then, the terminal device 4 performs face recognition processing on the photographed image photographed by the in-camera (step S14) by, for example, the processing unit 43, and detects a face image for verification (step S15). The face recognition processing of the photographed image is processing for extracting the face of the person included in the image from the photographed image. The verification image is an image including the face image of the applicant photographed by the in-camera. The CPU of the terminal device 4 may store the obtained face image for verification in the memory of the terminal device 4. Further, when photographing the applicant's face with the in-camera, in order to ensure liveness, an instruction to blink or an instruction to turn the face up, down, left, or right may be given to the applicant, and a function to confirm whether the applicant himself / herself is performing the action may be provided.

[0053] The terminal device 4 transmits, by the communication unit 45, the face image acquired from the identity certificate and the face image for verification acquired from the in-camera to the server 2 (step S16). Then, the terminal device 4 receives, by the communication unit 45, the collation result information of the face image acquired from the identity certificate and the face image for verification acquired from the in-camera in the server 2 (step S17). The CPU of the terminal device 4 determines whether the collation result information indicates a match in the collation result. If it indicates a match (YES in step S18), the identity verification process ends. If it indicates a mismatch (NO in step S18), the process returns to step S13 or S14. If it is determined that there is a mismatch a predetermined number of times in the determination in step S18, the identity verification process may end. Also, since it may be difficult to recognize the face image acquired from the identity certificate due to light reflection, the process may return to the photographing of the surface of the identity certificate in step S05 and be redone.

[0054] FIG. 16 is a flowchart showing the operation of the server. As shown in FIG. 16, when the server 2 receives the face image of the identity certificate and the face image for verification acquired by the in-camera from the terminal device 4 (step S21), it performs a face verification process for verifying the faces in both images (step S22). In addition to the face verification process, the server 2 may determine that the identity certificate belongs to the applicant himself / herself by receiving from the terminal device 4 an image including the front and side surfaces of the identity certificate obtained by photographing the identity certificate obliquely. The server 2 transmits the verification result of the face verification process to the terminal device 4 as verification result information (step S23), and then ends.

[0055] (Process of photographing the identity certificate obliquely) As shown in FIG. 17, first, the terminal device 4 acquires a random value from the server 2 by the communication unit 45 (step S31). The acquisition of the random value can be achieved, for example, by the terminal device 4 requesting the server 2, and in response to this request, the server 2 generates a random value and transmits it to the terminal device 4. Here, as an example, random values are generated for the size, position, and type of trapezoidal shape of the guide frame F, respectively, and the terminal device 4 receives each random value.

[0056] Next, the display control unit 434 determines the guide frame F to be displayed on the screen of the display unit 42 (step S32). For example, the display control unit 434 determines the size of the guide frame F based on the table stored in the storage unit 44 and the random value for the size of the guide frame F, determines the position of the guide frame F based on the table stored in the storage unit 44 and the random value for the position of the guide frame F, and determines the type of trapezoidal shape of the guide frame F based on the table stored in the storage unit 44 and the random value for the type of trapezoidal shape of the guide frame F, thereby determining one guide frame F.

[0057] After determining the guide frame F, the display control unit 434 causes the display unit 42 to display the determined guide frame F (step S33).

[0058] Next, the determination unit 431 determines whether the contour including the thickness of the identification document follows the guide frame F (step S34). If it is determined by the determination unit 431 that it does not follow the guide frame F (NO in step S34), the determination result is output to the display control unit 434, and the display control unit 434 causes the guidance G to be displayed on the display unit 42 (step S35). Thereby, the user is guided so that the identification document follows the guide frame F. Note that the determination by the determination unit 431 as to whether it follows the guide frame F may include some errors for improving convenience.

[0059] On the other hand, if it is determined by the determination unit 431 that it follows the guide frame F (YES in step S34), the determination result is output to the imaging control unit 432, and the imaging control unit 432 controls the imaging means 41 to automatically capture the identification document (step S36). Note that after the automatic capture, the captured image may be displayed on the display unit 42 to prompt the user (applicant) to confirm.

[0060] The image extraction unit 433 generates an extracted image from the image obtained by the automatic capture (step S37). The extracted image is, for example, an image obtained by deleting the backgrounds above and below the guide frame F of the image obtained by the automatic capture.

[0061] Furthermore, after generating the extracted image, a predetermined area R within the guide frame F is masked (step S38). Then, the masked extracted image is transmitted to the server 2 by the communication unit 45 (step S39), and the process ends. Note that instead of the transmission in step S39, in step S16, the face image obtained from the identification document and the reference image obtained from the in-camera may be transmitted to the server 2.

[0062] [Operation and Effect] (1) The terminal device 4 of this embodiment includes an imaging means 41 for capturing an image including an identity certificate, a display unit 42 for displaying an image of the imaging area of the imaging means 41 on the screen, and a processing unit 43 for controlling the imaging of the image. The display unit 42 displays a trapezoidal guide frame F on the screen in a random display mode. The processing unit 43 includes a determination unit 431 for determining whether the contour including the thickness of the identity certificate satisfies the imaging condition of being along the guide frame F, and an imaging control unit 432 for causing the imaging means 41 to capture an image when it is determined by the determination unit 431 that the imaging condition is satisfied.

[0063] As a result, it is possible to facilitate identity verification while ensuring liveness. That is, since the guide frame F for storing the identity certificate is randomly displayed, the liveness can be ensured by capturing the identity certificate so as to fit the guide frame F. On the other hand, since the guide frame F is trapezoidal, the user needs to capture the identity certificate obliquely, increasing the difficulty of imaging. Therefore, the determination unit 431 and the imaging control unit 432 automatically capture the identity certificate on the condition that the identity certificate is along the guide frame F so that its thickness can be recognized, thereby improving the convenience for the user in identity verification. In addition, since it is determined that the identity certificate is along the guide frame F and then the identity certificate including its thickness is automatically captured, the presence of the obtained image is presumed to ensure liveness. Therefore, even if the service provider visually checks the obtained image, it can be a simple check, and the labor of the identity verification process can be reduced.

[0064] (2) It is provided with a communication unit 45 for communicating with a server 2 which is an external device via a network 3. The processing unit 43 further includes an image extraction unit 433 for generating an extraction image by extracting the inside of the contour including the thickness of the identity certificate from the image captured by the imaging control unit 432 and the imaging means 41. The communication unit 45 is configured to transmit the extraction image to the server 2. As a result, identity verification can be completed online, and since an image obtained by extracting the necessary part for identity verification is transmitted, the amount of transmitted data can be reduced.

[0065] (3) The display unit 42 is configured to display the guide frame F in a random display mode based on the random value transmitted from the server 2. As a result, since the guide frame F is displayed within the screen of the display unit 42 based on the random value obtained from the server 2, which is a device different from the terminal device 4, the reliability of the identity verification can be enhanced. That is, when the guide frame F is randomly displayed by a random value generated on the terminal device 4 side without being caused by the server 2, which is an external device, randomness may not be ensured due to forgery of the terminal device 4 or the like. However, since the guide frame F is displayed on the screen of the terminal device 4 side due to the random value obtained from the server 2 of a different device, it is difficult to prepare in advance an unauthorized image of the identity certificate that matches such a guide frame F, so the reliability of the identity verification can be enhanced.

[0066] (4) The random display mode is such that at least any one of the size of the guide frame F, the position of the guide frame F, and the type of the trapezoidal shape of the guide frame F is randomly determined. Thereby, it is possible to suppress passing the identity verification procedure by an unauthorized image of the identity certificate.

[0067] (5) The display unit 42 is configured to display a guidance G for guiding the user so that the contour of the identity certificate can be made to match the guide frame F when the shooting conditions are not satisfied. Specifically, when the determination unit 431 of the processing unit 43 determines that the shooting conditions are not satisfied, the processing unit 43 has a display control unit that causes the display unit 42 to display the guidance G for guiding the user so that the contour of the identity certificate can be made to match the guide frame F. Thereby, the convenience of the user can be improved.

[0068] (6) The image extraction unit 433 masked a predetermined area R in the extracted image where the confidential information of the identity certificate was described. This can reduce the labor of masking. That is, the image of the identity certificate may be stored in an external device such as the server 2. In that case, the masking process is performed on the corresponding part and then stored so as not to store unnecessary information of the identity certificate. The confidential information is information unnecessary for personal identification, and since the area where the information is described is masked in advance, masking on the external device side can be made unnecessary.

[0069] [Other Embodiments] The present invention is not limited to the above-described embodiment, but also includes other embodiments shown below. Further, the present invention includes a form in which all or any combination of the above-described embodiment and the following other embodiments are combined. Furthermore, various omissions, replacements, and changes can be made without departing from the scope of the invention, and such modifications are also included in the present invention.

[0070] For example, although the image extraction unit 433 is provided in the processing unit 43, it is not necessarily provided. That is, the image automatically captured by the imaging means 41 and the imaging control unit 432 may be transmitted to the server 2 via the communication unit 45. Also, although the predetermined area R is masked by the image extraction unit 433, it is not necessarily masked.

[0071] Although the table stored in the storage unit 44 is used for the display control unit 434 to determine the guide frame F, a function that determines one of the patterns of the guide frame F using a random value from the server 2 as an input may be used. Also, it is possible to transmit the size of the guide frame F, the position of the guide frame F, and the type of the trapezoidal shape of the guide frame F from the server 2 to the terminal device 4 as random values. In this case, the terminal device 4 specifies the size of the guide frame F, the position of the guide frame F, and the type of the trapezoidal shape of the guide frame F from the received random values, and displays the guide frame F at a relative position corresponding to the size of the display unit 42 of the terminal device 4. Thereby, the guide frame F corresponding to the display unit 42 of various sizes can be displayed, and it becomes easy to ensure the liveness.

Description of Symbols

[0072] 1 System 2 Server 3 Network 4 Terminal device 41 Imaging means 42 Display unit 43 Processing unit 431 Judgment unit 432 Shooting control unit 433 Image extraction unit 434 Display control unit 44 Storage unit 45 Communication unit C Identity certificate E Outline of identity certificate F Guide frame F1 First frame F2 Second frame G Guidance M Margin area S1 Front surface of identity certificate S2 Side surface (thickness) of identity certificate

Claims

1. Imaging means for capturing an image including an identity certificate, A display unit for displaying an image of the imaging area of the imaging means on the screen, A processing unit for controlling the capture of the image, Comprising, The display unit displays a trapezoidal guide frame in a random display mode within the screen, and the guide frame has a margin area. The processing unit, A determination unit for determining whether the contour including the thickness of the identity certificate satisfies the imaging condition that the contour is along the guide frame by fitting within the margin area. An imaging control unit for causing the imaging means to capture the image when it is determined by the determination unit that the imaging condition is satisfied. Having, A terminal device characterized by the above.

2. Comprising a communication unit for communicating with an external device via a network, The processing unit, Further having an image extraction unit for generating an extraction image by extracting the inside of the contour including the thickness of the identity certificate from the image captured by the imaging control unit and the imaging means. The communication unit, Transmitting the extraction image to the external device. The terminal device according to claim 1, characterized by the above.

3. The external device is a server, The display unit displays the guide frame in a random display mode based on a random value transmitted from the server. The terminal device according to claim 2, characterized by the above.

4. The random display mode is such that at least one of the size of the guide frame, the position of the guide frame, and the type of the trapezoidal shape of the guide frame is randomly determined. The terminal device according to any one of claims 1 to 3, characterized by the above.

5. When the imaging condition is not satisfied, the display unit displays guidance for guiding the user so that the contour of the identity certificate can be made to coincide with the guide frame. The terminal device according to any one of claims 1 to 4, characterized by the above.

6. The processing unit, When the determination unit determines that the imaging condition is not satisfied, having a display control unit for causing the display unit to display guidance for guiding the user so that the contour of the identity certificate can be made to coincide with the guide frame. The terminal device according to any one of claims 1 to 5, characterized by the above.

7. The image extraction unit masks a predetermined area in the extraction image where the minute information of the identity certificate is described. The terminal device according to claim 2 or 3, characterized by the above.

8. A method for controlling a terminal device including imaging means for capturing an image including an identity certificate, and a display unit for displaying an image of a shooting area of the imaging means on a screen, comprising: A guide frame display step of displaying a trapezoidal guide frame in a random display mode within the screen of the display unit, the guide frame having a margin area; A determination step of determining whether the contour including the thickness of the identity certificate fits within the margin area and thus satisfies the shooting condition that the contour is along the guide frame; A shooting control step of causing the imaging means to capture the image when it is determined by the determination step that the shooting condition is satisfied; Comprising; A method for controlling a terminal device, characterized by the above.

9. A program executed on a terminal device including imaging means for capturing an image including an identity certificate, a display unit for displaying an image of a shooting area of the imaging means on a screen, and a computer, comprising: Causing the computer to: A guide frame display step of displaying a trapezoidal guide frame in a random display mode within the screen of the display unit, the guide frame having a margin area; A determination step of determining whether the contour including the thickness of the identity certificate fits within the margin area and thus satisfies the shooting condition that the contour is along the guide frame; A shooting control step of causing the imaging means to capture the image when it is determined by the determination step that the shooting condition is satisfied; To execute; A program, characterized by the above.

10. A program executed on a terminal device including imaging means for capturing an image including an identity certificate, a display unit for displaying an image of a shooting area of the imaging means on a screen, and a computer, comprising: Causing the computer to: An acquisition step of acquiring a random value for specifying the size and position of a guide frame displayed within the screen of the display unit; A display step of displaying the guide frame on the display unit by determining the numerical range to which the random value belongs; A determination step of determining whether the contour including the thickness of the identity certificate satisfies the shooting condition that the contour is along the guide frame; A shooting control step of causing the imaging means to capture the image when it is determined by the determination step that the shooting condition is satisfied; To execute; A program, characterized by the above.

11. A program executed on a terminal device including an imaging unit that captures an image including an identity document, a display unit that displays an image of the imaging area of the imaging unit on a screen, and a computer, wherein the computer is caused to perform an acquisition step of acquiring a random value for specifying the size and position of a guide frame displayed within the screen of the display unit; perform a display step of specifying the random value and displaying the guide frame at a relative position corresponding to the size of the display unit; perform a determination step of determining whether or not a photographing condition that the contour including the thickness of the identity document is along the guide frame is satisfied; perform a photographing control step of causing the imaging unit to capture the image when it is determined in the determination step that the photographing condition is satisfied; and cause the above to be executed, a program characterized by the above.

Images