Information processing method, program, and information processing apparatus

Abstract

To increase the probability that an image of an identification card taken under a desired condition is provided.SOLUTION: An information processing device includes a detection unit that, in response to acquiring an image of an identification card at a first angle, acquires an image of the identification card at a third angle greater than the first angle and smaller than a second angle greater than the first angle, before acquiring an image of the identification card at the second angle, and an image output unit that outputs output information including at least one of the acquired images.SELECTED DRAWING: Figure 3

Description

Technical Field

[0001] The present invention relates to the analysis of images of identity certificates.

Background Art

[0002] When opening a bank account or issuing a credit card, identity verification using an identity certificate is performed. And in cases such as opening an account via the Internet, instead of the original identity certificate, an image obtained by imaging the identity certificate with a camera may be used for identity verification.

[0003] In cases where identity verification is performed using an image of an identity certificate, it is necessary to prevent forgery. Patent Document 1 discloses a system for verifying that an identity document belongs to a user by comparing the imaging data of the face photo of the identity document with the imaging data of the user.

[0004] Also, in the system of Patent Document 1, in order to obtain images of multiple sides of an identity document (identity certificate), on the user terminal, while giving instructions such as "Please take a picture of the front surface of the identity document" and "Please take a picture of the back surface of the identity document", a video of the identity document being imaged is generated. Then, the video is transmitted to the authentication server.

[0005] Here, the timing of the above instructions is predetermined as a relative time from the start of video shooting. The authentication server uses, as images of the front or back surface of the identity document, the images corresponding to the timing of each instruction (that is, the images at a predetermined timing starting from the start point of video shooting) from the received video.

Prior Art Documents

Patent Documents

[0006]

Patent Document 1

Summary of the Invention

[0007] In the system described in Patent Document 1, the authentication server determines whether the video sent from the user terminal to the authentication server contains images of the identity verification document (e.g., images of the front or back) taken in the desired state. Therefore, the video received by the authentication server may not contain images of the identity verification document taken in the desired state. In this case, an authentication error is sent to the user terminal, and the user terminal needs to retake the video recording.

[0008] The present invention has been made in view of the above-mentioned problems, and one of its objectives is to provide a technology that increases the probability of providing an image of an identification document captured in a desired state. [Means for solving the problem]

[0009] The information processing method of the present invention is One or more computers, In response to obtaining an image of the identification document at a first angle, and before obtaining an image of the identification document at a second angle greater than the first angle, an image of the identification document at a third angle greater than the first angle and smaller than the second angle is obtained. Output information including at least one of the acquired images, Based on the background of the image of the identification document at the first angle and the background of the image of the identification document at the third angle, Encourage imaging to be performed in a situation where the background does not change. Display the message on the display device.

[0010] The program of the present invention, On one or more computers, In response to obtaining an image of the identification document at a first angle, and before obtaining an image of the identification document at a second angle greater than the first angle, an image of the identification document at a third angle greater than the first angle and smaller than the second angle is obtained. Output information including at least one of the acquired images, Based on the background of the image of the identification document at the first angle and the background of the image of the identification document at the third angle, Encourage imaging to be performed in a situation where the background does not change. This is a program that causes a message to be displayed on a display device.

[0011] The information processing device of the present invention is A detection means that, in response to obtaining an image of the identification document at a first angle, obtains an image of the identification document at a second angle greater than the first angle, and then obtains an image of the identification document at a third angle greater than the first angle and smaller than the second angle, Output information including at least one of the acquired images, and based on the background of the image of the identification document at the first angle and the background of the image of the identification document at the third angle, Encourage imaging to be performed in a situation where the background does not change. It includes an image output means for displaying a message on a display device. [Effects of the Invention]

[0012] The present invention provides a technique that increases the probability of providing an image of an identification document captured in a desired state. [Brief explanation of the drawing]

[0013] [Figure 1] This is a diagram illustrating the overview of the image analysis device of this embodiment. [Figure 2] This is a plan view illustrating how an identification document is photographed using a camera at an angle X. [Figure 3] This figure illustrates the functional configuration of the image analysis device of Embodiment 1. [Figure 4] This diagram illustrates a computer used to realize an image analysis device. [Figure 5] This is a flowchart illustrating the processing flow performed by the image analysis device of Embodiment 1. [Figure 6]It is a diagram illustrating the usage environment of the image analysis device. [Figure 7] It is a block diagram illustrating the functional configuration of the image analysis device according to Embodiment 2. [Figure 8] It is a flowchart illustrating the flow of processing executed by the image analysis device according to Embodiment 2. [Figure 9] It is a diagram illustrating the guide output by the guide output unit. [Embodiment for Carrying Out the Invention]

[0014] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all the drawings, the same components are denoted by the same reference numerals, and the description will be omitted as appropriate. Also, in each block diagram, unless otherwise specified, each block represents a configuration of a functional unit rather than a hardware unit configuration.

[0015] [Embodiment 1] [Overview] FIG. 1 is a diagram for explaining the overview of the image analysis device 2000 of the present embodiment. Note that FIG. 1 is an illustration for facilitating the understanding of the image analysis device 2000, and the functions of the image analysis device 2000 are not limited to those shown in FIG. 1.

[0016] The image analysis device 2000 analyzes a plurality of captured images 30 including the identification document 20 of the user 10. The camera 40 is a camera that generates the captured image 30. The camera 40 generates a time-series captured image 30 by repeatedly capturing the identification document 20 of the user 10. For example, the time-series captured image 30 constitutes one moving image. The identification document 20 is any document that can be used to prove a person's identity. For example, the identification document 20 is a driver's license or other license, a passport, various certification documents, a student ID, a company identification document, or an insurance certificate.

[0017] The captured image 30 is used to verify the identity of user 10. For example, there are cases where users are asked to present an image containing their identification document instead of the original document. An example of this is when opening a bank account or applying for a credit card via the internet. In such cases, it is difficult to present the original identification document. Therefore, the user 10's identity is verified using image data obtained by photographing the identification document (such as the captured image 30 mentioned above).

[0018] In cases where identity verification is performed using image data from identification documents, one possible method is for the user to specify image data of the main side (the side containing the main information) and the back side (hereinafter referred to as the back side) of the identification document, and then use the image data specified by the user for identity verification. However, this method makes it difficult to prevent the fraudulent use of identification documents. For example, if a user can somehow obtain copies of the main side and back side of another person's identification document, they can impersonate that person by providing image data of these copies taken with a camera.

[0019] To address this problem, the image analysis device 2000 confirms that the identification document 20 has been captured at n different angles (where n is an integer greater than or equal to 2), and then outputs the captured image 30 containing the identification document 20. Specifically, the image analysis device 2000 detects the captured image 30 that satisfies each of the n predetermined conditions. That is, the image analysis device 2000 detects the captured image 30 that satisfies the first predetermined condition, the captured image 30 that satisfies the second predetermined condition, ..., and the captured image 30 that satisfies the nth predetermined condition. Hereinafter, the process of detecting the captured image 30 that satisfies the ith predetermined condition (where i is an integer satisfying 1 ≤ i ≤ n) will be referred to as the ith detection process.

[0020] The i-th predetermined condition includes the condition that "the identification document 20 captured at the i-th predetermined angle is included in the captured image 30." Therefore, in the first to nth detection processes, the captured image 30 containing the identification document 20 captured at the first predetermined angle, the captured image 30 containing the identification document 20 captured at the second predetermined angle, ..., and the captured image 30 containing the identification document 20 captured at the n-th predetermined angle are detected, respectively. Note that 0° ≤ first predetermined angle < second predetermined angle < ... < n-th predetermined angle < 360°. By detecting the captured image 30 that satisfies each of the n predetermined conditions in this way, it can be confirmed that the identification document 20 was captured at n different angles.

[0021] Figure 2 is a plan view showing how the camera 40 captures an image of the identification document 20 at an angle X. As shown in Figure 2, "capturing an image of the identification document 20 at an angle X" means "rotating the identification document 20 by an angle X from a state where the main surface of the identification document 20 is facing the front of the camera 40, and then capturing an image of the identification document 20 in that state." Therefore, at 0°, the main surface of the identification document 20 is captured; at 180°, the back surface of the identification document 20 is captured; and at 90° and 270°, the side surface of the identification document 20 is captured.

[0022] When the image analysis device 2000 detects all of the captured images 30 that satisfy the nth predetermined condition from the captured images 30 that satisfy the first predetermined condition, it outputs one or more of these detected n captured images 30. That is, one or more of the captured images 30 containing the identification card 20 captured at the first predetermined angle, the captured images 30 containing the identification card 20 captured at the second predetermined angle, ..., the captured images 30 containing the identification card 20 captured at the nth predetermined angle are output.

[0023] <Typical effects and benefits> According to the image analysis device 2000 of this embodiment, after confirming that the identification document 20 has been captured at n different angles, the captured image 30 containing the identification document 20 is output. Therefore, compared to the case where an image of the identification document 20 is output without such confirmation, the probability that the output captured image 30 will contain an image of the identification document 20 captured in the desired state can be increased. As a result, when the captured image 30 output from the image analysis device 2000 is used for verifying the identity of the user 10, the captured image 30 necessary for identity verification can be obtained more reliably from the image analysis device 2000.

[0024] The following describes this embodiment in more detail.

[0025] <Example of functional configuration> Figure 3 is a diagram illustrating the functional configuration of the image analysis device 2000 of Embodiment 1. The image analysis device 2000 has a detection unit 2020 and an image output unit 2040. The detection unit 2020 performs each of the i-th detection processes described above. As described above, since 1 ≤ i ≤ n and n ≥ 2, the detection unit 2020 performs at least a first detection process to detect an image 30 that satisfies a first predetermined condition, and a second detection process to detect an image 30 that satisfies a second predetermined condition. The image output unit 2040 outputs one or more of the multiple image 30 detected by the detection unit 2020.

[0026] <Example of hardware configuration for image analysis device 2000> Each functional component of the image analysis device 2000 may be implemented by hardware (e.g., hardwired electronic circuits) or by a combination of hardware and software (e.g., a combination of electronic circuits and programs that control them). The following will further explain the case where each functional component of the image analysis device 2000 is implemented by a combination of hardware and software.

[0027] Figure 4 illustrates a computer 1000 for realizing the image analysis device 2000. Computer 1000 is any computer. For example, computer 1000 could be a portable computer such as a smartphone or tablet. Alternatively, computer 1000 could be a stationary computer such as a PC (Personal Computer) or a server machine.

[0028] Computer 1000 may be a dedicated computer designed to implement the image analysis device 2000, or it may be a general-purpose computer. In the latter case, for example, the functions of the image analysis device 2000 are realized on computer 1000 by installing a predetermined application on computer 1000. The above application consists of programs for realizing each functional component of the image analysis device 2000. That is, this program causes computer 1000 to execute the processing performed by the detection unit 2020 and the processing performed by the image output unit 2040, respectively.

[0029] Computer 1000 includes a bus 1020, a processor 1040, a memory 1060, a storage device 1080, an input / output interface 1100, and a network interface 1120. The bus 1020 is a data transmission path for the processor 1040, memory 1060, storage device 1080, input / output interface 1100, and network interface 1120 to send and receive data from each other. However, the method of connecting the processor 1040 and other components is not limited to bus connection.

[0030] Processor 1040 is a variety of processors such as CPU (Central Processing Unit), GPU (Graphics Processing Unit), and FPGA (Field-Programmable Gate Array). Memory 1060 is main memory implemented using RAM (Random Access Memory), etc. Storage device 1080 is auxiliary storage implemented using hard disk, SSD (Solid State Drive), memory card, or ROM (Read Only Memory), etc.

[0031] The input / output interface 1100 is an interface for connecting the computer 1000 with input / output devices. For example, input devices such as keyboards and output devices such as display devices are connected to the input / output interface 1100.

[0032] In addition, for example, a camera 40 is connected to the input / output interface 1100. In this way, each captured image 30 generated by the camera 40 is input to the computer 1000. The captured images 30 are stored in the memory 1060 or the storage device 1080.

[0033] The network interface 1120 is an interface for connecting the computer 1000 to a communication network. This communication network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network).

[0034] The storage device 1080 stores program modules (program modules that implement the aforementioned applications) that realize each functional component of the image analysis device 2000. The processor 1040 reads these program modules into the memory 1060 and executes them to realize the functions corresponding to each program module.

[0035] <About Camera 40> Camera 40 is any camera that takes images and generates image data (imported image 30) representing the results. For example, camera 40 is a camera built into a smartphone, tablet device, or notebook PC. However, camera 40 may also be an external camera attached to the image analysis device 2000.

[0036] <Processing flow> Figure 5 is a flowchart illustrating the processing flow performed by the image analysis device 2000 of Embodiment 1. S102 to S108 is a loop process that performs the first detection process to the nth detection process. In S102, the detection unit 2020 determines whether i ≤ n. The initial value of i is set to 1.

[0037] If i ≤ n, the process in Figure 4 proceeds to S104. On the other hand, if i ≤ n is not true, the process in Figure 4 proceeds to S112.

[0038] In S104, the image output unit 2040 detects an captured image 30 that satisfies the i-th predetermined condition (executes the i-th detection process). The detection unit 2020 adds 1 to i (S106). Since S08 is the end of loop processing A, the process in Figure 4 proceeds to S102.

[0039] When the process shown in Figure 4 reaches S110, the image output unit 2040 outputs one or more of the detected n captured images 30.

[0040] Here, the flowchart in Figure 4 does not show the process to be performed when an image 30 satisfying the i-th predetermined condition is not detected in the i-th detection process. The process performed by the image analysis device 2000 when an image 30 satisfying the i-th predetermined condition is not detected in the i-th detection process is arbitrary. For example, the image analysis device 2000 may terminate the process shown in Figure 4. In other words, in this case, the image 30 will not be output. Here, before terminating the process shown in Figure 4, the image analysis device 2000 may output a warning message indicating that an image 30 satisfying the predetermined condition was not detected (i.e., the image of the identification card 20 was not captured correctly).

[0041] In addition, for example, if the image analysis device 2000 does not detect an image 30 that satisfies the i-th predetermined condition in the i-th detection process, it may output a warning message or the like instructing the device to correctly image the identification document 20 so that the i-th predetermined condition is met (for example, to image the identification document 20 at the i-th predetermined angle), and then execute the i-th detection process again.

[0042] The conditions for terminating the i-th detection process when no captured image 30 satisfying the i-th predetermined condition is detected are arbitrary. For example, the detection unit 2020 terminates the i-th detection process when a predetermined time has elapsed since the start of the i-th detection process, or when the i-th detection process has been performed on a predetermined number of captured images 30 or more.

[0043] <Example of the usage environment for the Image Analysis System 2000> To make the following explanation easier to understand, a more specific usage environment for the image analysis device 2000 will be provided as an example. However, the usage environment for the image analysis device 2000 is not limited to that described here.

[0044] Figure 6 illustrates an example of the usage environment for the image analysis device 2000. In this example, the image analysis device 2000 is implemented on a user terminal 50. The user terminal 50 is, for example, a smartphone equipped with a camera 40.

[0045] User 10 uses user terminal 50 to provide an image of the identification document 20 to server device 60. For example, user terminal 50 has an application installed that allows it to function as an image analysis device 2000. User 10 starts and operates this application. As a result, a message prompting user 10 to take an image of the identification document 20 is displayed on the user terminal 50's display device, and camera 40 is activated. User 10 takes an image of the identification document 20 using camera 40. For example, user 10 rotates the identification document 20 while having camera 40 take an image of it.

[0046] The user terminal 50 sequentially analyzes the time-series captured images 30 generated by the camera 40 through the operations described above. For example, the user terminal 50 performs a first detection process on each captured image 30, starting from the first captured image 30 in the time series. If the first detection process detects an captured image 30 that satisfies a first predetermined condition, the user terminal 50 performs a second detection process on each captured image 30 generated after that captured image 30. Furthermore, if the second detection process detects an captured image 30 that satisfies a second predetermined condition, the user terminal 50 performs a third detection process on each captured image 30 generated after that captured image 30. The user terminal 50 then performs the nth detection process in the same manner.

[0047] For example, in the example shown in Figure 6, the first to fourth detection processes are performed. The first to fifth predetermined angles are 0°, 45°, 135°, and 180°, respectively. That is, four images 30 are detected by the detection unit 2020: an image 30 of the main face of the identification card 20 taken from the front, an image 30 of the main face of the identification card 20 taken from a 45° angle, an image 30 of the back of the identification card 20 taken from a 45° angle, and an image 30 of the back of the identification card 20 taken from the front.

[0048] The user terminal 50 provides the server device 60 with at least one of the captured images 30 detected in each detection process. For example, all four of the aforementioned captured images 30 are sent to the server device 60. These captured images 30 are used to verify the identity of the user 10. Any method can be used to verify the user's identity using the images of the identification document 20.

[0049] Here, by executing the detection processes described above, the captured image 30, which includes the identification document 20 captured at a predetermined angle, is transmitted from the user terminal 50 to the server device 60. Therefore, when the identification document 20 captured at a predetermined angle is necessary for identity verification, it is possible to prevent a situation where "the server device 60 requests the user terminal 50 to resubmit the captured image 30 because the identification document 20 was not captured at the predetermined angle." This makes it possible to verify the identity of user 10 more smoothly.

[0050] As mentioned above, the operating environment for the image analysis device 2000 is not limited to that described here. For example, the image analysis device 2000 is not limited to a mobile device such as a smartphone. For instance, a desktop PC may be used as the image analysis device 2000, and a camera connected to that desktop PC may be used as camera 40.

[0051] <Acquisition of captured image 30> The detection unit 2020 acquires the captured image 30 and performs various detection processes. There are various ways in which the detection unit 2020 acquires the captured image 30. For example, the detection unit 2020 receives the captured image 30 transmitted from the camera 40. Alternatively, for example, the detection unit 2020 can access the camera 40 and acquire the captured image 30 stored in the camera 40.

[0052] The camera 40 may store the captured image 30 in a storage device (for example, a storage device 1080) located outside the camera 40. In this case, the detection unit 2020 accesses this storage device to acquire the captured image 30.

[0053] The timing at which the detection unit 2020 acquires the captured image 30 is arbitrary. For example, the detection unit 2020 acquires the newly generated captured image 30 each time the camera 40 generates one. Alternatively, the detection unit 2020 may periodically acquire any unacquired captured images 30. For example, if the detection unit 2020 acquires the captured image 30 once per second, the detection unit 2020 will acquire one or more captured images 30 generated per second (for example, if the camera 40 is a video camera with a frame rate of 30 fps (frames / second), then 30 captured images 30) all at once.

[0054] <Execution of detection process: S104> The detection unit 2020 executes the first detection process to the nth detection process (S104). There are various ways to implement the detection process. For example, the detection unit 2020 is equipped with a classifier (hereinafter referred to as the ith classifier) ​​that has been trained to determine whether or not the captured image 30 satisfies the ith predetermined condition. The ith classifier outputs a determination result indicating whether or not the captured image 30 satisfies the ith predetermined condition when the captured image 30 is input. For example, the first classifier outputs a determination result indicating whether or not the captured image 30 satisfies the first predetermined condition when the captured image 30 is input. Similarly, the second classifier outputs a determination result indicating whether or not the captured image 30 satisfies the second predetermined condition when the captured image 30 is input. For example, the determination result is a flag that shows 1 if the captured image 30 satisfies the ith predetermined condition and 0 if the captured image 30 does not satisfy the ith predetermined condition. Here, various models such as neural networks and SVMs (support vector machines) can be used as the classifier model.

[0055] Identifiers are provided for each type of identification document. For example, if a driver's license and a passport are to be used as identification documents, both an identifier for the image 30 containing the driver's license and an identifier for the image 30 containing the passport are provided. Furthermore, if any of multiple types of identification documents can be used for identity verification, the user 10 will specify in advance which type of identification document is included in the image 30 provided by the user 10 (i.e., which type of identification document the user will provide for identity verification).

[0056] The classifier is pre-trained to perform the aforementioned processing. Specifically, the i-th classifier is trained using positive example data such as "image satisfying the i-th predetermined condition, classification result = 1" and negative example data such as "image not satisfying the i-th predetermined condition, classification result = 0" as training data. Existing techniques can be used to train the classifier using positive and negative example data.

[0057] For example, suppose the i-th predetermined condition is "includes an identification document 20 photographed at the i-th predetermined angle." In this case, the images included in the positive example data are images that include an identification document 20 photographed at the i-th predetermined angle. Conversely, the images included in the negative example data are images that do not include an identification document 20 photographed at the i-th predetermined angle.

[0058] Here, the classifier is not used only for the identification documents of a specific individual, but for the identification documents of various users. Therefore, the identification documents included in the images used for training do not need to perfectly match the identification document 20 to be detected; it is sufficient if they capture to some extent the features of the same type of identification document as the identification document 20 to be detected. For example, if a driver's license is used as the identification document 20, the positive example data used to train the i-th classifier only needs to include images that show the features of a driver's license viewed from a predetermined i-th angle.

[0059] Therefore, the images used for training do not necessarily have to be images of official identification documents (for example, original identification documents issued by government agencies). For example, the images used for training can be generated by taking images of sample identification documents 20. In addition, the images used for training can also be artificially generated using technologies such as GANs (Generative Adversarial Networks).

[0060] Furthermore, in the condition that "the image includes an identification document 20 photographed at the i-th predetermined angle," some deviation in the angle of the identification document 20 may be permitted. For example, when using the predetermined condition that "the image includes an identification document 20 photographed at a 45° angle," images 30 in which the identification document 20 is photographed at 44° or 46° can also be treated as satisfying the predetermined condition. Such an i-th classifier that can tolerate some error can be constructed, for example, by using not only images in which the identification document 20 is photographed at the i-th predetermined angle, but also images in which the identification document 20 is photographed at an angle that deviates from the i-th predetermined angle within the range of acceptable error, as positive example data images used for training.

[0061] The method for implementing each detection process is not limited to using a classifier. For example, suppose the i-th predetermined condition is "includes an identification document 20 captured at the i-th predetermined angle." In this case, for each i-th predetermined angle, image features of the image region representing the identification document 20 captured at the i-th predetermined angle (hereinafter referred to as the i-th image feature) are prepared and stored in advance in a storage device accessible from the detection unit 2020. The detection unit 2020 uses the image features stored in this storage device.

[0062] For example, the detection unit 2020 determines whether the captured image 30 contains an image feature that has a high degree of similarity to the i-th image feature (i.e., the similarity is above a predetermined threshold). If the captured image 30 contains an image feature that has a high degree of similarity to the i-th image feature, the detection unit 20 determines that the captured image 30 contains an identification document 20 captured at the i-th predetermined angle. On the other hand, if the captured image 30 does not contain an image feature that has a high degree of similarity to the i-th image feature, the detection unit 20 determines that the captured image 30 does not contain an identification document 20 captured at the i-th predetermined angle.

[0063] Here, similar to the training of the classifier described above, the image used to generate the i-th image feature does not necessarily have to be a photograph of an official identification document. For example, image features extracted from an image generated by photographing a replica of identification document 20, or image features extracted from an image artificially generated using techniques such as GANs, may be used.

[0064] <Timing of each detection process> The first to the nth detection processes may be executed in parallel, in any order, or in a predetermined order. When executed in a predetermined order, for example, the detection unit 2020 executes the first to the nth detection processes one by one in this order, as shown in the flowchart of Figure 5. That is, the detection unit 2020 performs the (i+1)th detection process when an image 30 satisfying the ith predetermined condition is detected in the ith detection process. In other words, detection of an image 30 satisfying the (i+1)th predetermined condition will not be performed until an image 30 satisfying the ith predetermined condition is detected.

[0065] <Other conditions included in the i-th prescribed condition> The i-th predetermined condition may include other conditions in addition to the condition "includes an image area representing the identification document 20 captured at the i-th predetermined angle." For example, suppose the identification document 20 includes a photograph of the person's face. In this case, if the image is captured so that the face of the person who provided the identification document 20 is included in the captured image 30 as well as the identification document 20 (see captured image 30 in Figure 1), then by determining the degree of agreement between the image of the person who provided the identification document 20 included in the captured image 30 and the face image of the identification document 20 included in the captured image 30, it is possible to determine whether the person who provided the identification document 20 is the legitimate owner of the identification document 20 (the person whose identity is proven by the identification document 20).

[0066] For example, the predetermined conditions may include the condition that "the degree of agreement between the image of the provider of the identification document 20 contained in the captured image 30 and the face image of the identification document 20 contained in the captured image 30 meets the standard (the degree of agreement is above a threshold)." In this case, for example, the detection unit 2020 extracts the face of the provider of the identification document 20 and the face image of the identification document 20 from the captured image 30 and calculates the degree of agreement between them. Existing technologies can be used for the technology to calculate the degree of agreement between face images.

[0067] Furthermore, depending on the angle at which the identification document 20 is captured, the facial image of the identification document 20 may not be included in the captured image 30, or facial features may not be accurately extracted from the facial image included in the captured image 30. For this reason, it is preferable to include only the condition that the i-th predetermined angle is the angle at which the identification document 20 is captured in a way that allows sufficient extraction of facial features from the facial image included in the identification document 20, in the conditions regarding the degree of agreement between the face of the provider of the identification document 20 and the facial image of the identification document 20. For example, if the facial image is included on the main surface of the identification document 20 and the first predetermined angle is 0° (the angle at which the main surface of the identification document 20 is captured from the front), then the condition that "the degree of agreement between the facial image of the provider of the identification document 20 included in the captured image 30 and the facial image of the identification document 20 included in the captured image 30 satisfies the standard" should be included only in the first predetermined condition.

[0068] For example, the i-th predetermined condition includes a condition relating to the background of the identification document 20 in the captured image 30 (hereinafter, the background of the captured image 30). Specifically, the i-th predetermined condition may include a condition that the degree of agreement between the background of the captured image 30 to be judged and the background of the captured image 30 detected in the (i-1) detection process or earlier detection processes meets a standard (for example, the degree of agreement is equal to or greater than a standard value). By using such predetermined conditions, the degree of background agreement between the captured images 30 detected in each detection process will increase. Therefore, it is possible to prevent fraud such as tampering with a series of captured images 30. Note that if the first detection process to the nth detection process is performed in this order, the first predetermined condition does not need to include a background condition.

[0069] For example, the i-th predetermined condition is that "the degree of agreement between the background of the captured image 30 detected in the i-th detection process and the background of the captured image 30 detected in the (i-1) detection process satisfies the criterion." In other words, the degree of agreement between the backgrounds of the captured images 30 detected in each of the two consecutive detection processes satisfies the criterion.

[0070] In addition, for example, the i-th predetermined condition is that "the degree of agreement between the background of the captured image 30 detected in the i-th detection process and the background of the captured image 30 detected in the first detection process satisfies the criterion." In other words, the degree of agreement between the background of each captured image 30 detected in the second to n-th detection processes and the background of the first detected (detected in the first detection process) captured image 30 satisfies the criterion.

[0071] The degree of background similarity between two different captured images 30 can be determined in various ways. For the sake of clarity, the two captured images 30 being compared are referred to as captured image A and B. For example, the detection unit 2020 calculates the background image features for each of captured image A and B. Here, the background of captured image 30 is the portion of captured image 30 from which the image region representing the identification document 20 has been removed (captured image 30 with the image region representing the identification document 20 masked). The detection unit 2020 then determines whether the degree of similarity of the background image features is equal to or greater than a standard value. If the degree of similarity of the background image features is equal to or greater than the standard value, the detection unit 2020 determines that the degree of background similarity between captured image A and B meets the standard. On the other hand, if the degree of similarity of the background image features is not equal to or greater than the standard value, the detection unit 2020 determines that the degree of background similarity between captured image A and B does not meet the standard.

[0072] The detection unit 2020 may divide the background comparison into two parts: 1) comparison of the user 10's face, and 2) comparison of the rest of the background. For example, it may do so as follows: First, the detection unit 2020 calculates the image features of the image region representing the face for each of the captured images A and B. Then, the detection unit 2020 calculates the degree of agreement between the image features of the face in captured image A and the image features of the face in captured image B.

[0073] Furthermore, the detection unit 2020 calculates the image features of the background other than the face (the captured image 30 excluding the image area of ​​the identification document 20 and the image area of ​​the face) for both captured image A and B. The detection unit 2020 then calculates the degree of agreement between the image features of the background other than the face in captured image A and the image features of the background other than the face in captured image B.

[0074] The detection unit 2020 determines that the degree of matching of the backgrounds of captured images A and B meets the criteria if both the degree of matching of the image features of the face and the degree of matching of the image features of the background other than the face are above a threshold. On the other hand, if at least one of the degree of matching of the image features of the face and the degree of matching of the image features of the background other than the face is not above a threshold, the detection unit 2020 determines that the degree of matching of the backgrounds of captured images A and B does not meet the criteria.

[0075] <Output of detected captured image 30> The image output unit 2040 outputs information that includes one or more of the n captured images 30 detected in the first to nth detection processes. Hereinafter, this information will be referred to as output information. For example, the output information is information that associates the user 10's identification information with one or more captured images 30. By associating it with the user 10's identification information, it is possible to identify whose identity the captured image 30 included in the output information is used to verify. In other words, the captured image 30 included in the output information is used to verify the identity of the user 10 identified by the identification information included in that output information.

[0076] However, if the device receiving the output information (such as the server device 60 in Figure 6) can associate the captured image 30 with the user 10's identification information, the user 10's identification information does not need to be included in the output information. For example, a predetermined connection can be established between the server device 60 and the image analysis device 2000, and the user 10's identification information and output information can be transmitted from the image analysis device 2000 to the server device 60 via that connection. With this method, even if the user 10's identification information and the output information are transmitted at different times, the server device 60 can associate the user 10's identification information with the output information (i.e., the captured image 30 included in the output information). For the sake of simplicity, unless otherwise specified, it will be assumed that the user 10's identification information is included in the output information below.

[0077] The output information may include all captured images 30 detected by the detection unit 2020, or it may include only some of the captured images 30. In the latter case, for example, the output information may include only the captured images 30 detected by a predetermined number of n detection processes or less.

[0078] In this case, the captured image 30 showing the main face of the identification document 20 is considered to be highly useful in verifying the identity of user 10. Therefore, it is preferable that the output information includes at least the captured image 30 showing the main face of the identification document 20. In addition, the information written on the back of the identification document 20 may also be important. In this case, the captured image 30 showing the back of the identification document 20 is also considered to be highly useful in verifying the identity of user 10. Therefore, in this case, it is preferable that the output information also includes the captured image 30 showing the back of the identification document 20.

[0079] Furthermore, it can be determined in advance which detection process will detect the main and back sides of the identification document 20, respectively. For example, suppose the first predetermined angle is 0° (the main side of the identification document 20 is facing the camera 40) and the nth predetermined angle is 180° (the back side of the identification document 20 is facing the camera 40). In this case, the captured image 30 that includes the main side of the identification document 20 is the captured image 30 detected in the first detection process, and the captured image 30 that includes the back side of the identification document 20 is the captured image 30 detected in the nth detection process. Therefore, the image output unit 2040 includes at least the captured image 30 detected in the first detection process and the captured image 30 detected in the nth detection process in its output information.

[0080] Furthermore, the output information may include not only the captured images 30 detected by the detection unit 2020, but also other captured images 30. For example, the image output unit 2040 may include all captured images 30 generated by the camera 40 in the output information. Alternatively, for example, the image output unit 2040 may include all captured images 30 in a time series from the captured image 30 detected in the first detection process to the captured image 30 detected in the nth detection process in the output information. For example, in this case, if the first predetermined angle is set to 0° and the nth predetermined angle is set to 180°, the output information will include a time series of captured images 30 (video) that captures the entire sequence from when the main side of the identification card 20 is facing the camera 40 to when the back side of the identification card 20 is facing the camera 40.

[0081] The process of verifying the identity of user 10 using the captured image 30 may be performed manually or automatically by a device. Furthermore, any method can be used to verify the user's identity using an image containing the user's identification document.

[0082] [Embodiment 2] Figure 7 is a block diagram illustrating the functional configuration of the image analysis device 2000 of Embodiment 2. Except for the points described below, the image analysis device 2000 of Embodiment 2 has the same functions as the image analysis device 2000 of Embodiment 1.

[0083] In the image analysis device 2000 of Embodiment 2, the premise is that the first detection process to the nth detection process are executed one by one in this order. That is, if the captured image 30 is detected in the ith detection process, the detection unit 2020 performs the (i+1)th detection process.

[0084] The image analysis device 2000 of Embodiment 2 has a guide output unit 2060. The guide output unit 2060 outputs a guide to the user 10 in order to increase the probability of obtaining an image 30 that satisfies each predetermined condition. Hereinafter, the guide output to increase the probability of obtaining an image 30 that satisfies the i-th predetermined condition will be referred to as the i-th guide.

[0085] The image analysis device 2000 outputs the i-th guide and then performs the i-th detection process. If the i-th detection process detects an image 30 that satisfies the i-th predetermined condition, the image analysis device 2000 outputs the (i+1)-th guide. Subsequently, the image analysis device 2000 outputs the (i+1)-th guide.

[0086] <Typical effects and benefits> According to the image analysis device 2000 of this embodiment, the probability that the identification document 20 will be imaged in a way that satisfies predetermined conditions is increased. Therefore, the probability that the image analysis device 2000 can detect an imaged image 30 that satisfies predetermined conditions in each detection process is increased.

[0087] Furthermore, by taking images of the identification document 20 according to the guide output by the image analysis device 2000, user 10 will be able to provide a correct image of the identification document 20. Therefore, the usability of the image analysis device 2000 for user 10 is improved.

[0088] The image analysis device 2000 of this embodiment will be described in more detail below.

[0089] <Processing flow> Figure 8 is a flowchart illustrating the processing flow performed by the image analysis device 2000 of Embodiment 2. The flowchart in Figure 8 is the same as the flowchart in Figure 4, except that the output of the i-th guide (S202) is added before the i-th detection process (S104).

[0090] <About the guide> The guide output unit 2060 outputs a guide that increases the probability of capturing an image 30 that satisfies predetermined conditions. For example, if the i-th predetermined condition is "includes an identification document 20 captured at the i-th predetermined angle", then the i-th guide is a guide that prompts the camera to capture the identification document 20 at the i-th predetermined angle.

[0091] Figure 9 illustrates the guide output by the guide output unit 2060. In this example, the image analysis device 2000 performs the first detection process to the fourth detection process, i.e., n=4. The first predetermined angle to the fifth predetermined angle are 0°, 45°, 135°, and 180°, respectively.

[0092] In this case, the guide output unit 2060 first outputs a first guide 70 containing information such as "Please stop with the main side facing the camera" to a display device that the user 10 can view (for example, the display device provided on the user terminal 50 in Figure 6). By looking at this first guide, the user 10 can understand that they should point the main side of the identification card 20 towards the camera 40.

[0093] When an image 30 that satisfies the first predetermined condition is detected, the guide output unit 2060 outputs a second guide 80 with a message such as "Slowly rotate it to 45° and stop." When an image 30 that satisfies the second predetermined condition is detected, the guide output unit 2060 outputs a third guide 90 with a message such as "Slowly rotate it to 135° and stop." When an image 30 that satisfies the third predetermined condition is detected, the guide output unit 2060 outputs a fourth guide 100 with a message such as "Rotate it until the back side faces the camera and stop."

[0094] The method of outputting the guide is not limited to outputting a message to a display device. For example, the guide output unit 2060 may output the aforementioned guide as audio.

[0095] In addition to the guide described above, a message may be displayed to inform the user 10 that an image 30 satisfying predetermined conditions has been detected. For example, in the example in Figure 9, after outputting the first guide 70, if an image 30 satisfying the first predetermined condition is detected, a message such as "OK" or "Shooting successful" is output to indicate that an image 30 satisfying the first predetermined condition has been detected. This message may be output simultaneously with the second guide 80, or before outputting the second guide 80. The same can be done when an image 30 satisfying other predetermined conditions is detected. By outputting a message indicating that an image 30 satisfying predetermined conditions has been detected in this way, the usability of the image analysis device 2000 for the user 10 is further improved.

[0096] Furthermore, as mentioned above, the specified conditions may include conditions related to background matching. For example, the guide may also include messages encouraging users to take images in situations where the background does not change, such as "Do not change the background," "Do not change the shooting location," or "Do not move."

[0097] The embodiments of the present invention have been described above with reference to the drawings, but these are merely examples of the present invention, and combinations of the above embodiments or various other configurations can also be adopted.

[0098] Some or all of the above embodiments may also be described as follows, but are not limited to the following: 1. A first detection process that detects an image that satisfies a first predetermined condition from one or more images generated by the camera, A second detection process for detecting an image that satisfies a second predetermined condition from one or more images generated by the aforementioned camera, When both the captured image that satisfies the first predetermined condition and the captured image that satisfies the second predetermined condition are detected, the computer is instructed to perform an image output process that outputs at least one of these captured images. The first predetermined condition includes that the captured image includes an image region representing an identification document captured at a first predetermined angle, The program includes the second predetermined condition being that an image region representing the identification document, captured at a second predetermined angle, is included in the captured image. 2. The second detection process is the program described in 1, which is executed after the captured image that satisfies the first predetermined condition is detected. 3. After detecting the captured image that satisfies the first predetermined condition, the computer is instructed to perform a guide output process that outputs a guide prompting the user to change the angle of the identification card to the second predetermined angle. The program described in 2., which executes the second detection process after the guide output process. 4. The program according to any one of 1 to 3, wherein the second predetermined condition further includes that the degree of agreement between the background of the identification document in the captured image to be processed in the second detection process and the background of the identification document in the captured image detected in the first detection process satisfies the criteria. 5. The captured image to be processed in the first detection process and the captured image to be processed in the second detection process are included in the video generated by the camera, and are the program described in any one of 1 to 4. 6. A detection unit that performs a first detection process for detecting an image that satisfies a first predetermined condition from one or more captured images generated by a camera, and a second detection process for detecting an image that satisfies a second predetermined condition from one or more captured images generated by the camera. The system includes an image output unit that outputs at least one of the captured images when both the captured image satisfying the first predetermined condition and the captured image satisfying the second predetermined condition are detected, The first predetermined condition includes that the captured image includes an image region representing an identification document captured at a first predetermined angle, The image analysis device includes the second predetermined condition that an image region representing the identification document, captured at a second predetermined angle, is included in the captured image. 7. The image analysis apparatus according to 6, wherein the second detection process is performed after the captured image satisfying the first predetermined condition is detected. 8. After detecting the captured image that satisfies the first predetermined condition, the device has a guide output unit that outputs a guide prompting the user to change the angle of the identification card to a second predetermined angle, The image analysis apparatus according to 7, wherein the detection unit performs the second detection process after the guide is output. 9. The image analysis apparatus according to any one of 6 to 8, wherein the second predetermined condition further includes that the degree of agreement between the background of the identification document in the captured image to be processed in the second detection process and the background of the identification document in the captured image detected in the first detection process satisfies the criteria. 10. The image analysis device according to any one of 6. to 9., wherein the captured image to be processed in the first detection process and the captured image to be processed in the second detection process are included in the video generated by the camera. 11. A control method performed by a computer, A first detection step involves detecting an image that satisfies a first predetermined condition from one or more captured images generated by a camera, A second detection step involves detecting an image that satisfies a second predetermined condition from one or more images generated by the camera, The system includes an image output step, in which, when both the captured image satisfying the first predetermined condition and the captured image satisfying the second predetermined condition are detected, at least one of these captured images is output. The first predetermined condition includes that the captured image includes an image region representing an identification document captured at a first predetermined angle, The control method includes the second predetermined condition being that an image region representing the identification document, captured at a second predetermined angle, is included in the captured image. 12. The control method according to 11, wherein the second detection step is performed after the captured image satisfying the first predetermined condition is detected. 13. After detecting the captured image that satisfies the first predetermined condition, the system has a guide output step that outputs a guide prompting the user to change the angle of the identification card to a second predetermined angle, The control method according to 12, wherein the second detection step is performed after the guide output step. 14. The control method according to any one of 11 to 13, wherein the second predetermined condition further includes that the degree of agreement between the background of the identification document in the captured image to be processed in the second detection step and the background of the identification document in the captured image detected in the first detection step satisfies a criterion. 15. The control method according to any one of 11 to 14, wherein the captured image to be processed in the first detection step and the captured image to be processed in the second detection step are included in the video generated by the camera.

[0099] This application claims priority based on Japanese Patent Application No. 2019-166160, filed on 12 September 2019, and incorporates all of its disclosures herein. [Explanation of Symbols]

[0100] 10 users 10 Target Products 20. Identification card 30 Acquired Images 40 Cameras 50 User Terminals 60 Server Devices 70 Guide 1 80 Guide 2 90 Third Guide 100 Guide 4 1000 calculator 1020 Bus 1040 processor 1060 memory 1080 Storage Devices 1100 Input / Output Interface 1120 Network Interface 2000 Image Analysis Device 2020 Detection Unit 2040 Image Output Unit 2060 Guide Output Unit

Claims

1. One or more computers, In response to obtaining an image of the identification document at a first angle, and before obtaining an image of the identification document at a second angle greater than the first angle, an image of the identification document at a third angle greater than the first angle and smaller than the second angle is obtained. Output information including at least one of the acquired images, Based on the background of the image of the identification document at the first angle and the background of the image of the identification document at the third angle, a message prompting the user to take images in a situation where the background does not change is displayed on the display device. Information processing methods.

2. Furthermore, in response to obtaining an image of the identification document at the first angle, the information at the third angle is displayed on the display device. The information processing method according to claim 1.

3. Furthermore, in response to acquiring an image of the identification document at the first angle, the display device will show a guide for acquiring an image of the identification document at the third angle. The information processing method according to claim 1 or 2.

4. On one or more computers, In response to obtaining an image of the identification document at a first angle, and before obtaining an image of the identification document at a second angle greater than the first angle, an image of the identification document at a third angle greater than the first angle and smaller than the second angle is obtained. Output information including at least one of the acquired images, A program for causing a display device to display a message prompting the user to take images in a situation where the background does not change, based on the background of the image of the identification document at the first angle and the background of the image of the identification document at the third angle.

5. A detection means that, in response to obtaining an image of the identification document at a first angle, obtains an image of the identification document at a second angle greater than the first angle, and then obtains an image of the identification document at a third angle greater than the first angle and smaller than the second angle, The system includes an image output means that outputs output information including at least one of the acquired images, and displays a message on a display device prompting the user to take images in a situation where the background does not change, based on the background of the image of the identification document at the first angle and the background of the image of the identification document at the third angle. Information processing device.

Images