Image processing system

Abstract

To provide a technique that improves the convenience of a system that uses an image of a personal identification document to perform personal identification.SOLUTION: An image processing system (1) includes an image analysis unit (110), an image acquisition unit (120), and a guide information output unit (130). The image analysis unit (110) analyzes an image in a video captured by an image capturing apparatus, and detects a pose of a personal identification document in the image. The image acquisition unit (120) acquires, as an image to be processed, an image in the video in which a pose associated to any of a plurality of reference poses preset for the personal identification document is detected. The guide information output unit (130) outputs, onto the video, guide information representing a reference pose of the personal identification document while changing a type and a display position thereof, depending on a detection result of a pose of the personal identification document.SELECTED DRAWING: Figure 1

Description

[Technical field]

[0001] The present invention relates to a technique for identity verification using an image of an identity document. [Background technology]

[0002] For example, when opening a bank account or applying for a credit card, identity verification is performed using identification documents such as a driver's license. In recent years, so-called eKYC (electronic Know Your Customer) services have been provided that use a camera to take an image of an identification document and verify the identity of the person online.

[0003] In cases where identity verification is performed using an image of an identity document, there is a risk that a malicious person may use a counterfeit identity document. Therefore, a mechanism is required to verify that a genuine identity document is being used. For example, the following Patent Document 1 discloses a technology for strict online identity verification.

[0004] Patent Document 1 discloses a technique for outputting a guide screen for specifying the position of a driver's license and the position of a coin to be photographed together with the driver's license, and acquiring an image for identity verification in which the driver's license and coins are arranged according to the guide screen. In the technique of Patent Document 1, the position of the coin on the guide screen is randomly determined. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] JP 2020-161191 A Summary of the Invention [Problem to be solved by the invention]

[0006] In the technology of Patent Document 1, an identification document is photographed in various positions (front, side, back, etc.) and the authenticity of the identification document is determined using the images. However, after photographing an identification document in one position, the user must operate the terminal each time to photograph the same identification document in a different position.

[0007] The present invention has been made in view of the above-mentioned problems, and an object of the present invention is to provide a technique for improving the convenience of a system for performing identity verification using an image of an identity verification document. [Means for solving the problem]

[0008] A first image processing system according to the present disclosure includes: an image analysis means for analyzing an image in a video captured by an imaging device and detecting a posture of the identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document is detected; a guide information output means for outputting guide information representing a reference orientation of the identification document onto the image while changing the type and display position of the guide information according to a detection result of the orientation of the identification document; Equipped with.

[0009] A second image processing system according to the present disclosure includes: an image analysis means for analyzing an image in a video captured by an imaging device and detecting a posture of the identification document in the image; an image acquisition means for acquiring an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document is detected as an image to be processed; a guide information output means for outputting guide information representing a reference orientation of the identification document onto the image while changing the type of the guide information according to a detection result of the orientation of the identification document; Equipped with.

[0010] The first image processing method of the present disclosure is executed by a computer. The first image processing method includes: Analyzing an image in a video captured by an imaging device to detect a posture of the identification document in the image; acquiring an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document is detected as an image to be processed; outputting guide information representing a reference orientation of the identification document onto the image while changing the type and display position according to a detection result of the orientation of the identification document; This includes:

[0011] The second image processing method of the present disclosure is executed by a computer. The second image processing method includes: Analyzing an image in a video captured by an imaging device to detect a posture of the identification document in the image; acquiring an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document is detected as an image to be processed; outputting guide information representing a reference orientation of the identification document onto the image while changing the type of guide information according to a detection result of the orientation of the identification document; This includes:

[0012] The first program in the present disclosure is Computer, an image analysis means for analyzing an image in a video captured by an imaging device and detecting a posture of the identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document has been detected; a guide information output means for outputting guide information representing a reference orientation of the identification document onto the image while changing the type and display position of the guide information according to a detection result of the orientation of the identification document; Function as.

[0013] The second program in the present disclosure is Computer, an image analysis means for analyzing an image in a video captured by an imaging device and detecting a posture of the identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document has been detected; a guide information output means for outputting guide information representing a reference orientation of the identification document onto the image while changing the type of the guide information depending on a detection result of the orientation of the identification document; Function as. Effect of the Invention

[0014] According to the present invention, it is possible to improve the convenience of a system for performing identity verification using an image of an identity verification document. [Brief description of the drawings]

[0015] [Figure 1] 1 is a diagram illustrating an example of a system configuration of an image processing system according to a first embodiment. [Diagram 2] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an image processing system. [Diagram 3] 4 is a flowchart illustrating the flow of processing executed by the image processing system of the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a screen including guide information displayed by a guide information output unit in the first embodiment. [Diagram 5] FIG. 4 is a diagram showing an example of a screen including guide information displayed by a guide information output unit in the first embodiment. [Figure 6] FIG. 4 is a diagram showing an example of a screen including guide information displayed by a guide information output unit in the first embodiment. [Figure 7] 13A and 13B are diagrams illustrating examples of the relationship between the display positions of guide information before and after a change. [Figure 8]10 is a flowchart illustrating the flow of processing executed by an image processing system according to a second embodiment. [Figure 9] FIG. 13 is a diagram showing an example of a screen including guide information displayed by a guide information output unit according to the second embodiment. [Figure 10] FIG. 13 is a diagram showing an example of a screen including guide information displayed by a guide information output unit according to the second embodiment. [Figure 11] FIG. 13 is a diagram showing an example of a screen including guide information displayed by a guide information output unit according to the second embodiment. DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENTS

[0016] Hereinafter, an embodiment of the present invention will be described with reference to the drawings. In all drawings, similar components are given similar reference numerals, and descriptions thereof will be omitted as appropriate. In addition, unless otherwise specified, in each block diagram, each block represents a functional configuration, not a hardware configuration. In addition, the direction of the arrows in the drawings is merely for making the flow of information easier to understand, and does not limit the direction of communication (one-way communication / two-way communication) unless otherwise specified.

[0017] [First embodiment] FIG. 1 is a diagram illustrating an example of a system configuration of an image processing system according to the first embodiment. The imaging device 20 captures an image of the identification document at an arbitrary frame rate and supplies the image to the image processing system 1. The image captured by the imaging device 20 is displayed on the display 30. A user who presents an identification document to the imaging device 20 moves the identification document while checking the current position of the identification document by looking at the image displayed on the display 30. When the user performs such an action, the image processing system 1 can obtain an image of the identification document that is required for subsequent identity verification. The image processing system 1 transmits the image of the identification document obtained in the process described below to a server 40 that executes identity verification processing.

[0018] <Example of functional configuration of image processing system 1> The image processing system 1 shown in FIG. 1 includes an image analysis unit 110, an image acquisition unit 120, and a guide information output unit .

[0019] The image analysis unit 110 acquires an image of the identification document captured by the imaging device 20. Then, the image analysis unit 110 analyzes the images that compose the acquired video, and detects the identification document from the images. At this time, the image analysis unit 110 also detects the orientation of the identification document. Here, the "orientation" of the identification document means how the identification document appears in the image (for example, the orientation of the identification document, such as the front / side / back, or the inclination angle of the identification document).

[0020] The image analysis unit 110 can detect an image area corresponding to the identification document based on, for example, edge features extracted from an image. The image analysis unit 110 can also estimate the orientation of the identification document based on features obtained from the detected image area. For example, the image analysis unit 110 can obtain text information from the image area using OCR (Optical Character Recognition) technology or the like, and estimate the orientation (front / side / rear) of the identification document based on a specific keyword detected from the text information. The image analysis unit 110 can also estimate the tilt angle of the identification document based on tilt information of the text information obtained by analyzing the image. The image analysis unit 110 may also be configured to detect the orientation of the identification document based on the analysis result of the image area of ​​the identification document (for example, the detection state of the face photo, the tilt angle of the face photo, etc.). In addition, the image analysis unit 110 may also be configured to determine whether or not the identification document is present in the image using a learning model constructed to be able to detect any type of identification document and the orientation of the identification document by machine learning. Alternatively, the image analysis unit 110 may be configured to detect the identification document and the pose of the identification document from the target image by performing a matching process with images of the identification document in various poses that have been registered in advance.

[0021] The image acquisition unit 120 acquires an image to be used in the identity verification process (hereinafter, also referred to as a "processing target image") based on the orientation of the identity verification document detected by the image analysis unit 110. For example, the image acquisition unit 120 compares the orientation of the identity verification document detected by the image analysis unit 110 with a plurality of reference orientations previously set for the identity verification document. Here, if the orientation of the identity verification document detected by the image analysis unit 110 corresponds to any one of the plurality of reference orientations, the image acquisition unit 120 acquires the image in which the orientation is detected as the processing target image. For example, the image acquisition unit 120 reads the data of the plurality of reference orientations from a storage unit (not shown) that stores the data, and performs a process of determining the similarity between the orientation of the identity verification document detected by the image analysis unit 110 and each of the plurality of reference orientations. As a result, if a similarity equal to or greater than a predetermined threshold value is obtained with respect to any one of the plurality of reference orientations, the image acquisition unit 120 acquires the image in which the orientation showing such a similarity is detected as the processing target image.

[0022] In order to allow the user to understand the reference orientation of the identification document, the guide information output unit 130 outputs guide information indicating the reference orientation of the identification document onto the image displayed on the display 30. In this embodiment, the guide information output unit 130 changes the type and display position of the guide information to be output onto the image according to the detection result of the orientation of the identification document by the image analysis unit 110.

[0023] For example, it is assumed that the reference posture of the identification document is preset as three postures. In this case, the guide information output unit 130 first outputs guide information (first guide information) corresponding to any one of the three reference postures onto the image displayed on the display 30. At this time, the guide information output unit 130 outputs the first guide information to a position (first position) determined randomly or according to a predetermined rule. Thereafter, it is assumed that the user moves the identification document while checking the display related to the first guide information, and the posture corresponding to the reference posture specified by the first guide information is detected by the image analysis unit 110 at a position corresponding to the first position. In response to this detection result, the guide information output unit 130 outputs one of the remaining two pieces of guide information (second guide information) onto the image displayed on the display 30 in place of the first guide information. At this time, the guide information output unit 130 outputs the second guide information to a position (second position) different from the first position where the first guide information was output. Thereafter, it is assumed that the user further moves the identification document while checking the display related to the second guide information, and thus a posture corresponding to the reference posture specified by the second guide information is detected by the image analysis unit 110 at a position corresponding to the second position. In response to this detection result, the guide information output unit 130 outputs the last guide information (third guide information) that has not yet been displayed, instead of the second guide information, onto the image displayed on the display 30. At this time, the guide information output unit 130 outputs the third guide information to a third position that is different from at least the second position where the second guide information was output. Thereafter, it is assumed that the user further moves the identification document while checking the display related to the third guide information, and thus a posture corresponding to the reference posture specified by the third guide information is detected by the image analysis unit 110 at a position corresponding to the third position. In response to this detection result, the guide information output unit 130 can recognize that all of the three preset reference postures have been detected (i.e., all of the necessary processing target images have been acquired by the image acquisition unit 120).In this case, the guide information output unit 130 can display, for example, on the display of the user terminal 10, a message indicating that acquisition of an image required for identity verification has been completed.

[0024] Note that the operation of the guide information output unit 130 is not limited to the specific example described above. For example, the guide information output unit 130 may be configured to switch the type and display position of the guide information in response to receiving a notification indicating that the processing target image has been acquired from the image acquisition unit 120.

[0025] When all processing target images necessary for identity verification are acquired, the image processing system 1 transmits those processing target images to the server 40 which performs identity verification processing. The server 40 executes identity verification processing using the processing target images received from the image processing system 1. If the user is authenticated as the original user by the processing of the server 40, the user can enjoy services such as opening an account and creating a credit card.

[0026] <Hardware configuration example> Each functional component of the image processing system 1 may be realized by hardware that realizes each functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a further explanation will be given of the case where each functional component of the image processing system 1 is realized by a combination of hardware and software.

[0027] 2 is a block diagram illustrating an example of a hardware configuration of the image processing system 1. In this diagram, the image processing system 1 is implemented on a user terminal 10. For example, the environment of the image processing system 1 is constructed by installing a dedicated application on the user terminal 10.

[0028] The user terminal 10 includes a bus 1010, a processor 1020, a memory 1030, a storage device 1040, an input / output interface 1050, and a network interface 1060.

[0029] The bus 1010 is a data transmission path for transmitting and receiving data between the hardware components. However, the method for connecting the hardware components of the user terminal 10 is not limited to a bus connection.

[0030] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0031] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0032] The storage device 1040 is an auxiliary storage device realized by a hard disk drive (HDD), a solid state drive (SSD), a memory card, a read only memory (ROM), or the like. The storage device 1040 stores program modules that realize each function of the image processing system 1 (such as the image analysis unit 110, the image acquisition unit 120, and the guide information output unit 130). The processor 1020 loads and executes each of these program modules onto the memory 1030, thereby realizing each function corresponding to each program module on the user terminal 10.

[0033] The input / output interface 1050 is an interface for connecting the user terminal 10 to various input / output devices. In the example of Fig. 2, the imaging device 20 and the display 30 of Fig. 1 are connected to the input / output interface 1050. In other words, in the example of this figure, the imaging device 20 and the display 30 are mounted on the user terminal 10. Note that the display 30 connected to the input / output interface 1050 may be a touch panel display equipped with a touch panel. Other input / output devices such as a keyboard, a mouse, and a speaker may be connected to the input / output interface 1050.

[0034] The network interface 1060 is an interface for connecting the user terminal 10 to a network. The network interface 1060 connects the user terminal 10 to the network by wire or wirelessly. This network is, for example, a LAN (Local Area Network) or a WAN (Wide Area Network). The user terminal 10 can communicate with other devices on the network via the network interface 1060 to transmit and receive various data. For example, the user terminal 10 can communicate with the server 40 via the network interface 1060 and transmit an image to be processed to the server 40. In addition, when the imaging device 20 exists as a device separate from the user terminal 10 and is connected to the network, the user terminal 10 can communicate with the imaging device 20 via the network interface 1060 to obtain an image of an identification document from the imaging device 20.

[0035] Note that the configuration in Fig. 2 is merely an example, and the present invention is not limited to the contents shown in Fig. 2. For example, some or all of the functions of the image processing system 1 may be provided in a device other than the user terminal 10. For example, the server 40 may have an image analysis unit 110, an image acquisition unit 120, and a guide information output unit 130. In this case, the user terminal 10 transmits an image captured by the imaging device 20 to the server 40, and the server 40 executes the above-mentioned processes using the image acquired from the user terminal 10.

[0036] <Processing flow> 3 is a flowchart illustrating the flow of processing executed by the image processing system 1 of the first embodiment. This diagram illustrates an example of a flowchart in the case where the image processing system 1 is implemented on the user terminal 10.

[0037] First, the user operates the user terminal 10 to start the application of the image processing system 1 installed in the user terminal 10 (S102). In response to the start of the application, the user terminal 10 communicates with the imaging device 20 connected to the user terminal 10 and starts acquiring video (S104).

[0038] In response to the start of the application, the guide information output unit 130 determines the type and display position of the guide information to be output on the image acquired from the imaging device 20 (S106). For example, the guide information output unit 130 refers to data of the guide information pre-stored in a storage area such as the storage device 1040, and determines the type of guide information to be output first. If the type of guide information to be output first is predetermined, the guide information output unit 130 reads out data of the guide information pre-defined as the guide information to be output first. For example, when rotating an identification document and photographing various postures of the identification document with the imaging device 20, the guide information related to the front side may be set as the guide information to be output first. In addition, the guide information output unit 130 determines the display position of the read guide information on the display 30, for example, randomly or based on a predetermined rule within the range in which the image of the imaging device 20 is displayed.

[0039] Here, depending on the performance of the imaging device 20 and the display 30, there is a possibility that distortion or blurring occurs in an area close to the boundary part of the angle of view of the imaging device 20 in the image. If the guide information is displayed in such an area, there is a possibility that the detection accuracy of the identification document and the posture of the identification document will decrease. Therefore, the guide information output unit 130 controls the display position of the guide information to avoid such an area. Specifically, the guide information output unit 130 controls the display position of the guide information in a range narrower than the angle of view of the imaging device 20. Information on the angle of view of the imaging device 20 is added to the image of the imaging device 20 as metadata, for example. In addition, the guide information output unit 130 may acquire model information stored on the user terminal 10 and acquire specification information (for example, information on the angle of view) of the imaging device 20 mounted on the user terminal 10 based on the model information.

[0040] The guide information output unit 130 outputs the type of guide information determined in the process of S106 to the display position also determined in the process of S106 (S108). The user moves the identification document in accordance with the guide information displayed on the display 30 so that the identification document moves to the specified position, and changes the orientation of the identification document so that it assumes the specified posture.

[0041] As described above, even while the user is moving the identification document, the video captured by the imaging device 20 is supplied to the image processing system 1. The image analysis unit 110 sequentially analyzes the frames (images) of the video supplied to the image processing system 1 (S110). The image analysis unit 110 may select all frames (images) of the supplied video as frames to be analyzed, or may select frames (images) to be analyzed at regular time intervals. Then, based on the image analysis result, the image analysis unit 110 determines whether or not a posture corresponding to a reference posture specified by the guide information output on the display 30 has been detected at a position corresponding to the display position of the guide information (S112).

[0042] When a posture corresponding to the reference posture specified by the guide information is not detected at a position corresponding to the display position of the guide information (S112: NO), the image analysis unit 110 continues to analyze the frames (images) of the video supplied to the image processing system 1. On the other hand, when a posture corresponding to the reference posture specified by the guide information is detected at a position corresponding to the display position of the guide information (S112: YES), the guide information output unit 130 further determines whether or not all of a plurality of preset reference postures have been detected (S114).

[0043] If all of the plurality of preset reference postures have not been detected (S114: NO), all images necessary for identity verification have not been acquired, and the process transitions to S106. Then, the guide information output unit 130 changes the type of guide information and the display position of the guide information. For example, the guide information output unit 130 displays another type of guide information newly determined in the process of S106 at a certain distance (for example, a fixed value of 20 to 30 pixels) from the display position of the current guide information. Then, the processes from S110 to S114 are repeatedly executed. On the other hand, if all of the plurality of preset reference postures have been detected (S114: YES), all images necessary for identity verification have been acquired, and the process transitions to S116. In the process of S116, the image processing system 1 transmits all acquired images to be processed to the server 40 that executes identity verification processing (S116).

[0044] As described above, according to the image processing system 1 of the present embodiment, guide information showing the reference posture of the identification document is output on the image that the user confirms in order to obtain an image required for identity verification. The user who presents the identification document can easily capture an image required for identity verification by moving the identification document according to this guide information. In addition, in the image processing system 1 of the present embodiment, the type of guide information output on the image that the user confirms is automatically switched in response to detection of a posture corresponding to the reference posture of the identification document. This allows the user to continuously capture images required for identity verification without performing any further operation on the user terminal 10 to continue capturing images of the identification document. In other words, the convenience of the system is improved. In addition, in the image processing system 1 of the present embodiment, when switching the guide information, the guide information after switching is output at a position different from the position where the guide information before switching was output. In this way, the image required for identity verification cannot be acquired unless the display position of the guide information is confirmed in real time and the display is followed. This makes it possible to prevent, for example, the image of the identification document from being captured using a malicious program. For example, even if a malicious program that uses a pre-recorded image is used against this system, unless the identification document shown in the image moves accurately in accordance with the display position of the guide information, an image for identification will not be acquired and the identification process will not be executed. In this respect, the image processing system 1 of this embodiment can improve the security of the identification process.

[0045] <Guide information display location> When changing the display position of the guide information, if the difference (amount of movement) between the display position of the guide information before and after the change is too small, the effect on the above-mentioned security is reduced. Therefore, it is preferable that the guide information output unit 130 is configured to determine the display position of the changed guide information so that the difference (amount of movement) between the display position of the guide information before and after the change is equal to or greater than a first threshold value.

[0046] On the other hand, when changing the display position of the guide information, if the difference (movement amount) between the display position of the guide information before and after the change is too large, the user must move the identification document by a large amount, and the above-mentioned effect on convenience is reduced. Therefore, it is preferable to set an upper limit on the movement amount of the display position of the guide information. For example, the guide information output unit 130 may be configured to determine the display position of the changed guide information so that the movement amount before and after the change of the guide information falls within a range from the above-mentioned first threshold value to a second threshold value larger than the first threshold value.

[0047] <Example of guide information screen display> 4 to 6 are diagrams showing an example of a screen including guide information displayed by the guide information output unit 130 of the first embodiment. In Fig. 4 to Fig. 6, screens sequentially displayed on the display 30 over time are shown as examples.

[0048] Specifically, the guide information output unit 130 first displays a screen S1, as exemplified in FIG. 4, on the display 30. The screen S1, as exemplified in FIG. 4, has guide information G1 near the center of the display area of ​​the display 30. The guide information G1 is guide information for acquiring an image of the front side of an identification document (here, a driver's license). In the example of FIG. 4, the guide information output unit 130 adjusts the area other than the guide information G1 (the hatched area) so that it is displayed darker than the area of ​​the guide information G1. This makes it possible to make the guide information G1 stand out. When the user presents the identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G1 and at a position corresponding to the display position of the guide information G1, a first processing target image in which the front side of the identification document (driver's license) is captured is acquired.

[0049] When the first processing target image is acquired on the screen S1 (when a posture corresponding to the reference posture specified by the guide information G1 is detected), the guide information output unit 130 displays the screen S2 illustrated in FIG. 5 on the display 30. The screen S2 illustrated in FIG. 5 has the guide information G2 near the bottom of the display area of ​​the display 30. The guide information G2 is guide information for acquiring an image of the side of an identification document (here, a driver's license). In the example of FIG. 5, the guide information output unit 130 changes the display position of the guide information along the height direction (downward direction) of the display 30 (display area that displays an image). Also, in the example of FIG. 5, the guide information output unit 130 adjusts the area excluding the guide information G2 (hatched area) so that it is displayed darker than the area of ​​the guide information G2, as in FIG. 4. This makes the guide information G2 stand out. When the user presents an identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G2 and at a position corresponding to the display position of the guide information G2, a second processing target image showing the side of the identification document (driver's license) is obtained.

[0050] When the second processing target image is acquired on the screen S2 (the posture corresponding to the reference posture specified by the guide information G2 is detected), the guide information output unit 130 displays the screen S3 exemplified in FIG. 6 on the display 30. The screen S3 exemplified in FIG. 6 has the guide information G3 near the center of the display area of ​​the display 30. The guide information G3 is guide information for acquiring an image of the back side of an identification document (here, a driver's license). In the example of FIG. 6, the guide information output unit 130 changes the display position of the guide information along the height direction (upward direction) of the display 30 (display area that displays an image). Also, in the example of FIG. 6, the guide information output unit 130 adjusts the area excluding the guide information G3 (the hatched area) so that it is displayed darker than the area of ​​the guide information G3, as in FIG. 4 and FIG. 5. This makes the guide information G3 stand out. When the user presents his / her identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G3, and at a position corresponding to the display position of the guide information G3, a third image to be processed, showing the back side of the identification document (driver's license), is obtained.

[0051] As described above, if the amount of movement of the guide information is too small, security may be reduced, and if the amount of movement of the guide information is too large, the convenience of the system may be reduced. Therefore, the guide information output unit 130 may display the changed guide information at a position where it at least partially overlaps with the display position of the guide information before the change. Specifically, as shown in FIG. 7, for example, the guide information output unit 130 may adjust the display position of the changed guide information so that the display position of the guide information before the change and the display position of the guide information after the change overlap by about 1 / 4 to 1 / 3 based on the display size. FIG. 7 is a diagram illustrating the relationship between the display positions of the guide information before and after the change. In the example of FIG. 7, the display position of the guide information before the change is indicated by a dotted line. Also, in the example of FIG. 7, the display position of the guide information after the change is indicated by a dashed line. Also, in the example of FIG. 7, the overlap of the display positions of the guide information before and after the change is indicated by a diagonal line. The guide information output unit 130 controls the size of this hatched area so that it is approximately 1 / 4 to 1 / 3 of the size (height, width, area) of the guide information to be displayed.

[0052] <Modification> The guide information output unit 130 may be configured to change the guide information along the width direction of the display 30 (display area displaying an image). In this case, the guide information output unit 130 recognizes the orientation of the user terminal 10 (the orientation of the display 30) based on information obtained from an inertial measurement device such as a gyro sensor mounted on the user terminal 10. Then, based on the orientation of the user terminal 10 (the orientation of the display 30), the guide information output unit 130 controls the direction in which the display position of the guide information is changed. Furthermore, the guide information output unit 130 may control the display position of the guide information in the height direction and the width direction regardless of the orientation of the user terminal 10 (the orientation of the display 30).

[0053] Although not shown, the image processing system 1 may further include a function of outputting any information on the display 30 to inform the user that the image to be processed has been acquired. For example, the image processing system 1 may include a function of outputting a progress bar or a predetermined message when acquiring the image to be processed. In this way, the user can visually recognize that the image required for identity verification has been acquired.

[0054] [Second embodiment] The image processing system 1 of this embodiment is similar to the image processing system 1 of the first embodiment, except for the points described below.

[0055] <Functional configuration of image processing system 1> The image processing system 1 of this embodiment has an image analysis unit 110, an image acquisition unit 120, and a guide information output unit 130, similar to the configuration example (FIG. 1) of the image processing system 1 of the first embodiment. The image analysis unit 110 and the image acquisition unit 120 of this embodiment are similar to those of the first embodiment. The guide information output unit 130 of this embodiment differs from the guide information output unit 130 of the first embodiment in that it does not change the display position of the guide information. Specifically, the guide information output unit 130 of this embodiment changes the type of guide information to be output on the video according to the detection result of the posture of the identification document by the image analysis unit 110.

[0056] <Hardware configuration example> The hardware configuration of the image processing system 1 of this embodiment is similar to the hardware configuration example (FIG. 2) of the image processing system 1 of the first embodiment. The storage device 1040 stores program modules that realize each function (image analysis unit 110, image acquisition unit 120, guide information output unit 130, etc.) of the image processing system 1 of this embodiment. The processor 1020 loads and executes these program modules on the memory 1030, thereby realizing each function corresponding to each program module on the user terminal 10.

[0057] <Processing flow> 8 is a flowchart illustrating the flow of processing executed by the image processing system 1 of the second embodiment. This diagram illustrates an example of a flowchart in the case where the image processing system 1 is implemented on the user terminal 10.

[0058] First, the user operates the user terminal 10 to start the application of the image processing system 1 installed in the user terminal 10 (S202). In response to the start of the application, the user terminal 10 communicates with the imaging device 20 connected to the user terminal 10 and starts acquiring video (S204). These processes are similar to the processes of S102 and S104 in FIG. 3.

[0059] In response to the start of the application, the guide information output unit 130 determines the type of guide information to be output on the video acquired from the imaging device 20 (S206). For example, the guide information output unit 130 refers to data of guide information stored in advance in a storage area such as the storage device 1040, and determines the type of guide information to be output first. When the type of guide information to be output first is predetermined, the guide information output unit 130 reads out data of the guide information that is predetermined to be output first. For example, when the identification document is rotated and various postures of the identification document are photographed by the imaging device 20, the guide information related to the front side may be set as the guide information to be output first. In this embodiment, the display position of the guide information may be predetermined. For example, the guide information output unit 130 may output the guide information so that the center of the area of ​​the guide information overlaps with the center of the display 30 (display area).

[0060] The guide information output unit 130 outputs the type of guide information determined in the process of S206 to the display 30 (S208). The user moves the identification document in accordance with the guide information displayed on the display 30 so that the identification document moves to a specified position, and changes the orientation of the identification document so that the identification document assumes a specified posture.

[0061] As described above, while the user is moving the identification document, the image captured by the imaging device 20 is supplied to the image processing system 1. The image analysis unit 110 sequentially analyzes frames (images) of the image supplied to the image processing system 1 (S210). Then, based on the image analysis result, the image analysis unit 110 determines whether or not a posture corresponding to a reference posture specified by the guide information output on the display 30 has been detected at a position corresponding to the display position of the guide information (S212). These processes are similar to the processes of S110 and S112 in FIG. 3.

[0062] If the posture corresponding to the reference posture specified by the guide information is not detected at the position corresponding to the display position of the guide information (S212: NO), the image analysis unit 110 continues to analyze the frames (images) of the video supplied to the image processing system 1. On the other hand, if the posture corresponding to the reference posture specified by the guide information is detected at the position corresponding to the display position of the guide information (S212: YES), the guide information output unit 130 further determines whether or not all of a plurality of reference postures set in advance have been detected (S214). These processes are similar to the processes of S112 and S114 in FIG. 3.

[0063] If all of the plurality of preset reference postures have not been detected (S214: NO), the process proceeds to S206 since all images necessary for identity verification have not been acquired. Then, the guide information output unit 130 changes the type of guide information. For example, the guide information output unit 130 displays another type of guide information newly determined in the process of S206 at the display position of the current guide information. Then, the processes from S210 to S214 are repeatedly executed. On the other hand, if all of the plurality of preset reference postures have been detected (S214: YES), the process proceeds to S216 since all images necessary for identity verification have been acquired. In the process of S216, the image processing system 1 transmits all acquired images to be processed to the server 40 that executes identity verification processing (S216). The process of S216 is the same as the process of S116 in FIG. 3.

[0064] As described above, according to the image processing system 1 of the present embodiment, guide information showing the reference posture of the identification document is output on the image that the user confirms in order to obtain an image required for identity verification. The user who presents the identification document can easily capture the image required for identity verification by moving the identification document according to this guide information. In addition, in the image processing system 1 of the present embodiment, the type of guide information output on the image that the user confirms is automatically switched in response to detection of a posture corresponding to the reference posture of the identification document. This allows the user to continuously capture images required for identity verification without performing further operations on the user terminal 10 to continue capturing images of the identification document. In other words, the convenience of the system is improved. In addition, in the image processing system 1 of the present embodiment, when switching the guide information, the display position of the guide information is not changed before and after switching, unlike the image processing system 1 of the first embodiment. In this case, the user does not need to move the identification document, and the convenience of the system for the user is improved.

[0065] <Example of guide information screen display> 9 to 11 are diagrams showing an example of a screen including guide information displayed by the guide information output unit 130 of the second embodiment. In Fig. 9 to Fig. 11, screens sequentially displayed on the display 30 over time are shown.

[0066] Specifically, the guide information output unit 130 first displays a screen S4 exemplified in FIG. 9 on the display 30. The screen S4 exemplified in FIG. 9 has guide information G4 near the center of the display area of ​​the display 30. The guide information G4 is guide information for acquiring an image of the front side of an identification document (here, a driver's license). In the example of FIG. 9, the guide information output unit 130 adjusts the area (hatched area) excluding the guide information G4 so that it is displayed darker than the area of ​​the guide information G4. This makes it possible to make the guide information G4 stand out. When the user presents the identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G4 and at a position corresponding to the display position of the guide information G4, a first processing target image in which the front side of the identification document (driver's license) is captured is acquired.

[0067] When the first processing target image is acquired on the screen S4 (when a posture corresponding to the reference posture specified by the guide information G4 is detected), the guide information output unit 130 displays the screen S5 illustrated in FIG. 10 on the display 30. The screen S5 illustrated in FIG. 10 has the guide information G5 at the same position as the guide information G4 on the screen S4 in FIG. 9. The guide information G5 is guide information for acquiring an image of the side of an identification document (here, a driver's license). In the example of FIG. 10, the guide information output unit 130 adjusts the area (hatched area) other than the guide information G5 to be displayed darker than the area of ​​the guide information G5, as in FIG. 9. This makes it possible to make the guide information G5 stand out. When the user presents the identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G5 at a position corresponding to the display position of the guide information G5, a second processing target image showing the side of the identification document (driver's license) is acquired.

[0068] When the second processing target image is acquired on the screen S5 (when a posture corresponding to the reference posture specified by the guide information G5 is detected), the guide information output unit 130 displays the screen S6 illustrated in FIG. 11 on the display 30. The screen S6 illustrated in FIG. 11 has the guide information G6 at the same position as the guide information G4 on the screen S4 in FIG. 9 and the guide information G5 on the screen S5 in FIG. 10. The guide information G6 is guide information for acquiring an image of the back side of an identification document (here, a driver's license). In the example of FIG. 11, the guide information output unit 130 adjusts the area (hatched area) excluding the guide information G6 so as to be displayed darker than the area of ​​the guide information G6, as in the case of FIG. 9 and FIG. 10. This makes it possible to make the guide information G6 stand out. When the user presents the identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G6 at a position corresponding to the display position of the guide information G6, a third processing target image showing the back side of the identification document (driver's license) is acquired.

[0069] Although the embodiments of the present invention have been described above with reference to the drawings, the present invention should not be interpreted as being limited thereto, and various modifications, improvements, etc. can be made based on the knowledge of a person skilled in the art without departing from the gist of the present invention. In addition, the multiple components disclosed in the embodiments can be appropriately combined to form various inventions. For example, some components may be deleted from all the components shown in the embodiments, or components of different embodiments may be appropriately combined.

[0070] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the figures can be changed to the extent that the content is not affected. In addition, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0071] A part or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. an image analysis means for analyzing an image in a video captured by an imaging device and detecting a posture of the identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document is detected; a guide information output means for outputting guide information representing a reference orientation of the identification document onto the image while changing the type and display position of the guide information according to a detection result of the orientation of the identification document; An image processing system comprising: 2. The guide information output means changing a type and a display position of the guide information in response to a posture corresponding to a reference posture specified by the guide information being detected as a posture of the identification document; 1. The image processing system according to claim 1. 3. The guide information output means A movement amount before and after the change of the display position of the guide information is set to a predetermined first threshold value or more. 1. An image processing system according to claim 1 or 2. 4. The guide information output means a movement amount before and after the change of the display position of the guide information is set to be equal to or less than a second threshold value that is larger than the first threshold value; 3. The image processing system according to claim 1. 5. The guide information output means includes: controlling a display position of the guide information within a range narrower than an angle of view of the imaging device; 5. An image processing system according to any one of 1 to 4. 6. The guide information output means includes: changing a display position of the guide information along a height direction of a display area in which the video is displayed; 6. An image processing system according to any one of 1 to 5. 7. The guide information output means includes: changing a display position of the guide information along a width direction of a display area in which the video is displayed; 7. An image processing system according to any one of 1 to 6. 8. an image analysis means for analyzing an image in a video captured by an imaging device and detecting a posture of the identification document in the image; an image acquisition means for acquiring an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document is detected as an image to be processed; a guide information output means for outputting guide information representing a reference orientation of the identification document onto the image while changing the type of the guide information according to a detection result of the orientation of the identification document; An image processing system comprising: 9. 1. A computer-implemented image processing method, comprising the steps of: Analyzing an image in a video captured by an imaging device to detect a posture of the identification document in the image; acquiring an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document is detected as an image to be processed; outputting guide information representing a reference orientation of the identification document onto the image while changing the type and display position according to a detection result of the orientation of the identification document; An image processing method comprising: 10. The computer, changing a type and a display position of the guide information in response to a posture corresponding to a reference posture specified by the guide information being detected as a posture of the identification document; 9. The image processing method according to claim 9, 11. The computer, A movement amount before and after the change of the display position of the guide information is set to a predetermined first threshold value or more. The image processing method according to claim 9 or 10, further comprising: 12. The computer, a movement amount before and after the change of the display position of the guide information is set to be equal to or less than a second threshold value that is larger than the first threshold value; 12. The image processing method according to claim 11, further comprising: 13. The computer, controlling a display position of the guide information within a range narrower than an angle of view of the imaging device; 13. The image processing method according to any one of 9. to 12., 14. The computer, changing a display position of the guide information along a height direction of a display area in which the video is displayed; 14. The image processing method according to any one of 9. to 13., 15. The computer, changing a display position of the guide information along a width direction of a display area in which the video is displayed; 15. The image processing method according to any one of 9. to 14., 16. 1. A computer-implemented image processing method, comprising the steps of: Analyzing an image in a video captured by an imaging device to detect a posture of the identification document in the image; acquiring an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document is detected as an image to be processed; outputting guide information representing a reference orientation of the identification document onto the image while changing the type of guide information according to a detection result of the orientation of the identification document; An image processing method comprising: 17. Computer, an image analysis means for analyzing an image in a video captured by an imaging device and detecting a posture of the identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document has been detected; a guide information output means for outputting guide information representing a reference orientation of the identification document onto the image while changing the type and display position of the guide information according to a detection result of the orientation of the identification document; A program to function as a 18. The guide information output means includes: changing a type and a display position of the guide information in response to a posture corresponding to a reference posture specified by the guide information being detected as a posture of the identification document; 17. The program described in. 19. The guide information output means includes: A movement amount before and after the change of the display position of the guide information is set to a predetermined first threshold value or more. 17. The program according to 18. 20. The guide information output means includes: a movement amount before and after the change of the display position of the guide information is set to be equal to or less than a second threshold value that is larger than the first threshold value; 19. The program described in. twenty one. The guide information output means includes: controlling a display position of the guide information within a range narrower than an angle of view of the imaging device; 20. The program according to any one of 17. to 20. twenty two. The guide information output means includes: changing a display position of the guide information along a height direction of a display area in which the video is displayed; 2. A program according to any one of 17. to 21. twenty three. The guide information output means includes: changing a display position of the guide information along a width direction of a display area in which the video is displayed; 2. A program according to any one of 17. to 22. twenty four. Computer, an image analysis means for analyzing an image in a video captured by an imaging device and detecting a posture of the identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to any one of a plurality of reference postures preset for the identification document has been detected; a guide information output means for outputting guide information representing a reference orientation of the identification document onto the image while changing the type of the guide information depending on a detection result of the orientation of the identification document; A program to function as a

[0072] This application claims priority based on Japanese Patent Application No. 2021-007923, filed on January 21, 2021, the disclosure of which is incorporated herein in its entirety. [Explanation of symbols]

[0073] 1. Image Processing System 10 User terminal 20 Imaging device 30 Display 40 Servers 110 Image Analysis Unit 120 Image acquisition unit 130 Guide information output section 1010 Bus 1020 Processor 1030 Memory 1040 Storage Device 1050 Input / Output Interface 1060 Network Interface G1, G2, G3, G4, G5, G6 Guide Information S1, S2, S3, S4, S5, S6 screen

Claims

[Claim 1] an image analysis means for analyzing an image constituting a video image captured by an imaging device and detecting a posture of the identification document in the image; an image acquisition means for acquiring an image in which a posture corresponding to any one of a plurality of reference postures preset for the identification document is detected in an image displayed on a display as an image to be processed; a guide information output means for outputting guide information representing a reference orientation of the identification document onto an image displayed on a display while changing the type and display position of the guide information according to a detection result of the orientation of the identification document; An image processing system comprising:

Images