Image processing system, image processing method and program

Abstract

An image processing system (1) comprises an image analysis unit (110), an image acquisition unit (120), and a guide information output unit (130). The image analysis unit (110) analyzes images in a picture captured by an imaging device to detect postures of an identification document in the images. The image acquisition unit (120) acquires, as an image to be processed, an image in the picture in which a posture corresponding to any one of a plurality of reference postures predetermined for the identification document has been detected. The guide information output unit (130) outputs guide information representing the reference posture of the identification document onto the picture while changing the type and the display position of the identification document depending on the detection result of the posture of the identification document.

Description

[Technical Field]

[0001] The present invention relates to a technique for verifying identity using an image of an identity document. [Background technology]

[0002] For example, when opening a bank account or applying for a credit card, identity verification is performed using identification documents such as a driver's license. In recent years, so-called eKYC (electronic Know Your Customer) services have been provided, which use a camera to take an image of the identification document and verify the identity online.

[0003] In cases where identity verification is performed using an image of an identity verification document, there is a risk that a malicious person may use a counterfeit identity verification document. Therefore, a mechanism is required to verify that the identity verification document being used is genuine. For example, Patent Document 1 listed below discloses a technology for strict online identity verification.

[0004] Patent Document 1 discloses a technology that outputs a guide screen that specifies the position of a driver's license and the position of a coin to be photographed together with the driver's license, and acquires an image for personal identification in which the driver's license and coin are arranged according to the guide screen. In the technology of Patent Document 1, the position of the coin on the guide screen is determined randomly. [Prior art documents] [Patent documents]

[0005] [Patent Document 1] Japanese Patent Publication No. 2020-161191 Summary of the Invention [Problem to be solved by the invention]

[0006] The technology of Patent Document 1 photographs an identification document in various positions (front, side, back, etc.) and uses the images to determine the authenticity of the identification document. However, after photographing an identification document in one position, the user must operate the terminal each time they photograph the identification document in a different position.

[0007] The present invention has been made in view of the above-mentioned problems, and one of the objects of the present invention is to provide a technique for improving the convenience of a system for verifying identity using an image of an identity verification document. [Means for solving the problem]

[0008] A first image processing system in the present disclosure includes: an image analysis means for analyzing an image in a video captured by an imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected; a guide information output means for outputting guide information representing a reference orientation of the personal identification document onto the image while changing the type and display position of the guide information according to the detection result of the orientation of the personal identification document; Equipped with.

[0009] A second image processing system in the present disclosure includes: an image analysis means for analyzing an image in a video captured by an imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected; a guide information output means for outputting guide information representing a reference orientation of the personal identification document onto the image while changing the type of guide information according to the detection result of the orientation of the personal identification document; Equipped with.

[0010] The first image processing method of the present disclosure is executed by a computer. The first image processing method includes: Analyzing an image in a video captured by an imaging device to detect the orientation of the identification document in the image; acquiring an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected as an image to be processed; outputting guide information representing a reference orientation of the identification document onto the video image while changing the type and display position according to the detection result of the orientation of the identification document; This includes:

[0011] The second image processing method of the present disclosure is executed by a computer. The second image processing method includes: Analyzing an image in a video captured by an imaging device to detect the orientation of the identification document in the image; acquiring an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected as an image to be processed; outputting guide information representing a reference orientation of the identification document onto the video while changing the type of guide information depending on the detection result of the orientation of the identification document; This includes:

[0012] The first program in the present disclosure is Computer, image analysis means for analyzing an image in a video captured by an imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document has been detected; a guide information output means for outputting guide information representing a reference orientation of the personal identification document onto the image while changing the type and display position of the guide information according to the detection result of the orientation of the personal identification document; Function as.

[0013] The second program in this disclosure is Computer, image analysis means for analyzing an image in a video captured by an imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected; a guide information output means for outputting guide information representing a reference orientation of the personal identification document onto the image while changing the type of guide information according to the detection result of the orientation of the personal identification document; Function as. [Effects of the Invention]

[0014] According to the present invention, it is possible to improve the convenience of a system for verifying identity using an image of an identity verification document. [Brief explanation of the drawings]

[0015] [Figure 1] FIG. 1 is a diagram illustrating an example of a system configuration of an image processing system according to a first embodiment. [Figure 2] FIG. 1 is a block diagram illustrating an example of a hardware configuration of an image processing system. [Figure 3] 4 is a flowchart illustrating the flow of processing executed by the image processing system of the first embodiment. [Figure 4] FIG. 4 is a diagram showing an example of a screen including guide information displayed by a guide information output unit according to the first embodiment. [Figure 5] FIG. 4 is a diagram showing an example of a screen including guide information displayed by a guide information output unit according to the first embodiment. [Figure 6] FIG. 4 is a diagram showing an example of a screen including guide information displayed by a guide information output unit according to the first embodiment. [Figure 7] 10A and 10B are diagrams illustrating examples of the relationship between the display positions of guide information before and after a change. [Figure 8]10 is a flowchart illustrating the flow of processing executed by an image processing system according to a second embodiment. [Figure 9] FIG. 11 is a diagram showing an example of a screen including guide information displayed by a guide information output unit according to the second embodiment. [Figure 10] FIG. 11 is a diagram showing an example of a screen including guide information displayed by a guide information output unit according to the second embodiment. [Figure 11] FIG. 11 is a diagram showing an example of a screen including guide information displayed by a guide information output unit according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0016] Hereinafter, embodiments of the present invention will be described with reference to the drawings. In all drawings, like components are denoted by like reference numerals, and descriptions thereof will be omitted where appropriate. Unless otherwise specified, each block in each block diagram represents a functional configuration, not a hardware configuration. The direction of the arrows in the drawings is merely intended to make the flow of information easier to understand, and does not limit the direction of communication (one-way communication / two-way communication) unless otherwise specified.

[0017] [First embodiment] FIG. 1 is a diagram illustrating an example of the system configuration of an image processing system according to the first embodiment. The imaging device 20 captures an image of the identification document at an arbitrary frame rate and supplies the image to the image processing system 1. The image captured by the imaging device 20 is displayed on the display 30. A user presenting an identification document to the imaging device 20 moves the identification document while checking the current position of the identification document by looking at the image displayed on the display 30. When the user performs such actions, the image processing system 1 can acquire an image of the identification document that will be required for subsequent identity verification. The image processing system 1 transmits the image of the identification document acquired in the process described below to a server 40 that executes identity verification processing.

[0018] <Example of functional configuration of image processing system 1> The image processing system 1 shown in FIG. 1 includes an image analysis unit 110, an image acquisition unit 120, and a guide information output unit .

[0019] The image analysis unit 110 acquires an image of the personal identification document captured by the imaging device 20. Then, the image analysis unit 110 analyzes the images that make up the acquired video and detects the identification document from the images. At this time, the image analysis unit 110 also detects the orientation of the identification document. Here, the "orientation" of the identification document means how the identification document appears in the image (for example, the orientation of the identification document, such as the front / side / back, or the tilt angle of the identification document).

[0020] The image analysis unit 110 can detect an image area corresponding to an identification document based on, for example, edge features extracted from an image. The image analysis unit 110 can also estimate the orientation of the identification document based on features obtained from the detected image area. For example, the image analysis unit 110 can acquire text information from the image area using OCR (Optical Character Recognition) technology and estimate the orientation (front / side / back) of the identification document based on specific keywords detected from the text information. The image analysis unit 110 can also estimate the tilt angle of the identification document based on tilt information of the text information obtained by analyzing the image. The image analysis unit 110 may also be configured to detect the orientation of the identification document based on the analysis results of the image area of ​​the identification document (e.g., the detection status of the facial photograph, the tilt angle of the facial photograph, etc.). Alternatively, the image analysis unit 110 may be configured to determine whether or not an identification document is present in an image using, for example, a learning model constructed by machine learning to be capable of detecting any type of identification document and the orientation of the identification document. Alternatively, the image analysis unit 110 may be configured to detect the identification document and the pose of the identification document from the target image by performing a matching process with images of the identification document in various poses that have been registered in advance.

[0021] The image acquisition unit 120 acquires an image to be used in the identity verification process (hereinafter also referred to as a "processing target image") based on the orientation of the identification document detected by the image analysis unit 110. For example, the image acquisition unit 120 compares the orientation of the identification document detected by the image analysis unit 110 with a plurality of reference orientations preset for the identification document. Here, if the orientation of the identification document detected by the image analysis unit 110 corresponds to one of the plurality of reference orientations, the image acquisition unit 120 acquires the image in which that orientation is detected as the processing target image. For example, the image acquisition unit 120 reads the data of the plurality of reference orientations from a storage unit (not shown) that stores the data, and performs a process to determine the similarity between the orientation of the identification document detected by the image analysis unit 110 and each of the plurality of reference orientations. As a result, if a similarity equal to or greater than a predetermined threshold is obtained with respect to one of the plurality of reference orientations, the image acquisition unit 120 acquires the image in which the orientation showing such similarity is detected as the processing target image.

[0022] In order to allow the user to understand the reference orientation of the identification document, the guide information output unit 130 outputs guide information indicating the reference orientation of the identification document onto the image displayed on the display 30. In this embodiment, the guide information output unit 130 changes the type and display position of the guide information to be output onto the image according to the detection result of the orientation of the identification document by the image analysis unit 110.

[0023] For example, the standard posture of identification documents asIt is assumed that three postures are set in advance. In this case, the guide information output unit 130 first outputs guide information (first guide information) corresponding to one of the three reference postures onto the image displayed on the display 30. At this time, the guide information output unit 130 outputs the first guide information to a position (first position) determined randomly or according to a predetermined rule. Thereafter, it is assumed that the user moves the identification document while checking the display related to the first guide information, and the posture corresponding to the reference posture specified by the first guide information is detected by the image analysis unit 110 at a position corresponding to the first position. In response to this detection result, the guide information output unit 130 outputs one of the remaining two pieces of guide information (second guide information) onto the image displayed on the display 30 in place of the first guide information. At this time, the guide information output unit 130 outputs the second guide information to a position (second position) different from the first position where the first guide information was output. Thereafter, it is assumed that the user further moves the identification document while checking the display related to the second guide information, and thereby a posture corresponding to the reference posture specified by the second guide information is detected by the image analysis unit 110 at a position corresponding to the second position. In response to this detection result, the guide information output unit 130 outputs the last guide information that has not yet been displayed (third guide information) on the image displayed on the display 30, in place of the second guide information. At this time, the guide information output unit 130 outputs the third guide information to a third position that is different from at least the second position where the second guide information was output. Thereafter, it is assumed that the user further moves the identification document while checking the display related to the third guide information, and thereby a posture corresponding to the reference posture specified by the third guide information is detected by the image analysis unit 110 at a position corresponding to the third position. In response to this detection result, the guide information output unit 130 can recognize that all of the three preset reference postures have been detected (i.e., that all of the necessary processing target images have been acquired by the image acquisition unit 120). In this case, the guide information output unit 130 can display, for example, a message on the display of the user terminal 10 indicating that acquisition of an image required for identity verification has been completed.

[0024] Note that the operation of the guide information output unit 130 is not limited to the content of the above specific example. For example, the guide information output unit 130 may be configured to switch the type and display position of the guide information in response to receiving a notification indicating that the processing target image has been acquired from the image acquisition unit 120.

[0025] When all processing target images necessary for identity verification have been acquired, the image processing system 1 transmits these processing target images to a server 40 that performs identity verification processing. The server 40 executes identity verification processing using the processing target images received from the image processing system 1. If the user is authenticated as the identity of the user in the processing by the server 40, the user can enjoy services such as opening an account and creating a credit card.

[0026] <Hardware configuration example> Each functional component of the image processing system 1 may be realized by hardware that realizes the functional component (e.g., a hardwired electronic circuit, etc.), or may be realized by a combination of hardware and software (e.g., a combination of an electronic circuit and a program that controls it, etc.). Below, a case where each functional component of the image processing system 1 is realized by a combination of hardware and software will be further described.

[0027] 2 is a block diagram illustrating an example of the hardware configuration of the image processing system 1. In this diagram, the image processing system 1 is implemented on a user terminal 10. For example, the environment of the image processing system 1 is constructed by installing a dedicated application on the user terminal 10.

[0028] The user terminal 10 includes a bus 1010 , a processor 1020 , a memory 1030 , a storage device 1040 , an input / output interface 1050 , and a network interface 1060 .

[0029] The bus 1010 is a data transmission path for transmitting and receiving data between the hardware components, but the method for connecting the hardware components of the user terminal 10 is not limited to bus connection.

[0030] The processor 1020 is implemented by a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), or the like.

[0031] The memory 1030 is a main storage device realized by a RAM (Random Access Memory) or the like.

[0032] The storage device 1040 is an auxiliary storage device realized by an HDD (Hard Disk Drive), an SSD (Solid State Drive), a memory card, a ROM (Read Only Memory), or the like. The storage device 1040 stores program modules that realize each function of the image processing system 1 (such as the image analysis unit 110, the image acquisition unit 120, and the guide information output unit 130). The processor 1020 loads and executes each of these program modules onto the memory 1030, thereby realizing each function corresponding to each program module on the user terminal 10.

[0033] The input / output interface 1050 is an interface for connecting the user terminal 10 with various input / output devices. In the example of FIG. 2, the image capture device 20 and the display 30 of FIG. 1 are connected to the input / output interface 1050. In other words, in the example of this figure, the image capture device 20 and the display 30 are mounted on the user terminal 10. Note that the display 30 connected to the input / output interface 1050 may be a touch panel display equipped with a touch panel. Other input / output devices such as a keyboard, a mouse, and a speaker may also be connected to the input / output interface 1050.

[0034] The network interface 1060 is an interface for connecting the user terminal 10 to a network. The network interface 1060 connects the user terminal 10 to the network via a wired or wireless connection. This network is, for example, a local area network (LAN) or a wide area network (WAN). The user terminal 10 can communicate with other devices on the network via the network interface 1060 to send and receive various data. For example, the user terminal 10 can communicate with a server 40 via the network interface 1060 and send an image to be processed to the server 40. In addition, if the imaging device 20 exists as a device separate from the user terminal 10 and is connected to the network, the user terminal 10 can communicate with the imaging device 20 via the network interface 1060 and acquire an image of an identification document from the imaging device 20.

[0035] Note that the configuration in FIG. 2 is merely an example, and the present invention is not limited to the content shown in FIG. 2. For example, some or all of the functions of the image processing system 1 may be provided in a device other than the user terminal 10. For example, the server 40 may have an image analysis unit 110, an image acquisition unit 120, and a guide information output unit 130. In this case, the user terminal 10 may transmit the video captured using the imaging device 20 to the server. 40 and send it to the server 40 performs the above-mentioned processes using the video acquired from the user terminal 10.

[0036] <Processing flow> 3 is a flowchart illustrating the flow of processing executed by the image processing system 1 of the first embodiment. This diagram illustrates an example of a flowchart when the image processing system 1 is implemented on the user terminal 10.

[0037] First, the user operates the user terminal 10 to start the application of the image processing system 1 installed in the user terminal 10 (S102). In response to the start of the application, the user terminal 10 communicates with the imaging device 20 connected to the user terminal 10 and starts capturing video (S104).

[0038] In response to the activation of the application, the guide information output unit 130 determines the type and display position of guide information to be output on the video acquired from the imaging device 20 (S106). For example, the guide information output unit 130 refers to guide information data stored in advance in a storage area such as the storage device 1040, and determines the type of guide information to be output first. If the type of guide information to be output first is predetermined, the guide information output unit 130 reads out the data of the guide information that is predetermined to be output first. For example, the guide information output unit 130 rotates the identification document to determine the type of guide information to be output first. of When capturing images of various postures with the imaging device 20, the guide information related to the front surface may be set as the guide information to be output first. Furthermore, the guide information output unit 130 determines the display position of the read guide information on the display 30, for example, randomly within the range in which the image of the imaging device 20 is displayed or based on a predetermined rule.

[0039] Here, depending on the performance of the imaging device 20 and the display 30, distortion or blurring may occur in an area of ​​the image near the boundary of the angle of view of the imaging device 20. If guide information is displayed in such an area, the accuracy of detecting the identification document and the orientation of the identification document may decrease. Therefore, the guide information output unit 130 controls the display position of the guide information to avoid such an area. Specifically, the guide information output unit 130 controls the display position of the guide information within a range narrower than the angle of view of the imaging device 20. Information regarding the angle of view of the imaging device 20 is added to the image of the imaging device 20, for example, as metadata. Alternatively, the guide information output unit 130 may acquire model information stored in the user terminal 10 and, based on the model information, acquire specification information (e.g., information regarding the angle of view) of the imaging device 20 installed in the user terminal 10.

[0040] The guide information output unit 130 outputs the type of guide information determined in the process of S106 to the display position also determined in the process of S106 (S108). The user moves the identification document in accordance with the guide information displayed on the display 30 so that the identification document moves to the specified position, and changes the orientation of the identification document so that it assumes the specified posture.

[0041] As described above, while the user is moving the identification document, the video captured by the imaging device 20 is supplied to the image processing system 1. The image analysis unit 110 sequentially analyzes frames (images) of the video supplied to the image processing system 1 (S110). The image analysis unit 110 may select all frames (images) of the supplied video as frames to be analyzed, or may select frames (images) to be analyzed at regular time intervals. Then, based on the analysis result of the image, the image analysis unit 110 determines whether or not a posture corresponding to the reference posture specified by the guide information output on the display 30 has been detected at a position corresponding to the display position of the guide information (S112).

[0042] When a posture corresponding to the reference posture specified by the guide information is not detected at a position corresponding to the display position of the guide information (S112: NO), the image analysis unit 110 continues to analyze the frames (images) of the video supplied to the image processing system 1. On the other hand, when a posture corresponding to the reference posture specified by the guide information is detected at a position corresponding to the display position of the guide information (S112: YES), the guide information output unit 130 further determines whether or not all of a plurality of preset reference postures have been detected (S114).

[0043] If all of the plurality of preset reference postures have not been detected (S114: NO), it means that all images necessary for identity verification have not been acquired, and the process proceeds to S106. Then, the guide information output unit 130 changes the type of guide information and the display position of the guide information. For example, the guide information output unit 130 displays another type of guide information newly determined in the process of S106 at a certain distance (for example, a fixed value of 20 to 30 pixels) from the display position of the current guide information. Then, the processes from S110 to S114 are repeatedly executed. On the other hand, if all of the plurality of preset reference postures have been detected (S114: YES), it means that all images necessary for identity verification have been acquired, and the process proceeds to S116. In the process of S116, the image processing system 1 transmits all of the acquired images to be processed to the server 40 that executes identity verification processing (S116).

[0044] As described above, according to the image processing system 1 of the present embodiment, guide information indicating the reference position of the identification document is output on the image viewed by the user to acquire an image required for identity verification. A user presenting an identification document can easily capture an image required for identity verification by moving the identification document according to the guide information. Furthermore, the image processing system 1 of the present embodiment automatically switches the type of guide information output on the image viewed by the user in response to the detection of a position corresponding to the reference position of the identification document. This allows the user to continuously capture images required for identity verification without performing any additional operations on the user terminal 10 to continue capturing images of the identification document. This improves the convenience of the system. Furthermore, according to the image processing system 1 of the present embodiment, when switching guide information, the new guide information is output at a position different from the position where the old guide information was output. This prevents the image required for identity verification from being acquired unless the display position of the guide information is confirmed in real time and the user follows the display. This can prevent, for example, the capture of an image of an identification document using a malicious program. For example, even if a malicious program that uses pre-recorded video is used against this system, unless the identification document shown in the video moves accurately in accordance with the display position of the guide information, an image for identification will not be acquired and the identification process will not be executed. In this respect, the image processing system 1 of this embodiment can improve the security of the identification process.

[0045] <Guide information display location> When changing the display position of the guide information, if the difference (amount of movement) between the display position of the guide information before and after the change is too small, the effect on the security described above will be reduced. Therefore, it is preferable that the guide information output unit 130 is configured to determine the display position of the changed guide information so that the difference (amount of movement) between the display position of the guide information before and after the change is equal to or greater than a predetermined first threshold.

[0046] On the other hand, when changing the display position of the guide information, if the difference (movement amount) between the display position of the guide information before and after the change is too large, the user will have to move their identification document by a large amount, which will reduce the above-mentioned convenience effect. Therefore, it is preferable to set an upper limit on the movement amount of the display position of the guide information. For example, the guide information output unit 130 may be configured to determine the display position of the changed guide information so that the movement amount before and after the change of the guide information falls within a range from the above-mentioned first threshold to a second threshold that is larger than the first threshold.

[0047] <Example of guide information screen display> 4 to 6 are diagrams showing examples of screens including guide information displayed by the guide information output unit 130 of the first embodiment. In Fig. 4 to Fig. 6, screens that are sequentially displayed on the display 30 over time are shown as examples.

[0048] Specifically, the guide information output unit 130 first displays a screen S1, as exemplified in FIG. 4, on the display 30. The screen S1, as exemplified in FIG. 4, has guide information G1 near the center of the display area of ​​the display 30. The guide information G1 is guide information for acquiring an image of the front of an identification document (here, a driver's license). In the example of FIG. 4, the guide information output unit 130 adjusts the area excluding the guide information G1 (the hatched area) so that it is displayed darker than the area of ​​the guide information G1. This makes it possible to make the guide information G1 stand out. When the user presents the identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G1 and at a position corresponding to the display position of the guide information G1, a first processing target image showing the front of the identification document (driver's license) is acquired.

[0049] When the first processing target image is acquired on the screen S1 (when a posture corresponding to the reference posture specified by the guide information G1 is detected), the guide information output unit 130 displays the screen S2 illustrated in FIG. 5 on the display 30. The screen S2 illustrated in FIG. 5 has the guide information G2 near the bottom of the display area of ​​the display 30. The guide information G2 is guide information for acquiring an image of the side of an identification document (here, a driver's license). In the example of FIG. 5, the guide information output unit 130 changes the display position of the guide information along the height direction (downward direction) of the display 30 (the display area that displays the image). Also, in the example of FIG. 5, the guide information output unit 130 adjusts the area excluding the guide information G2 (the hatched area) so that it is displayed darker than the area of ​​the guide information G2, as in FIG. 4. This makes the guide information G2 stand out. When the user presents his / her identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G2 and at a position corresponding to the display position of the guide information G2, a second processing target image showing the side of the identification document (driver's license) is obtained.

[0050] When the second processing target image is acquired on the screen S2 (when a posture corresponding to the reference posture specified by the guide information G2 is detected), the guide information output unit 130 displays the screen S3 illustrated in FIG. 6 on the display 30. The screen S3 illustrated in FIG. 6 has the guide information G3 near the center of the display area of ​​the display 30. The guide information G3 is guide information for acquiring an image of the back side of an identification document (here, a driver's license). In the example of FIG. 6, the guide information output unit 130 changes the display position of the guide information along the height direction (upward) of the display 30 (the display area that displays the image). Also, in the example of FIG. 6, the guide information output unit 130 adjusts the area excluding the guide information G3 (the hatched area) so that it is displayed darker than the area of ​​the guide information G3, as in FIGS. 4 and 5. This makes the guide information G3 stand out. When the user presents his / her identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G3 and at a position corresponding to the display position of the guide information G3, a third processing target image showing the back side of the identification document (driver's license) is obtained.

[0051] As described above, if the amount of movement of the guide information is too small, security may be reduced, and if the amount of movement of the guide information is too large, the convenience of the system may be reduced. Therefore, the guide information output unit 130 moves the changed guide information at least at the same display position as the guide information before the change. AlsoThe guide information output unit 130 may display the guide information at a position where the display positions of the guide information before and after the change overlap. Specifically, as shown in Fig. 7, for example, the guide information output unit 130 may adjust the display position of the guide information after the change so that the display position of the guide information before the change and the display position of the guide information after the change overlap by about 1 / 4 to 1 / 3 based on the display size. Fig. 7 is a diagram illustrating the relationship between the display positions of the guide information before and after the change. In the example of Fig. 7, the display position of the guide information before the change is indicated by a dotted line. Also, in the example of Fig. 7, the display position of the guide information after the change is indicated by a dashed line. Also, in the example of Fig. 7, the overlap of the display positions of the guide information before and after the change is indicated by a diagonal line. The guide information output unit 130 controls the size of the diagonal line area so that it becomes about 1 / 4 to 1 / 3 based on the size (height, width, area) of the guide information to be displayed.

[0052] <Modification> The guide information output unit 130 may be configured to change the guide information along the width direction of the display 30 (display area for displaying an image). In this case, the guide information output unit 130 recognizes the orientation of the user terminal 10 (the orientation of the display 30) based on information obtained from an inertial measurement device such as a gyro sensor mounted on the user terminal 10. Then, based on the orientation of the user terminal 10 (the orientation of the display 30), it controls the direction in which the display position of the guide information is changed. Furthermore, the guide information output unit 130 may be configured to control the display position of the guide information in the height direction and the width direction regardless of the orientation of the user terminal 10 (the orientation of the display 30).

[0053] Furthermore, although not shown, the image processing system 1 may further have a function to output any information on the display 30 to inform the user that the image to be processed is being acquired. For example, the image processing system 1 may have a function to output a progress bar or a predetermined message when acquiring the image to be processed. In this way, the user can visually recognize that the image required for identity verification has been acquired.

[0054] [Second embodiment] The image processing system 1 of this embodiment is similar to the image processing system 1 of the first embodiment except for the points described below.

[0055] <Functional configuration of image processing system 1> The image processing system 1 of this embodiment has an image analysis unit 110, an image acquisition unit 120, and a guide information output unit 130, similar to the configuration example (FIG. 1) of the image processing system 1 of the first embodiment. The image analysis unit 110 and the image acquisition unit 120 of this embodiment are similar to those of the first embodiment. The guide information output unit 130 of this embodiment differs from the guide information output unit 130 of the first embodiment in that it does not change the display position of the guide information. Specifically, the guide information output unit 130 of this embodiment changes the type of guide information to be output on the video according to the detection result of the posture of the identification document by the image analysis unit 110.

[0056] <Hardware configuration example> The hardware configuration of the image processing system 1 of this embodiment is the same as the example hardware configuration of the image processing system 1 of the first embodiment (FIG. 2). The storage device 1040 stores program modules that realize each function of the image processing system 1 of this embodiment (image analysis unit 110, image acquisition unit 120, guide information output unit 130, etc.). The processor 1020 loads and executes each of these program modules onto the memory 1030, thereby realizing each function corresponding to each program module on the user terminal 10.

[0057] <Processing flow> 8 is a flowchart illustrating the flow of processing executed by the image processing system 1 of the second embodiment. This diagram illustrates a flowchart in the case where the image processing system 1 is implemented on the user terminal 10.

[0058] First, the user operates the user terminal 10 to start the application of the image processing system 1 installed on the user terminal 10 (S202). In response to the start of the application, the user terminal 10 communicates with the imaging device 20 connected to the user terminal 10 and starts capturing video (S204). These processes are similar to the processes of S102 and S104 in FIG. 3.

[0059] In response to the activation of the application, the guide information output unit 130 determines the type of guide information to be output on the video acquired from the imaging device 20 (S206). For example, the guide information output unit 130 refers to guide information data stored in advance in a storage area such as the storage device 1040, and determines the type of guide information to be output first. If the type of guide information to be output first is predetermined, the guide information output unit 130 reads out the data of the guide information that is predetermined to be output first. For example, when rotating an identification document to display the image of the identification document, of When capturing images of various postures with the imaging device 20, guide information regarding the front surface may be set as the guide information to be output first. In this embodiment, the display position of the guide information may be determined in advance. For example, the guide information output unit 130 may output the guide information so that the center of the area of ​​the guide information overlaps with the center of the display 30 (display area).

[0060] The guide information output unit 130 outputs the type of guide information determined in the process of S206 to the display 30 (S208). The user moves the identification document in accordance with the guide information displayed on the display 30 so that the identification document moves to a specified position, and changes the orientation of the identification document so that it assumes a specified posture.

[0061] As described above, while the user is moving the identification document, the video captured by the imaging device 20 is supplied to the image processing system 1. The image analysis unit 110 sequentially analyzes frames (images) of the video supplied to the image processing system 1 (S210). Then, based on the analysis result of the image, the image analysis unit 110 determines whether or not a posture corresponding to the reference posture specified by the guide information output on the display 30 has been detected at a position corresponding to the display position of the guide information (S212). These processes are the same as the processes of S110 and S112 in FIG. 3.

[0062] If a posture corresponding to the reference posture specified by the guide information is not detected at a position corresponding to the display position of the guide information (S212: NO), the image analysis unit 110 continues to analyze the frames (images) of the video supplied to the image processing system 1. On the other hand, if a posture corresponding to the reference posture specified by the guide information is detected at a position corresponding to the display position of the guide information (S212: YES), the guide information output unit 130 further determines whether or not all of a plurality of preset reference postures have been detected (S21 4) These processes are similar to the processes in S112 and S114 in FIG.

[0063] If all of the plurality of preset reference postures have not been detected (S214: NO), the process proceeds to S206 because all of the images required for identity verification have not been acquired. Then, the guide information output unit 130 changes the type of guide information. For example, the guide information output unit 130 displays another type of guide information newly determined in the process of S206 at the display position of the current guide information. Then, the processes from S210 to S214 are repeatedly executed. On the other hand, if all of the plurality of preset reference postures have been detected (S214: YES), the process proceeds to S216 because all of the images required for identity verification have been acquired. In the process of S216, the image processing system 1 transmits all of the acquired images to be processed to the server 40 that executes identity verification processing (S216). The process of S216 is the same as the process of S116 in FIG. 3.

[0064] As described above, according to the image processing system 1 of this embodiment, guide information indicating the reference posture of the identification document is output on the image that the user confirms in order to acquire the image required for identity verification. The user who presents the identification document can easily capture the image required for identity verification by moving the identification document according to this guide information. Furthermore, in the image processing system 1 of this embodiment, the type of guide information that is output on the image that the user confirms is automatically switched in response to the detection of the posture that corresponds to the reference posture of the identification document. This allows the user to continuously capture images required for identity verification without performing any further operations on the user terminal 10 to continue capturing images of the identification document. In other words, the convenience of the system is improved. Furthermore, in the image processing system 1 of this embodiment, when switching guide information, the display position of the guide information is not changed before and after switching, unlike the image processing system 1 of the first embodiment. In this case, the user can easily capture the image required for identity verification by moving the identification document according to the reference posture of the identification document. document This eliminates the need to move the system, improving the user's convenience.

[0065] <Example of guide information screen display> 9 to 11 are diagrams showing examples of screens including guide information displayed by the guide information output unit 130 of the second embodiment. In Fig. 9 to Fig. 11, screens sequentially displayed on the display 30 over time are shown.

[0066] Specifically, the guide information output unit 130 first displays a screen S4, as exemplified in FIG. 9, on the display 30. The screen S4, as exemplified in FIG. 9, has guide information G4 near the center of the display area of ​​the display 30. The guide information G4 is guide information for acquiring an image of the front of an identification document (here, a driver's license). In the example of FIG. 9, the guide information output unit 130 adjusts the area excluding the guide information G4 (the hatched area) so that it is displayed darker than the area of ​​the guide information G4. This makes it possible to make the guide information G4 stand out. When the user presents the identification document (driver's license) in a posture corresponding to the reference posture specified by the guide information G4 and at a position corresponding to the display position of the guide information G4, a first processing target image showing the front of the identification document (driver's license) is acquired.

[0067] When the first processing target image is acquired on the screen S4 (when a posture corresponding to the reference posture specified by the guide information G4 is detected), the guide information output unit 130 displays the screen S5 exemplified in FIG. 10 on the display 30. The screen S5 exemplified in FIG. 10 has the guide information G5 at the same position as the guide information G4 on the screen S4 of FIG. 9. The guide information G5 is guide information for acquiring an image of the side of an identification document (here, a driver's license). In the example of FIG. 10, the guide information output unit 130 displays the area excluding the guide information G5 (the hatched area) as the image of the side of the identification document G4, as in FIG. 9. 5 This adjusts the area to be displayed darker than the area of ​​the guide information G. 5 When the user presents his / her personal identification document (driver's license) in a position corresponding to the display position of the guide information G5 in a position corresponding to the reference position specified by the guide information G5, a second processing target image showing the side of the personal identification document (driver's license) is acquired.

[0068] When the second processing target image is acquired on screen S5 (when a posture corresponding to the reference posture specified by guide information G5 is detected), the guide information output unit 130 displays screen S6, exemplified in FIG. 11, on the display 30. Screen S6, exemplified in FIG. 11, has guide information G6 at the same position as guide information G4 on screen S4 in FIG. 9 and guide information G5 on screen S5 in FIG. 10. Guide information G6 is guide information for acquiring an image of the back side of an identification document (here, a driver's license). In the example of FIG. 11, the guide information output unit 130 displays the area excluding guide information G6 (the hatched area) as guide information G6, as in FIGS. 9 and 10. 6 The area G6 is adjusted to be displayed darker than the area G6. This makes it possible to make the guide information G6 stand out. When the user presents the identification document (driver's license) in a position corresponding to the display position of the guide information G6 in a position corresponding to the reference position specified by the guide information G6, a third processing target image showing the back side of the identification document (driver's license) is acquired.

[0069] Although the embodiments of the present invention have been described above with reference to the drawings, the present invention should not be construed as being limited thereto, and various modifications and improvements can be made based on the knowledge of those skilled in the art without departing from the gist of the present invention. Furthermore, the multiple components disclosed in the embodiments can be appropriately combined to form various inventions. For example, some components may be deleted from all the components shown in the embodiments, or components from different embodiments may be appropriately combined.

[0070] In addition, in the flowcharts used in the above description, multiple steps (processes) are described in order, but the order of execution of the steps performed in each embodiment is not limited to the order described. In each embodiment, the order of the steps shown in the drawings can be changed to the extent that the content is not affected. Furthermore, the above-mentioned embodiments can be combined to the extent that the content is not contradictory.

[0071] Some or all of the above-described embodiments can be described as, but are not limited to, the following supplementary notes. 1. an image analysis means for analyzing an image in a video captured by an imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected; a guide information output means for outputting guide information representing a reference orientation of the personal identification document onto the image while changing the type and display position of the guide information according to the detection result of the orientation of the personal identification document; An image processing system comprising: 2. The guide information output means changing the type and display position of the guide information in response to the detection of a posture corresponding to the reference posture specified by the guide information as the posture of the personal identification document; 1. The image processing system according to claim 1. 3. The guide information output means a movement amount before and after the change of the display position of the guide information is set to a predetermined first threshold value or more; 1. The image processing system according to 1. or 2. 4. The guide information output means a movement amount before and after the change of the display position of the guide information is set to a second threshold value that is greater than the first threshold value or less; 3. The image processing system according to claim 3. 5. The guide information output means controlling the display position of the guide information within a range narrower than the angle of view of the imaging device; 1. An image processing system according to any one of 1. to 4. 6. The guide information output means changing a display position of the guide information along a height direction of a display area in which the video is displayed; 6. An image processing system according to any one of 1. to 5. 7. The guide information output means changing a display position of the guide information along a width direction of a display area in which the video is displayed; 7. An image processing system according to any one of 1. to 6. 8. an image analysis means for analyzing an image in a video captured by an imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected; a guide information output means for outputting guide information representing a reference orientation of the personal identification document onto the image while changing the type of guide information according to the detection result of the orientation of the personal identification document; An image processing system comprising: 9. 1. A computer-implemented image processing method comprising: Analyzing an image in a video captured by an imaging device to detect the orientation of the identification document in the image; acquiring an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected as an image to be processed; outputting guide information representing a reference orientation of the identification document onto the video image while changing the type and display position according to the detection result of the orientation of the identification document; An image processing method comprising: 10. The computer changing the type and display position of the guide information in response to the detection of a posture corresponding to the reference posture specified by the guide information as the posture of the personal identification document; 9. The image processing method according to claim 9, further comprising: 11. The computer a movement amount before and after the change of the display position of the guide information is set to a predetermined first threshold value or more; 11. The image processing method according to 9. or 10., 12. The computer a movement amount before and after the change of the display position of the guide information is set to a second threshold value that is greater than the first threshold value or less; 12. The image processing method according to claim 11, further comprising: 13. The computer controlling the display position of the guide information within a range narrower than the angle of view of the imaging device; 13. The image processing method according to any one of 9. to 12., 14. The computer changing a display position of the guide information along a height direction of a display area in which the video is displayed; 14. The image processing method according to any one of 9. to 13., 15. The computer changing a display position of the guide information along a width direction of a display area in which the video is displayed; 15. The image processing method according to any one of 9. to 14., 16. 1. A computer-implemented image processing method comprising: Analyzing an image in a video captured by an imaging device to detect the orientation of the identification document in the image; acquiring an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected as an image to be processed; outputting guide information representing a reference orientation of the identification document onto the video while changing the type of guide information depending on the detection result of the orientation of the identification document; An image processing method comprising: 17. Computer, image analysis means for analyzing an image in a video captured by an imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document has been detected; a guide information output means for outputting guide information representing a reference orientation of the personal identification document onto the image while changing the type and display position of the guide information according to the detection result of the orientation of the personal identification document; A program to function as a 18. The guide information output means changing the type and display position of the guide information in response to the detection of a posture corresponding to the reference posture specified by the guide information as the posture of the personal identification document; 17. The program described in. 19. The guide information output means a movement amount before and after the change of the display position of the guide information is set to a predetermined first threshold value or more; 17. or 18. 20. The guide information output means a movement amount before and after the change of the display position of the guide information is set to a second threshold value that is greater than the first threshold value or less; 19. The program described in. twenty one. The guide information output means controlling the display position of the guide information within a range narrower than the angle of view of the imaging device; A program according to any one of 17. to 20. twenty two. The guide information output means changing a display position of the guide information along a height direction of a display area in which the video is displayed; 21. The program according to any one of 17. to 21. twenty three. The guide information output means changing a display position of the guide information along a width direction of a display area in which the video is displayed; A program according to any one of 17. to 22. twenty four. Computer, image analysis means for analyzing an image in a video captured by an imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, as a processing target image, an image in the video in which a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected; a guide information output means for outputting guide information representing a reference orientation of the personal identification document onto the image while changing the type of guide information according to the detection result of the orientation of the personal identification document; A program to function as a

[0072] This application claims priority based on Japanese Patent Application No. 2021-007923, filed on January 21, 2021, the disclosure of which is incorporated herein in its entirety. [Explanation of symbols]

[0073] 1. Image processing system 10 User terminal 20 Imaging device 30 Display 40 servers 110 Image analysis unit 120 Image acquisition unit 130 Guide information output unit 1010 Bus 1020 processor 1030 memory 1040 Storage Device 1050 Input / Output Interface 1060 Network Interface G1, G2, G3, G4, G5, G6 Guide Information S1, S2, S3, S4, S5, S6 screen

Claims

1. an image analysis means for analyzing an image constituting a video captured by the imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, when a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected in the image displayed on the display, the image in which the posture is detected as an image to be processed; a guide information output means for outputting guide information representing a reference orientation of the personal identification document onto the image displayed on the display while changing the type and display position of the guide information in response to a position corresponding to the reference orientation being detected as the orientation of the personal identification document; Equipped with the guide information output means controls a display position of a second guide to be displayed after the first guide information is displayed, based on a predetermined limit range of a movement amount; Image processing system.

2. The guide information output means a movement amount before and after the change of the display position of the guide information is set to a predetermined first threshold value or more; The image processing system according to claim 1 .

3. The guide information output means a movement amount before and after the change of the display position of the guide information is set to a second threshold value that is greater than the first threshold value or less; The image processing system according to claim 2 .

4. The guide information output means controlling the display position of the guide information within a range narrower than the angle of view of the imaging device; The image processing system according to any one of claims 1 to 3.

5. The guide information output means changing a display position of the guide information along a height direction of a display area in which the video is displayed; The image processing system according to any one of claims 1 to 4.

6. The guide information output means changing a display position of the guide information along a width direction of a display area in which the video is displayed; The image processing system according to any one of claims 1 to 5.

7. 1. A computer-implemented image processing method comprising: Analyzing the images constituting the video captured by the imaging device and detecting the orientation of the personal identification document in the images; When a posture corresponding to one of a plurality of reference postures preset for the personal identification document is detected in the image displayed on the display, the image in which the posture is detected is acquired as an image to be processed; outputting guide information indicating the reference posture of the identification document onto the image being displayed on the display while changing the type and display position of the guide information in response to a posture corresponding to the reference posture being detected as the posture of the identification document; This includes: The display position of the second guide information to be displayed after the first guide information is displayed is controlled based on a predetermined limit range of the amount of movement. Image processing methods.

8. Computer, image analysis means for analyzing images constituting a video image captured by the imaging device and detecting the orientation of the personal identification document in the image; an image acquisition means for acquiring, when a posture corresponding to one of a plurality of reference postures preset for the identification document is detected in the image displayed on the display, the image in which the posture is detected as an image to be processed; a guide information output means for outputting guide information representing a reference posture of the personal identification document onto the image being displayed on the display, while changing the type and display position of the guide information in response to a posture corresponding to the reference posture being detected as the posture of the personal identification document; It functions as the guide information output means controls a display position of a second guide to be displayed after the first guide information is displayed, based on a predetermined limit range of a movement amount; program.

9. The guide information output means controls a moving direction of the guide information according to an orientation of the imaging device. The image processing system according to any one of claims 1 to 6.

Images