Information processing device, information processing system, information processing method, and computer program

Abstract

An information processing device 1 comprises: a detection unit 11 that detects a landmark from an input moving image that has been input; a generation unit 12 that uses a still image that has been input and the landmark to generate a composite moving image; and a determination unit 13 that, on the basis of a comparison between the input moving image and the composite image, determines whether the input moving image is a moving image that has been synthesized.

Description

Technical Field

[0001] This disclosure relates to the technical fields of information processing apparatuses, information processing systems, information processing methods, and recording media.

Background Art

[0002] Patent Document 1 describes a technique in which a first face image is detected from a first captured image captured by a camera, the first face image is non-linearly deformed using a template face image, the deformed first face image is recorded as a registered face image, a second face image is detected from a second captured image captured by the camera, the second face image is non-linearly deformed using the template face image, and the deformed second face image is collated with the registered face image.

[0003] Patent Document 2 describes a technique in which a first face is detected from an input search target image, the feature amount of the detected first face is calculated and recorded, a second face is detected from a key image for input search, the face angle of the second face is calculated, a synthesis pattern is determined according to the calculated face angle, a synthesized face image is generated according to the determined synthesis pattern, the feature amount of the second face is calculated using the generated synthesized face image, a search is performed in a database using the calculated plurality of face feature amounts as a query, and the plurality of retrieved search results are integrated to make it difficult for bias to occur in the search results even if there are differences in the shooting angles of the faces.

[0004] Patent Document 3 describes a technique having a plurality of operation modes for processing a plurality of moving image data obtained by imaging a subject under different imaging conditions, synthesizing the processed plurality of moving image data to generate a moving image, and executing at least one of a first subject detection process and a second subject detection process on either the processed moving image data or the generated moving image, and switching the operation mode according to the state of the subject or the features of the captured image.

[0005] Patent Document 4 describes a technique that accepts input of multiple biometric information of a person to be registered, uses a generated synthesis function to create and store a composite image of the multiple biometric information of the person to be registered, and when an authentication request including a composite image created from the multiple biometric information of the person to be authenticated is received from a terminal device, this composite image is compared with the stored composite image.

[0006] Patent Document 5 describes a technique in which, when facial image data is acquired, facial image data is generated from the facial image data by removing noise using a specific algorithm, difference image data is generated between the acquired facial image data and the generated facial image data, it is determined whether the acquired facial image data is a composite image or not based on the information contained in the difference image data, and if it is not determined that the acquired facial image data is a composite image, it is determined whether the acquired facial image data is a composite image or not based on the information contained in the frequency data generated from the difference image data. [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] International Publication No. 2015 / 128961 [Patent Document 2] International Publication No. 2013 / 176263 [Patent Document 3] Japanese Patent Publication No. 2015-012567 [Patent Document 4] Japanese Patent Publication No. 2010-044588 [Patent Document 5] International Publication No. 2022 / 162760 [Overview of the project] [Problems that the invention aims to solve]

[0008] This disclosure aims to provide an information processing device, an information processing system, an information processing method, and a recording medium that improve upon the technologies described in prior art documents. [Means for solving the problem]

[0009] One aspect of the information processing device includes detection means for detecting landmarks from an input video; generation means for generating a composite video using an input still image and the landmarks; and determination means for determining whether the input video is a composite video based on a comparison between the input video and the composite video.

[0010] One aspect of the information processing system is an information processing device comprising: detection means for detecting landmarks from an input video; generation means for generating a composite video using an input still image and the landmarks; determination means for determining whether the input video is a composite video based on a comparison between the input video and the composite video; and matching means for matching at least one of an object in the still image and an object in the input video; and authentication means for authenticating the object based on at least one of the determination result by the determination means and the matching result by the matching means.

[0011] One aspect of the information processing method involves detecting landmarks from an input video, generating a composite video using an input still image and the landmarks, and determining whether the input video is a composite video based on a comparison between the input video and the composite video.

[0012] One embodiment of a recording medium contains a computer program that causes a computer to execute an information processing method that detects landmarks from an input video, generates a composite video using an input still image and the landmarks, and determines whether or not the input video is a composite video based on a comparison between the input video and the composite video. [Brief explanation of the drawing]

[0013] [Figure 1] Figure 1 is a block diagram showing the configuration of the information processing device in the first embodiment. [Figure 2]FIG. 2 is a block diagram showing the configuration of the information processing apparatus in the second embodiment. [Figure 3] FIG. 3 is a flowchart showing the flow of the information processing operation of the information processing apparatus in the second embodiment. [Figure 4] FIG. 4 is a block diagram showing the configuration of the information processing apparatus in the third embodiment. [Figure 5] FIG. 5 is a flowchart showing the flow of the information processing operation of the information processing apparatus in the third embodiment. [Figure 6] FIG. 6 is a block diagram showing the configuration of the information processing apparatus in the fourth embodiment. [Figure 7] FIG. 7 is a flowchart showing the flow of the information processing operation of the information processing apparatus in the fourth embodiment. [Figure 8] FIG. 8 is a block diagram showing the configuration of the information processing system in the fifth embodiment [Figure 9] FIG. 9 is a flowchart showing the flow of the personal identification operation of the information processing system in the fifth embodiment.

Embodiments for Carrying Out the Invention

[0014] Hereinafter, embodiments of an information processing apparatus, an information processing system, an information processing method, and a recording medium will be described with reference to the drawings. [1: First Embodiment]

[0015] The first embodiment of an information processing apparatus, an information processing system, an information processing method, and a recording medium will be described. Hereinafter, the first embodiment of an information processing apparatus, an information processing system, an information processing method, and a recording medium will be described using the information processing apparatus 1 to which the first embodiment is applied. [1-1: Configuration of Information Processing Apparatus 1]

[0016] FIG. 1 is a block diagram showing the configuration of an information processing apparatus 1 in the first embodiment. As shown in FIG. 1, the information processing apparatus 1 includes a detection unit 11, a generation unit 12, and a determination unit 13.

[0017] The detection unit 11 detects landmarks from the input input video. The generation unit 12 generates a synthesized video using the input still image and the landmarks. The determination unit 13 determines whether the input video is a synthesized video based on a comparison between the input video and the synthesized video. [1-2: Technical effects of the information processing apparatus 1]

[0018] Since the information processing apparatus 1 in the first embodiment determines whether the input video is a synthesized video based on a comparison between the still image and the synthesized video generated using the landmarks, it is possible to accurately determine whether the input video is a synthesized video. [2: Second embodiment]

[0019] Next, a second embodiment of an information processing apparatus, an information processing system, an information processing method, and a recording medium will be described. Hereinafter, a second embodiment of an information processing apparatus, an information processing system, an information processing method, and a recording medium will be described using an information processing apparatus 2 to which the second embodiment is applied. [2-1: Fake video]

[0020] There is a technology that synthesizes an image of a person based on the information from a single facial photograph. One example of this technology for synthesizing images of people is deepfake. Deepfake is known as a technology that synthesizes fake videos that depict events that did not actually happen. Hereafter, videos depicting events that did not actually happen may be referred to as fake videos. Conversely, videos depicting events that actually happened may be referred to as genuine videos. For example, a genuine video may include a video showing actions performed by person B in front of the camera, as captured by the camera. In contrast, a fake video may include a video that synthesizes the actions performed by person B in front of the camera to appear as if they were performed by a different person A.

[0021] There is a technique called reenactment that generates fake videos in which the facial expressions of people in the original image change to a desired expression, or the people in the original image turn in a desired direction. For example, a technique is known that uses at least one facial image of person A to change the facial expression of person A in that image to match the facial expression of person B, thereby generating a video that appears as if person A is changing their expression (hereinafter sometimes referred to as "converting a still image into a video").

[0022] A still image can be converted into a video by using the original image and the original video. The original video may be, for example, a video showing the actions of person B in front of the camera, which is being filmed by the camera. The original image may be a still image of person A, who is different from person B. In converting a still image into a video, first, landmarks are detected in the original image. Then, landmarks are detected in each of the video frames that make up the original video. Next, for each video frame that makes up the original video, the original image is edited to match the landmark of the original image with the landmark of the corresponding video frame, and a composite frame is generated. By connecting each of the generated composite frames, the still image can be converted into a video. Landmarks may be characteristic positions of subjects in the image.

[0023] For example, the orientation and expression of person A in the original image can be changed using landmarks on the face of person B in the original video, and a video in which the orientation and expression of person A's face are changed can be synthesized. The landmarks that change the orientation and expression of a person's face may be distinctive parts of the face. The distinctive locations on the face may be specific points on parts such as the eyes, nose, and mouth.

[0024] If the obtained video is similar to a composite video created using still images, there is a high probability that the obtained video is a fake. In this embodiment, this property is used to determine whether or not a video is fake. Specifically, in this embodiment, a composite video is generated using still images, and the obtained video is compared with the composite image to determine whether or not it is a fake video. [2-2: Configuration of Information Processing Device 2]

[0025] Figure 2 is a block diagram showing the configuration of the information processing device 2 in the second embodiment. As shown in Figure 2, the information processing device 2 comprises an arithmetic unit 21 and a storage device 22. Furthermore, the information processing device 2 may also comprise a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not have to comprise at least one of the communication device 23, the input device 24, and the output device 25. The arithmetic unit 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0026] The arithmetic unit 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic unit 21 reads a computer program. For example, the arithmetic unit 21 may read a computer program stored in the storage device 22. For example, the arithmetic unit 21 may read a computer program stored in a computer-readable and non-temporary recording medium using a recording medium reading device (not shown) provided by the information processing device 2 (for example, an input device 24 described later). The arithmetic unit 21 may obtain a computer program from a device (not shown) located outside the information processing device 2 via a communication device 23 (or other communication device) (i.e., it may download or read the program). The arithmetic unit 21 executes the read computer program. As a result, logical functional blocks for performing the operations that the information processing device 2 should perform are realized within the arithmetic unit 21. In other words, the arithmetic unit 21 can function as a controller for realizing logical functional blocks necessary for the information processing device 2 to perform its operations (in other words, processing).

[0027] Figure 2 shows an example of a logical functional block implemented within the arithmetic unit 21 to perform information processing operations. As shown in Figure 2, the arithmetic unit 21 implements a detection unit 211, which is a specific example of the "detection means" described in the appendix below; a generation unit 212, which is a specific example of the "generation means" described in the appendix below; a determination unit 213, which is a specific example of the "determination means" described in the appendix below; a still image receiving unit 214; and a video receiving unit 215. However, the arithmetic unit 21 does not necessarily have to implement either the still image receiving unit 214 or the video receiving unit 215. The details of the operation of each of the detection unit 211, generation unit 212, determination unit 213, still image receiving unit 214, and video receiving unit 215 will be explained later with reference to Figure 3.

[0028] The storage device 22 is capable of storing desired data. For example, the storage device 22 may temporarily store computer programs executed by the arithmetic unit 21. The storage device 22 may temporarily store data that the arithmetic unit 21 uses temporarily when it is executing a computer program. The storage device 22 may store data that the information processing device 2 stores long-term. The storage device 22 may include at least one of the following: RAM (Random Access Memory), ROM (Read Only Memory), hard disk drive, magneto-optical disk drive, SSD (Solid State Drive), and disk array device. In other words, the storage device 22 may include non-temporary recording media.

[0029] The communication device 23 can communicate with devices outside the information processing device 2 via a communication network (not shown). The communication device 23 may have a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), or USB (Universal Serial Bus).

[0030] The input device 24 is a device that receives information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operating device (for example, at least one of a keyboard, mouse, and touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on an external recording medium that can be attached to the information processing device 2.

[0031] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (so-called display) capable of displaying an image showing the information to be output. In other words, in the second embodiment, the information processing device 2 includes a display D as the output device 25. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (so-called speaker) capable of outputting sound. For example, the output device 25 may output information onto paper. That is, the output device 25 may include a printing device (so-called printer) capable of printing desired information onto paper. [2-3: Information processing operations performed by the information processing device 2] The information processing operations performed by the information processing device 2 will be explained with reference to Figure 3. Figure 3 is a flowchart showing the flow of information processing operations performed by the information processing device 2.

[0032] As shown in Figure 3, the still image receiving unit 214 receives a still image of the subject (step S20). The still image receiving unit 214 may acquire a still image that includes the face region of the subject. The detection unit 211 may detect landmarks from the still image. The detection unit 211 may detect characteristic positions in the face region as landmarks from the still image. The detection unit 211 may detect specific points of parts such as the eyes, nose, and mouth as landmarks from the still image.

[0033] The video receiving unit 215 receives input video of the subject (step S21). The video receiving unit 215 may acquire input video that includes the subject's face region. Each input frame that makes up the input video may include the subject's face region. Here, it is unknown whether the input video acquired by the video receiving unit 215 is a real video or a fake video, but the video receiving unit 215 may acquire input video that shows the same subject as the subject shown in the still image acquired by the still image receiving unit 214.

[0034] The detection unit 211 detects landmarks from the input video (step S22). The detection unit 211 may detect landmarks from each of the input frames that make up the input video. The detection unit 211 may also detect landmarks from the input video at positions equivalent to those of landmarks detected from still images.

[0035] The generation unit 212 generates a composite video using the input still images and landmarks (step S23). First, the generation unit 212 may generate a composite frame by editing each input frame that makes up the input video so that the landmark of the still image matches the landmark of the corresponding input frame. Subsequently, the generation unit 212 may create a composite video by connecting each of the composite frames to turn the still images into a video.

[0036] The determination unit 213 compares the input video with the synthesized video (step S24), and determines whether the input video and the synthesized video are more similar than the standard (step S25). If the input video and the synthesized video are more similar than the standard (step S25: Yes), the determination unit 213 determines that the input video is a forged fake video (step S26). On the other hand, if the input video and the synthesized video are not more similar than the standard (step S25: No), the determination unit 213 determines that the input video is a genuine video that has not been forged (step S27). [2-4: Technical effects of the information processing device 2]

[0037] Synthesized videos generated using still images and landmarks can capture the characteristics of fake videos generated using techniques such as deepfakes. The information processing device 2 in the second embodiment utilizes the property that if the characteristics of the synthesized video generated using the input still images are similar to those of the input video, there is a high probability that the input video is a fake video. The information processing device 2 can accurately determine whether the input video is a genuine video that has not been forged or a fake video that has been forged. [3: Third Embodiment]

[0038] Next, a third embodiment of the information processing device, information processing system, information processing method, and recording medium will be described. In the following, the third embodiment of the information processing device, information processing system, information processing method, and recording medium will be described using an information processing device 3 to which the third embodiment of the information processing device, information processing system, information processing method, and recording medium is applied. [3-1: Configuration of Information Processing Device 3]

[0039] As shown in Figure 4, the information processing device 3 in the third embodiment includes a calculation unit 21 and a storage device 22, similar to the information processing device 2 in the second embodiment. Furthermore, the information processing device 3 in the third embodiment may also include a communication device 23, an input device 24, and an output device 25, similar to the information processing device 2 in the second embodiment. However, the information processing device 3 does not have to include at least one of the communication device 23, the input device 24, and the output device 25. The information processing device 3 in the third embodiment differs from the information processing device 2 in the second embodiment in that the determination unit 313 includes an extraction unit 3131 and a calculation unit 3132. Other features of the information processing device 3 may be the same as other features of the information processing device 2 in the second embodiment. For this reason, the following will describe in detail the parts that differ from each embodiment already described, and will omit explanations of other overlapping parts as appropriate. [3-2: Information processing operations performed by the information processing device 3] The information processing operations performed by the information processing device 3 will be explained with reference to Figure 5. Figure 5 is a flowchart showing the flow of information processing operations performed by the information processing device 3.

[0040] As shown in Figure 5, the still image receiving unit 214 receives input still images of the subject (step S20). The video receiving unit 215 receives input video of the subject (step S21). The detection unit 211 detects landmarks from the input video (step S22). The generation unit 212 generates a composite video using the input still images and landmarks (step S23).

[0041] The extraction unit 3131 extracts input features of face images included in the input video and composite features of face images included in the synthesized video (step S30). The extraction unit 3131 may use a feature extraction model to extract the input features and composite features. The feature extraction model may be a model that outputs features when a face image is input. The feature extraction model may be a model used in a face recognition mechanism. The feature extraction model may be a model constructed by machine learning. The extraction unit 3131 may extract input features of face images included in each input frame that constitutes the input video, and composite features of face images included in each synthesized frame that corresponds to each input frame that constitutes the synthesized video.

[0042] The calculation unit 3132 calculates the similarity between the input features and the composite features (step S31). The calculation unit 3132 may calculate the similarity between the input features and the composite features for each input frame. The calculation unit 3132 may also calculate the cosine similarity between the input features and the composite features. In addition, the calculation unit 3132 may calculate the Euclidean distance between the input features and the composite features.

[0043] The determination unit 313 determines whether the similarity between the input feature and the composite feature is greater than or equal to a predetermined value (step S32). For example, if the calculation unit 3132 calculates the cosine similarity, the determination unit 313 may determine whether the cosine similarity is closer to "1" than or equal to a predetermined value. Alternatively, if the calculation unit 3132 calculates the Euclidean distance, the determination unit 313 may determine whether the Euclidean distance is closer than or equal to a predetermined value.

[0044] Furthermore, for example, the determination unit 313 may determine whether the amount representing the average similarity of each input frame constituting the input video is above a predetermined value. Alternatively, the determination unit 313 may determine whether the number of input frames constituting the input video in which the similarity of each input frame is above a predetermined value is above a predetermined value.

[0045] If the similarity between the input features and the composite features is greater than or equal to a predetermined value (Step S32: Yes), the determination unit 313 determines that the input video is a forged fake video (Step S26). On the other hand, if the similarity between the input features and the composite features is less than a predetermined value (Step S25: No), the determination unit 313 determines that the input video is a genuine video that has not been forged (Step S27). [3-3: Technical Effects of Information Processing Device 3]

[0046] In the third embodiment, the information processing device 3 determines whether an input video is a fake video or not based on the similarity between the feature quantities of the input video and the feature quantities of the synthesized video synthesized using characteristic positions, thus enabling highly accurate determination. [4: Fourth Embodiment]

[0047] Next, a fourth embodiment of the information processing device, information processing system, information processing method, and recording medium will be described. In the following, the fourth embodiment of the information processing device, information processing system, information processing method, and recording medium will be described using the information processing device 4 to which the fourth embodiment of the information processing device, information processing system, information processing method, and recording medium is applied. [4-1: Information processing operations performed by the information processing device 4] The information processing operations performed by the information processing device 4 will be explained with reference to Figure 7. Figure 7 is a flowchart showing the flow of information processing operations performed by the information processing device 4.

[0048] As shown in Figure 7, the still image receiving unit 214 receives input still images of the subject (step S20). The video receiving unit 215 receives input video of the subject (step S21). The detection unit 211 detects landmarks from the input video (step S22). The generation unit 212 generates a composite video using the input still images and landmarks (step S23).

[0049] The extraction unit 4131 extracts the input features of the face images included in the input video and the composite features of the face images included in the composite video (step S30). The calculation unit 4132 calculates the similarity between the input features and the composite features (step S31).

[0050] The extraction unit 4131 extracts features from the still image (step S40). The calculation unit 4132 calculates the similarity between the input features and the features of the still image (step S41).

[0051] The determination unit 413 determines whether the similarity between the input features and the composite features is greater than the similarity between the input features and the features of the still image (step S42). If the similarity between the input features and the composite features is greater than the similarity between the input features and the features of the still image (step S42: Yes), the determination unit 413 determines that the input video is a fake video (step S26). In other words, the determination unit 413 determines that the input video is a composite video if the input video and the composite video are more similar than the input video and the still image. On the other hand, if the similarity between the input features and the composite features is not greater than the similarity between the input features and the features of the still image (step S42: No), the determination unit 413 determines that the input video is a real video (step S27). [4-2: Technical effects of the information processing device 4]

[0052] The similarity between videos synthesized from the same still image is often higher than the similarity between a still image and a video of the same person. The information processing device 4 in the fourth embodiment can use this property to accurately determine whether an input video is a fake video or not. [5: Fifth Embodiment]

[0053] Next, a fifth embodiment of the information processing device, information processing system, information processing method, and recording medium will be described. In the following, the fifth embodiment of the information processing device, information processing system, information processing method, and recording medium will be described using an information processing system 5 to which the fifth embodiment of the information processing device, information processing system, information processing method, and recording medium is applied. [5-1: Electronic Identity Verification]

[0054] The use of online identity verification methods such as electronic Know Your Customer (eKYC) is increasing. Financial institutions are increasingly conducting identity verification online, such as through eKYC, instead of in person, when opening bank accounts or issuing credit cards.

[0055] eKYC may be performed, for example, at an imaging location prepared for the eKYC service. Alternatively, eKYC may be performed at any location using a terminal device that can be used by the subject, such as a smartphone equipped with imaging and communication functions. Here is an example of an eKYC flow.

[0056] (Step 1) Capture a still image: Take an image of one side of the photograph on your identification document, such as a driver's license or My Number Card. You may also take an image of the other side of the identification document and the thickness of the document. (Step 2) Video recording: For example, the person is instructed to perform an action, such as turning from facing forward to turning to the right, and the person performing the action is recorded on video. (Step 3) Face image matching: This step verifies whether the person in the face photograph on the identification document is the same person as the person standing in front of the camera. Face image matching involves extracting features from the face photograph and from the face region detected in the video, and comparing the extracted features to determine whether the similarity between the features is above a predetermined value. (Step 4) Impersonation detection: Determine whether the subject is impersonating someone else based on their response to instructions for action. (Step 5) Identity Verification: Based on the results of Steps 3 and 4, determine whether identity verification was successful.

[0057] As mentioned above, there is a technology that can synthesize an image of a person based on information from a single facial photograph, posing a threat of impersonation in eKYC. Accurately determining whether a video is fake or not is a crucial issue in enhancing the reliability of services like eKYC. Input to eKYC includes facial images from official documents, which can be used to synthesize fake videos. In other words, one possible method of impersonation in eKYC is to synthesize and input fake videos based on limited information, such as facial images from official documents like driver's licenses or My Number cards.

[0058] The information processing system 5 in the fifth embodiment may be applied to online identity verification such as eKYC. In the information processing system 5 in the fifth embodiment, the system may determine whether the input video is a fake video created by impersonating someone else by comparing the input video with a synthesized video generated based on a facial photograph from an identity verification document such as a driver's license or My Number Card. [5-2: Configuration of Information Processing System 5]

[0059] As shown in Figure 8, the information processing system 5 in the fifth embodiment includes a arithmetic unit 21 and a storage device 22, similar to the information processing devices 2 in the second embodiment to 4 in the fourth embodiment. Furthermore, the information processing system 5 in the fifth embodiment may also include a communication device 23, an input device 24, and an output device 25, similar to the information processing devices 2 in the second embodiment to 4 in the fourth embodiment. However, the information processing system 5 does not have to include at least one of the communication device 23, input device 24, and output device 25. The information processing system 5 in the fifth embodiment differs from the information processing devices 2 in the second embodiment to 4 in the fourth embodiment in that a matching unit 516, an impersonation determination unit 517, and an authentication unit 518 are further implemented within the arithmetic unit 21. Other features of the information processing system 5 may be the same as at least one other feature of the information processing devices 2 in the second embodiment to 4 in the fourth embodiment. Therefore, the following will provide a detailed explanation of the parts that differ from the embodiments already described, while other overlapping parts will be omitted as appropriate.

[0060] The information processing system 5 may be a system capable of performing biometric authentication of the subject. The information processing system 5 may also be a device that performs image-based matching and uses the image to determine whether or not the subject is impersonating someone else, and authenticates the subject. [5-3: Information processing operations performed by Information Processing System 5] The information processing operations performed by the information processing system 5 will be explained with reference to Figure 9. Figure 9 is a flowchart showing the flow of information processing operations performed by the information processing system 5.

[0061] As shown in Figure 9, the still image receiving unit 214 receives input of a still image of the subject (step S20). In the fifth embodiment, the still image may be a facial image from an official document such as a driver's license or My Number Card. Step S20 may correspond to step 1 of the example eKYC flow described above. If the official document is a My Number Card, the verification unit 616 may acquire a facial image stored in the integrated circuit incorporated in the My Number Card. The verification unit 616 may acquire a facial image from the My Number Card read using, for example, a short-range wireless communication function installed in a smartphone carried by the subject. The video receiving unit 215 receives input of a video from the subject (step S21). Step S21 may correspond to step 2 of the example eKYC flow described above.

[0062] The matching unit 516 matches the subject's face image (step S50). If the still image is a face image from an official document such as a driver's license or My Number card, the matching unit 516 may match the subject in the still image with the subject in the input video. In this case, if the matching of the subject in the still image with the subject in the input video fails, the information processing operation may be terminated. Alternatively, the matching unit 516 may match the received still image with a pre-registered face image. Alternatively, the matching unit 516 may match the received input video with a pre-registered face image. In other words, the matching unit 516 may match at least one of the subject in the still image and the subject in the input video. Step S50 may correspond to step 3 of the example eKYC flow described above.

[0063] Furthermore, since still images and the composite videos created based on those still images are similar, there is a high probability that the matching of the still images and the input videos will be successful even if the input video is a fake video.

[0064] The impersonation detection unit 517 performs impersonation detection using the input video (step S51). In the fifth embodiment, the input video may be used for both detection of whether or not it is a fake video and detection of impersonation. For example, the input video may be a video showing an action performed by the subject at the instruction of the information processing system 5. The information processing system 5 may instruct the direction of the face, the direction of the gaze, and the position of the face. The information processing system 5 may guide the gaze. The information processing system 5 may instruct gestures. The impersonation detection unit 517 may perform active liveness detection using the input video. Step S51 may correspond to step 4 of the example eKYC flow described above.

[0065] The detection unit 211 detects landmarks from the input video (step S22). The generation unit 212 generates a composite video using the input still images and landmarks (step S23). The determination unit 213 compares the input video with the composite video (step S24) and determines whether the input video and the composite video are more similar than the standard (step S25). If the input video and the composite video are more similar than the standard (step S25: Yes), the determination unit 213 determines that the input video is a forged fake video (step S26). On the other hand, if the input video and the composite video are not more similar than the standard (step S25: No), the determination unit 213 determines that the input video is a genuine video that has not been forged (step S27).

[0066] If the determination unit 213 determines that the input video is not a fake video, the authentication unit 518 authenticates the subject based on the matching result by the matching unit 516 and the determination result by the impersonation determination unit 517 (step S52). The authentication unit 518 may also authenticate the subject if the determination unit 213 determines that the input video and the synthesized video are not as similar as the standard, and the impersonation determination unit 517 determines that the subject performed the action in accordance with the instructions. Successful authentication of the subject by the authentication unit 518 may also mean that the subject's identity has been confirmed. Step S52 may correspond to step 5 of the example eKYC flow described above.

[0067] Furthermore, the impersonation detection operation in step S51, which follows the matching operation in step S50, and the fake video detection operations from steps S22 to S27 may be performed in parallel. [5-4: Technical Effects of Information Processing System 5] In the fifth embodiment, the information processing system 5 performs a determination as to whether the input video is a fake video or not, thereby enabling accurate identity verification. [6: Addendum] The following additional information is disclosed regarding the embodiments described above. [Note 1] A detection means for detecting landmarks from an input video, A generation means that generates a composite video using the input still image and the landmark, A determination means that determines whether the input video is a composite video based on a comparison between the input video and the composite video. An information processing device equipped with the following features. [Note 2] The determination means determines that the input video is a composite video if the input video and the composite video are more similar than the standard. The information processing device described in Appendix 1. [Note 3] The determination means is, An extraction means for extracting input feature quantities of face images included in the input video and composite feature quantities of face images included in the composite video, A calculation means for calculating the similarity between the input feature and the composite feature. has The information processing device described in Appendix 1 or 2. [Note 4] The determination means determines that the input video is a composite video if the input video and the composite video are more similar than the input video and the still image. The information processing device described in Appendix 1 or 2. [Note 5] A detection means for detecting landmarks from an input video, A generation means that generates a composite video using the input still image and the landmark, A determination means that determines whether the input video is a composite video based on a comparison between the input video and the composite video. An information processing device comprising, A matching means for matching at least one of the objects captured in the still image and the objects captured in the input video, An authentication means for authenticating the target based on at least one of the determination result by the determination means and the matching result by the matching means. An information processing system that includes this. [Note 6] Detect landmarks from the input video, Using the input still images and the landmarks, a composite video is generated. Based on a comparison between the input video and the synthesized video, it is determined whether or not the input video is a synthesized video. Information processing methods. [Note 7] On the computer, Detect landmarks from the input video, Using the input still images and the landmarks, a composite video is generated. Based on a comparison between the input video and the synthesized video, it is determined whether or not the input video is a synthesized video. A recording medium on which a computer program for executing an information processing method is stored.

[0068] This disclosure may be modified from time to time, insofar as it does not contradict the technical idea that can be inferred from the claims and the entire specification. Information processing devices, information processing systems, information processing methods, and recording media that include such modifications are also included in the technical idea of ​​this disclosure. [Explanation of Symbols]

[0069] 1,2,3,4 Information Processing Devices 11,211 Detection Unit 12,212 generator 13,213,313,413 Judgment section 214 Still Image Reception Section 215 Video Reception Department 3131,4131 Extraction part 3132,4132 Calculation Unit 5. Information Processing Systems Verification unit 516 Impersonation detection unit 517 Authentication section 518

Claims

1. A detection means for detecting landmarks of face regions in each video frame that constitutes the input video, A generation means that generates a composite video using the input still image and the landmark, A determination means that determines whether the input video is a composite video based on a comparison between the input video and the composite video. An information processing device equipped with the following features.

2. The determination means determines that the input video is a composite video if the input video and the composite video are more similar than the standard. The information processing apparatus according to claim 1.

3. The determination means is, An extraction means for extracting input feature quantities of face images included in the input video and composite feature quantities of face images included in the composite video, A calculation means for calculating the similarity between the input feature and the composite feature. has The information processing apparatus according to claim 1 or 2.

4. The determination means determines that the input video is a composite video if the similarity between the input video and the composite video is greater than the similarity between the input video and the still image. The information processing apparatus according to claim 1.

5. A detection means for detecting landmarks of face regions in each video frame that constitutes the input video, A generation means that generates a composite video using the input still image and the landmark, A determination means that determines whether the input video is a composite video based on a comparison between the input video and the composite video. An information processing device comprising, A matching means for comparing at least one of the objects captured in the still image and the objects captured in the input video with a pre-registered registered face image, An authentication means for authenticating the target based on at least one of the determination result by the determination means and the matching result by the matching means. An information processing system that includes this.

6. From each of the video frames constituting the input video, landmarks of the face region in the video frame are detected. Using the input still images and the landmarks, a composite video is generated. Based on a comparison between the input video and the synthesized video, it is determined whether or not the input video is a synthesized video. The information processing method performed by computers.

7. On the computer, From each of the video frames constituting the input video, landmarks of the face region in the video frame are detected. Using the input still images and the landmarks, a composite video is generated. Based on a comparison between the input video and the synthesized video, it is determined whether or not the input video is a synthesized video. A computer program that executes an information processing method.

Images