Information processing device, information processing method, and computer program

Abstract

An information processing device 1 comprises: a reception unit 11 that receives input of a first image and a second image; a synthesis unit 12 that synthesizes a third image on the basis of the first image and the second image; a difference emphasis unit 13 that emphasizes a difference between the second image and the third image; a calculation unit 14 that calculates an index representing the likeness of the second image to a synthesized image on the basis of the emphasized difference; and a determination unit 15 that determines whether the second image is a synthesized image according to the index.

Description

Information processing device, information processing method, and recording medium

[0001] The present disclosure relates to the technical fields of an information processing device, an information processing method, and a recording medium.

[0002] Patent Document 1 describes a technology in which a class indicates whether a person's face (object) is real (true) or fake (false), and a synthesized area (i.e., an area added to an original video) is detected as an element of the detection target class to detect whether the video is a fake video (e.g., a false video generated by a synthesis process, etc.).

[0003] International Publication No. 2022 / 054246

[0004] An object of the present disclosure is to provide an information processing device, an information processing method, and a recording medium that are intended to accurately detect whether an input image is a synthesized image.

[0005] One aspect of the information processing device includes a receiving means for receiving input of a first image and a second image, a synthesis means for synthesizing a third image based on the first image and the second image, a difference enhancement means for enhancing the difference between the second image and the third image, a calculation means for calculating an index representing the likelihood that the second image is a synthesized image based on the enhanced difference, and a determination means for determining whether the second image is a synthesized image based on the index.

[0006] One aspect of the information processing method includes accepting input of a first image and a second image, synthesizing a third image based on the first image and the second image, emphasizing the difference between the second image and the third image, calculating an index representing the likelihood of the second image being a synthesized image based on the emphasized difference, and determining whether the second image is a synthesized image based on the index.

[0007] One aspect of the recording medium has recorded thereon a computer program for causing a computer to execute an information processing method that accepts input of a first image and a second image, synthesizes a third image based on the first image and the second image, emphasizes the difference between the second image and the third image, calculates an index representing the likelihood that the second image is a synthesized image based on the emphasized difference, and determines whether the second image is a synthesized image based on the index.

[0008] The information processing device, information processing method, and recording medium according to the present disclosure can accurately detect whether an input image is a composite image.

[0009] FIG. 1 is a block diagram showing a configuration of a first information processing device according to the present disclosure. FIG. 2 is a block diagram showing a configuration of a second information processing device according to the present disclosure. FIG. 3 is a flowchart showing a processing operation of the second information processing device according to the present disclosure. FIG. 4 is a block diagram showing a configuration of a third information processing device according to the present disclosure. FIG. 5 is a flowchart showing a processing operation of the third information processing device according to the present disclosure. FIG. 6 is a block diagram showing a configuration of a fourth information processing device according to the present disclosure. FIG. 7 is a flowchart showing a processing operation of the fourth information processing device according to the present disclosure.

[0010] Hereinafter, an information processing device, an information processing method, and a recording medium according to an embodiment will be described with reference to the drawings. [1: First Embodiment]

[0011] A first embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the first embodiment of the information processing device, the information processing method, and the recording medium will be described using a first information processing device 1 according to the present disclosure. [1-1: Configuration of Information Processing Device 1]

[0012] FIG. 1 is a block diagram showing the configuration of a first information processing device 1 according to the present disclosure. As shown in FIG. 1, the information processing device 1 includes a receiving unit 11, a combining unit 12, a difference emphasizing unit 13, a calculation unit 14, and a determination unit 15. The receiving unit 11 receives input of a first image and a second image. The combining unit 12 combines a third image based on the first image and the second image. The difference emphasizing unit 13 emphasizes the difference between the second image and the third image. The calculation unit 14 calculates an index representing the likelihood of the second image being a combined image based on the emphasized difference. The determination unit 15 determines whether the second image is a combined image according to the index. [1-2: Technical Effects of the Information Processing Device 1]

[0013] The first information processing device 1 according to the present disclosure makes a determination based on the difference between an input image and a composite image, and therefore can accurately detect whether an input image is a composite image. [2: Second Embodiment]

[0014] A second embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the second embodiment of an information processing device, an information processing method, and a recording medium will be described using a second information processing device 2 according to the present disclosure. [2-1: Fake Image]

[0015] There is a technology that synthesizes an image of a person based on information from a single photograph of the person's face. Deepfake, for example, is known as a technology for synthesizing an image of a person. Deepfake is known as a technology that synthesizes a fake image that shows something that did not actually happen. Hereinafter, an image that shows something that did not actually happen may be referred to as a fake image. A synthesized image may also be referred to as a fake image. An image that shows something that actually happened may also be referred to as a real image. [2-2: Configuration of Information Processing Device 2]

[0016] 2 is a block diagram showing the configuration of the second information processing device 2. As shown in FIG. 2, the information processing device 2 includes a calculation device 21 and a storage device 22. The information processing device 2 may further include a communication device 23, an input device 24, and an output device 25. However, the information processing device 2 does not necessarily have to include at least one of the communication device 23, the input device 24, and the output device 25. The calculation device 21, the storage device 22, the communication device 23, the input device 24, and the output device 25 may be connected via a data bus 26.

[0017] The arithmetic device 21 includes, for example, at least one of a CPU (Central Processing Unit), a GPU (Graphics Processing Unit), and an FPGA (Field Programmable Gate Array). The arithmetic device 21 reads a computer program. For example, the arithmetic device 21 may read a computer program stored in the storage device 22. For example, the arithmetic device 21 may read a computer program stored in a computer-readable, non-transitory recording medium using a recording medium reading device (e.g., an input device 24 described later) not shown in the drawings that is included in the information processing device 2. The arithmetic device 21 may acquire (i.e., download or read) the computer program from a device (not shown) located outside the information processing device 2 via the communication device 23 (or another communication device). The arithmetic device 21 executes the read computer program. As a result, logical functional blocks for executing the operations to be performed by the information processing device 2 are realized within the arithmetic device 21. In other words, the arithmetic device 21 can function as a controller for realizing logical functional blocks for executing the operations (in other words, processing) to be performed by the information processing device 2.

[0018] FIG. 2 shows an example of logical functional blocks implemented within the arithmetic device 21 to perform information processing operations. As shown in FIG. 2 , the arithmetic device 21 includes a reception unit 211, which is a specific example of the "reception means" described in the appendix below; a synthesis unit 212, which is a specific example of the "synthesis means" described in the appendix below; a difference emphasis unit, which is a specific example of the "difference emphasis means" described in the appendix below; a calculation unit 214, which is a specific example of the "calculation means" described in the appendix below; a determination unit 215, which is a specific example of the "determination means" described in the appendix below; and an output unit 216. However, the output unit 216 does not necessarily have to be implemented within the arithmetic device 21. The difference emphasis unit may include an extraction unit 2131 and an emphasis unit 2132. Details of the operations of the reception unit 211, synthesis unit 212, difference emphasis unit, calculation unit 214, determination unit 215, and output unit 216 will be described later with reference to FIG. 3 .

[0019] The storage device 22 can store desired data. For example, the storage device 22 may temporarily store a computer program executed by the arithmetic device 21. The storage device 22 may temporarily store data that the arithmetic device 21 temporarily uses when the arithmetic device 21 is executing a computer program. The storage device 22 may store data that the information processing device 2 stores long-term. The storage device 22 may include at least one of a RAM (Random Access Memory), a ROM (Read Only Memory), a hard disk device, a magneto-optical disk device, an SSD (Solid State Drive), and a disk array device. In other words, the storage device 22 may include a non-temporary recording medium.

[0020] The communication device 23 is capable of communicating with devices external to the information processing device 2 via a communication network (not shown). The communication device 23 may be a communication interface based on standards such as Ethernet (registered trademark), Wi-Fi (registered trademark), Bluetooth (registered trademark), or USB (Universal Serial Bus).

[0021] The input device 24 is a device that accepts information input to the information processing device 2 from outside the information processing device 2. For example, the input device 24 may include an operation device (e.g., at least one of a keyboard, a mouse, and a touch panel) that can be operated by an operator of the information processing device 2. For example, the input device 24 may include a reading device that can read information recorded as data on a recording medium that can be externally attached to the information processing device 2.

[0022] The output device 25 is a device that outputs information to the outside of the information processing device 2. For example, the output device 25 may output information as an image. That is, the output device 25 may include a display device (a so-called display) that can display an image showing the information to be output. For example, the output device 25 may output information as sound. That is, the output device 25 may include an audio device (a so-called speaker) that can output sound. For example, the output device 25 may output information on paper. That is, the output device 25 may include a printing device (a so-called printer) that can print desired information on paper. [2-3: Information Processing Operation Performed by the Information Processing Device 2]

[0023] The information processing operation performed by the information processing device 2 will be described with reference to Fig. 3. Fig. 3 is a flowchart showing the flow of the information processing operation performed by the information processing device 2. Note that this disclosure assumes that the first image is not a fake image but a real image.

[0024] 3, the receiving unit 211 receives an input of a first image (step S20). The receiving unit 211 may receive an input of a face image including a facial region of a person as the first image. The first image may be a still image. Hereinafter, the "first image" may be referred to as a "source image."

[0025] The receiving unit 211 receives input of a second image (step S21). The receiving unit 211 may receive input of a face image including a person's face area as the second image. The second image may be a still image. The second image may be a moving image. In the second embodiment, a case where the second image that is a still image is processed will be described. Hereinafter, the "second image" may be referred to as a "target image."

[0026] The synthesis unit 212 synthesizes a third image based on the first image and the second image. Hereinafter, the "third image" may be referred to as a "synthetic image." The synthesis unit 212 may generate the synthetic image using, for example, a technique called face swap. Face swap is a technique for exchanging a facial region of a source image with a facial region of a target image. The synthesis unit 212 may generate the synthetic image by fitting a facial region of the determination target image into a facial region of the source image. The synthesis unit 212 may generate a synthetic image having characteristics of the source image. For example, the synthesis unit 212 may generate a synthetic image that maintains the facial expression of the source image.

[0027] The extraction unit 2131 extracts the difference between the determination target image and the composite image (step S23). In other words, the extraction unit 2131 extracts the part where the determination target image and the composite image differ. The extraction unit 2131 may calculate the difference in pixel value for each pixel and generate a difference image in which each pixel is represented by the difference in pixel value.

[0028] The enhancement unit 2132 enhances the difference (step S24). The enhancement unit 2132 may further enhance the difference by, for example, multiplying each pixel value of the difference image by a real number. That is, the enhancement unit 2132 may increase the pixel values ​​of pixels in the difference image whose pixel values ​​are not 0 (i.e., there is no difference in pixel value between the image to be determined and the composite image). Hereinafter, a difference image in which the difference is enhanced by the enhancement unit 2132 may be referred to as a "difference-enhanced image." For example, the enhancement unit 2132 may generate a difference-enhanced image by setting the pixel values ​​of pixels in the difference image whose pixel values ​​are not 0 to the maximum possible value. For example, the enhancement unit 2132 may generate a difference-enhanced image by setting the pixel values ​​of pixels in the difference image whose pixel values ​​are equal to or greater than a predetermined value to the maximum possible value. For example, the enhancement unit 2132 may set a range of pixel values ​​in the difference image to generate a difference-enhanced image that can have four types of pixel values: large difference, medium difference, small difference, and no difference. The enhancement unit 2132 may employ any method to enhance the difference.

[0029] The calculation unit 214 calculates an index representing the likelihood that the image to be determined is a fake image based on the emphasized difference (step S25). The index calculated by the calculation unit 214 may be a real number. The calculation unit 214 may calculate an index representing the likelihood that the image to be determined is a fake image using a calculation model. The calculation model is a model that outputs an index representing the likelihood that the image is a fake image when the emphasized difference is input. The calculation model may be a machine-learned model. A learning mechanism that trains the calculation model may train the calculation model using difference-enhanced images that have information indicating a correct answer (that the image to be determined is a real image, or that the image to be determined is a fake image) as training data. The learning mechanism may use the information indicating the correct answer and the index representing the likelihood that the image is a fake image output by the calculation model to have the calculation model learn a method for calculating the index.

[0030] The determination unit 215 determines whether the second image is a composite image based on the index (step S26). The determination unit 215 may determine whether the second image is a composite image by comparing the index with a predetermined threshold.

[0031] If the index exceeds the predetermined threshold (step S26: Yes), the determination unit 215 determines that the second image is a composite image (step S27).If the index does not exceed the predetermined threshold (step S26: No), the determination unit 215 determines that the second image is a non-composite image (step S28).

[0032] The output unit 216 outputs according to the determination result (step S29). The output unit 216 may control the output device 25 to cause the output device 25 to output according to the determination result. [2-4: Technical Effects of Information Processing Device 2]

[0033] When an emphasized difference is used in the comparison judgment between the judgment target image and the composite image, it is easy to see the difference between when the judgment target image is a fake image and when it is not, and it is easy to determine whether the judgment target image is a fake image or not. The second information processing device 2 of the present disclosure emphasizes the difference between the input judgment target image and the composite image and makes a judgment based on the emphasized difference, so it can accurately detect whether the input judgment target image is a fake image or not. Furthermore, since the information processing device 2 judges whether an image is a fake image by comparing it with a threshold value, it is possible to adjust how likely an image is to be judged as a fake image by setting the threshold value. [3: Third embodiment]

[0034] A third embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the third embodiment of an information processing device, an information processing method, and a recording medium will be described using a third information processing device 3 according to the present disclosure. In the third embodiment, a case will be described in which the second image is a moving image including a plurality of frames. Hereinafter, the "second image" may be referred to as a "determination target moving image." Furthermore, a moving image showing events that did not actually occur may be referred to as a fake moving image. Furthermore, a moving image showing events that actually occurred may be referred to as a real moving image. [3-1: Fake moving image]

[0035] For example, a genuine video may include a video showing an action taken by person B in front of the camera. In contrast, a fake video may include a moving image synthesized to make it appear as if person A, who is different from person B, had taken the action taken by person B in front of the camera.

[0036] There is a technology called reenactment that generates fake videos in which the facial expressions of people in original images are changed to desired expressions or people in original images are turned in desired directions. For example, a technology is known that, based on at least one facial image of person A, changes the facial expression of person A in the facial image to match the facial expression of person B, thereby generating a moving image that makes it appear as if person A is changing his or her facial expression (hereinafter, this may be referred to as "animating a still image").

[0037] By using an original image and an original video, a still image can be animated. The original video may be, for example, a video image showing the actions of person B in front of the camera as captured by the camera. The original image may also be a still image showing person A, who is different from person B. When animating a still image, landmarks in the original image are first detected. Landmarks are also detected from each of the video frames that make up the original video. Next, for each video frame that makes up the original video, the original image is edited so that the landmarks in the original image and the landmarks in the corresponding video frame are aligned to generate a composite frame. The still image can be animated by connecting each of the generated composite frames. Landmarks may be characteristic positions of a subject appearing in the image.

[0038] For example, the facial orientation, expression, etc. of person A appearing in the original image can be changed using landmarks on the face of person B appearing in the original video, thereby synthesizing a video in which the facial orientation, expression, etc. of person A changes. The landmarks that change the facial orientation, expression, etc. of a person may be characteristic parts of the face. The characteristic positions on the face may be specific points on parts such as the eyes, nose, and mouth.

[0039] If an acquired moving image is similar to a composite moving image created using still images, the acquired moving image is likely to be a fake moving image. In this embodiment, this property is used to determine whether or not the moving image is a fake moving image. That is, in the third embodiment, a composite moving image is generated using still images, and the acquired moving image is compared with the composite moving image to determine whether or not the acquired moving image is a fake moving image. [3-2: Configuration of Information Processing Device 3]

[0040] The configuration of the third information processing device 3 will be described with reference to Fig. 4. Fig. 4 is a block diagram showing the configuration of the third information processing device 3.

[0041] As shown in FIG. 4, the third information processing device 3 includes a calculation device 21 and a storage device 22, similar to the second information processing device 2. Furthermore, the third information processing device 3 may include a communication device 23, an input device 24, and an output device 25, similar to the second information processing device 2. However, the information processing device 3 does not have to include at least one of the communication device 23, the input device 24, and the output device 25. The third information processing device 3 differs from the second information processing device 2 in that the synthesis unit 312 includes a detection unit 3121. Other features of the information processing device 3 may be the same as other features of the information processing device 2. Therefore, hereinafter, differences from the embodiments already described will be described in detail, and descriptions of other overlapping parts will be omitted as appropriate. [3-3: Information Processing Operation Performed by Information Processing Device 3]

[0042] The flow of the information processing operation performed by the information processing device 3 will be described with reference to Fig. 5. Fig. 5 is a flowchart showing the flow of the information processing operation performed by the information processing device 3.

[0043] 5, the receiving unit 311 receives input of a source image as a first image (step S20). The detecting unit 3121 detects landmarks from the source image. The detecting unit 3121 may detect characteristic positions in a face area as landmarks from a still image. The detecting unit 3121 may detect specific points of parts such as the eyes, nose, and mouth as landmarks from the source image.

[0044] The receiving unit 311 receives input of a video to be determined as a second image (step S30). The detection unit 3121 detects landmarks from each of one or more frames included in the video to be determined (step S31). The one or more frames included in the video to be determined may be all frames included in the video to be determined. The one or more frames included in the video to be determined may be any one or more frames included in the moving image. The detection unit 3121 may detect, from the image to be determined, landmarks at positions equivalent to the landmarks detected from the source image.

[0045] The composition unit 312 composes a third image based on the source image, the landmarks of the source image, and the landmarks of one or more frames included in the target moving image (step S32). In the third embodiment, the third image is a composite moving image including one or more frames. Hereinafter, the "third image" may be referred to as a "composite moving image."

[0046] The composition unit 312 may first generate a composite frame by editing each input frame constituting the determination target moving image so as to match the landmarks of the source image with the landmarks of the corresponding input frame. Next, the composition unit 312 may connect each composite frame together to animate the source images, which are still images, and generate a composite moving image.

[0047] The extraction unit 3131 extracts the difference between the determination target moving image and the composite moving image (step S33). The extraction unit 3131 extracts the difference between a frame included in the determination target moving image and a frame included in the composite moving image corresponding to that frame. The extraction unit 3131 may extract the difference between the determination target moving image and the composite moving image and generate a difference moving image. Determination target moving image d ｉ If the target video d includes frames 1 to F, ｉ to [x ｉ １ , ..., x ｉ Ｆ ]. The composite video can be expressed as d ｆ If contains frames 1 to F, then the composite video d ｆ to [x ｆ １ , ..., xｉ Ｆ In this case, the difference video d ｄｉｆｆ is d ｆ -d ｉ =[|x ｆ １ -x ｉ １ |, ..., |x ｆ Ｆ -x ｉ Ｆ In step S31, if the detection unit 3121 detects landmarks from any one or more frames included in the determination target moving image, the extraction unit 3131 may generate a difference moving image including difference frames corresponding to the any one or more frames.

[0048] The highlighting unit 3132 may highlight the difference (step S34). The highlighting unit 3132 may generate a difference-highlighted video including a difference-highlighted frame that highlights the difference between a frame included in the determination target video and a frame included in the composite video corresponding to that frame. ｄｉｆｆ is [α|x ｆ １ -x ｉ １ |, ..., α|x ｆ Ｆ -x ｉ Ｆ |], where α is a real number and is a parameter for further emphasizing the difference.

[0049] The calculation unit 314 calculates an index representing the likelihood that the determination target video is a fake video based on one or more frames included in the difference-enhanced video (step S35). The calculation unit 314 may calculate an index representing the likelihood that the determination target image is a fake image using a calculation model. In the third embodiment, when one or more frames included in the difference-enhanced video are input, the calculation model outputs an index representing the likelihood that the image is a fake image. The one or more frames included in the difference-enhanced video may be all frames included in the difference-enhanced video. The one or more frames included in the difference-enhanced video may be any one or more frames included in the video.

[0050] The determination unit 315 determines whether the target video is a fake video or not based on the index (step S36). The determination unit 315 may determine whether the target video is a fake video or not by comparing the index with a predetermined threshold.

[0051] If the index exceeds a predetermined threshold (step S36: Yes), the determination unit 315 determines that the video to be determined is a fake video (step S37). If the index does not exceed the predetermined threshold (step S36: No), the determination unit 315 determines that the video to be determined is not a fake video (step S38). The output unit 316 outputs according to the determination result (step S39). [3-4: Technical Effects of Information Processing Device 3]

[0052] A composite video generated using still images and landmarks can capture the characteristics of a fake video generated using technology such as deepfake. The third information processing device 3 of the present disclosure utilizes the property that if the characteristics of a composite video generated using an input source image, which is a still image, are similar to those of the input video to be determined, the video to be determined is likely to be a fake video. The third information processing device 3 of the present disclosure can accurately determine whether the video to be determined is an authentic video that has not been forged, or whether the video to be determined is a forged fake video. Furthermore, the information processing device 3 can accurately detect whether the input video to be determined is a fake video or not, based on the difference between each frame. Furthermore, the information processing device 3 can accurately detect deepfakes generated using landmarks. [4: Fourth Embodiment]

[0053] A fourth embodiment of an information processing device, an information processing method, and a recording medium will be described below. Hereinafter, the fourth embodiment of an information processing device, an information processing method, and a recording medium will be described using a fourth information processing device 4 according to the present disclosure. [4-1: Configuration of Information Processing Device 4]

[0054] The configuration of the fourth information processing device 4 will be described with reference to Fig. 6. Fig. 6 is a block diagram showing the configuration of the fourth information processing device 4.

[0055] As shown in FIG. 6 , the fourth information processing device 4 includes a calculation device 21 and a storage device 22, similar to the second information processing device 2 and the third information processing device 3. Furthermore, the fourth information processing device 4 may include a communication device 23, an input device 24, and an output device 25, similar to the second information processing device 2 and the third information processing device 3. However, the information processing device 4 does not need to include at least one of the communication device 23, the input device 24, and the output device 25. The fourth information processing device 4 differs from the second information processing device 2 and the third information processing device 3 in that a matching unit 417, a spoofing determination unit 418, and an authentication unit 419 are further implemented within the calculation device 21. Other features of the information processing device 4 may be the same as other features of at least one of the information processing device 2 and the information processing device 3. Therefore, the following will describe in detail the differences from the already-described embodiments, and will omit descriptions of other overlapping features as appropriate.

[0056] The fourth information processing device 4 is a mechanism capable of performing biometric authentication of a person. The information processing device 4 may be a mechanism capable of performing a matching operation using an image and determining whether a person is impersonating another person using the image, thereby authenticating the person.

[0057] The fourth information processing device 4 of the present disclosure may be applied to online identity verification such as electronic know your customer (eKYC). As described above, there is a technology that synthesizes an image of a person based on information from a single photograph of the person's face, posing a threat of impersonation in eKYC. Accurately determining whether a video is fake or not is an important issue in increasing the reliability of services such as eKYC. Input to eKYC includes facial images from official documents that serve as information for synthesizing fake videos. In other words, one possible method of impersonating an eKYC user is to synthesize and input a fake video based on limited information, such as facial images from official documents such as a driver's license or My Number card. [4-2: Information Processing Operation Performed by Information Processing Device 4]

[0058] The flow of information processing operations performed by the information processing device 4 will be described with reference to Fig. 7. Fig. 7 is a flowchart showing the flow of information processing operations performed by the information processing device 4. Note that the fourth embodiment also describes a case where the second image is a moving image including a plurality of frames, and the second image is referred to as a determination target moving image.

[0059] 7 , the reception unit 311 receives an input of a source image as a first image (step S20). The reception unit 311 may receive an input of a facial photograph from an identification document such as a driver's license or a My Number card as the source image. The reception unit 311 receives an input of a video to be determined as a second image (step S30).

[0060] The matching unit 417 matches the facial image of the person (step S40). If the first image is a facial image on an official document such as a driver's license or My Number card, the matching unit 417 may match the person appearing in the first image with the person appearing in the video to be judged. In this case, if the matching between the person appearing in the first image and the person appearing in the video to be judged fails, the information processing operation may be terminated. Alternatively, the matching unit 417 may match the received first image with a registered facial image that has been registered in advance. Alternatively, the matching unit 417 may match the received video to be judged with a registered facial image that has been registered in advance. In other words, the matching unit 417 may match at least one of the person appearing in the first image and the person appearing in the video to be judged.

[0061] Furthermore, since the source image as the first image and the fake video synthesized based on the source image are similar, even if the video to be judged is a fake video, there is a high possibility that the first image will be successfully matched with the video to be judged.

[0062] The spoofing determination unit 418 performs spoofing determination using the determination target video (step S41). In the fourth embodiment, the determination target video may be used for spoofing determination as well as for determining whether or not it is a fake video. For example, the determination target video may be a video showing an action performed by a person in response to an instruction from the information processing device 4. The information processing device 4 may instruct the direction of the face, the direction of the line of sight, and the position of the face. The information processing device 4 may guide the line of sight. The information processing device 4 may instruct a gesture. The spoofing determination unit 418 may perform active liveness determination using the determination target video.

[0063] The detection unit 3121 detects landmarks from each of one or more frames included in the video to be determined (step S31). The synthesis unit 312 generates a composite video based on the source image and the landmarks from each of the one or more frames included in the video to be determined (step S32). The synthesis unit 312 generates a composite image based on a facial photograph as the source image.

[0064] The extraction unit 3131 extracts the difference between a frame included in the determination target moving image and a frame included in the composite moving image corresponding to that frame (step S33). The emphasis unit 3132 emphasizes the difference (step S34).

[0065] The calculation unit 314 calculates an index representing the likelihood that the target video is a fake video based on the video difference (step S35). The determination unit 315 determines whether the target video is a fake video or not based on the index (step S36). The determination unit 315 may determine whether the target video is a fake video or not by comparing the index with a predetermined threshold.

[0066] If the index exceeds a predetermined threshold (step S36: Yes), the determination unit 315 determines that the video to be determined is a fake video (step S37). If the index does not exceed the predetermined threshold (step S36: No), the determination unit 315 determines that the video to be determined is not a fake video (step S38).

[0067] If the determination unit 315 determines that the video to be determined is not a fake video, the authentication unit 419 authenticates the person based on the collation result by the collation unit 417 and the determination result by the impersonation determination unit 418 (step S42). Furthermore, the authentication unit 419 may authenticate the person on the condition that the determination unit 315 determines that the video to be determined is less likely to be a fake image than a predetermined standard, and the impersonation determination unit 418 determines that the person has acted in accordance with instructions. A case where the authentication unit 419 has successfully authenticated the person may be a case where the person's identity has been confirmed. The output unit 416 outputs the authentication result of the person (step S43). [4-3: Technical Effects of the Information Processing Device 4]

[0068] The fourth information processing device 4 of the present disclosure can accurately detect whether an input video to be determined is a fake video or not, and therefore can accurately verify the identity of the person. [5: Supplementary Note]

[0069] The following supplementary notes are further disclosed with respect to the above-described embodiments. [Supplementary Note 1] An information processing device comprising: a receiving means for receiving input of a first image and a second image; a compositing means for compositing a third image based on the first image and the second image; a difference emphasizing means for emphasizing a difference between the second image and the third image; a calculating means for calculating an index representing a likelihood that the second image is a composite image based on the emphasized difference; and a determining means for determining whether the second image is a composite image in accordance with the index. [Supplementary Note 2] The information processing device according to Supplementary Note 1, wherein the difference emphasizing means includes: an extracting means for extracting a difference between the second image and the third image; and an emphasizing means for emphasizing the difference. [Supplementary Note 3] The information processing device according to Supplementary Note 1 or 2, wherein the second image is a moving image including a plurality of frames; the compositing means composites the third image including one or more frames; and the difference emphasizing means emphasizes a difference between a frame included in the second image and a frame included in the third image corresponding to the frame. [Supplementary Note 4] The information processing device according to Supplementary Note 3, wherein the difference emphasis means generates a difference-emphasized video including difference frames in which a difference between a frame included in the second image and a frame corresponding to the frame included in the third image is emphasized, and the calculation means calculates an index indicating a likelihood that the second image is a synthesized image based on the difference-emphasized video. [Supplementary Note 5] The information processing device according to Supplementary Note 3, wherein the synthesis means includes detection means for detecting landmarks from the second image, and synthesizes the third image based on the first image and the landmarks. [Supplementary Note 6] The information processing device according to Supplementary Note 1 or 2, wherein the determination means determines whether the second image is a synthesized image by comparing the index with a predetermined threshold. [Supplementary Note 7] The information processing device according to Supplementary Note 1 or 2, further comprising: a matching means for matching at least one of an object appearing in the first image and an object appearing in the second image; and an authentication means for authenticating the object based on at least one of a determination result by the determination means and a matching result by the matching means.[Supplementary Note 8] An information processing method comprising: receiving an input of a first image and a second image, synthesizing a third image based on the first image and the second image, emphasizing a difference between the second image and the third image, calculating an index representing a likelihood that the second image is a synthesized image based on the emphasized difference, and determining whether or not the second image is a synthesized image based on the index. [Supplementary Note 9] A recording medium having recorded thereon a computer program for causing a computer to execute an information processing method comprising: receiving an input of a first image and a second image, synthesizing a third image based on the first image and the second image, emphasizing a difference between the second image and the third image, calculating an index representing a likelihood that the second image is a synthesized image based on the emphasized difference, and determining whether or not the second image is a synthesized image based on the index.

[0070] Although the present disclosure has been described above with reference to the embodiments, the present disclosure is not limited to the above-described embodiments. Various modifications that can be understood by those skilled in the art can be made to the configuration and details of the present disclosure within the scope of the present disclosure. Furthermore, each embodiment can be combined with other embodiments as appropriate.

[0071] 1, 2, 3, 4 Information processing device 11, 211, 311 Reception unit 12, 212, 312 Synthesis unit 13, 213, 313 Difference emphasis unit 2131, 3131 Extraction unit 2132, 3132 Emphasis unit 214, 314 Calculation unit 215, 315 Determination unit 216, 316, 416 Output unit 3121 Detection unit 417 Matching unit 418 Spoofing determination unit 419 Authentication unit

Claims

1. a receiving means for receiving input of a first image and a second image; a synthesis means for generating a third image based on the first image and the second image; a difference enhancement means for enhancing a difference between the second image and the third image; a calculation means for calculating an index representing a likelihood of the second image being a synthesized image based on the emphasized difference; a determination means for determining whether the second image is a synthesized image or not in accordance with the index; An information processing device comprising:

2. The difference emphasis means an extraction means for extracting a difference between the second image and the third image; and an emphasis means for emphasizing the difference. The information processing device according to claim 1 .

3. the second image is a moving image including a plurality of frames; the synthesizing means generates the third image including one or more frames; The difference emphasizing means emphasizes a difference between a frame included in the second image and a frame included in the third image corresponding to the frame.

3. The information processing device according to claim 1 or 2.

4. the difference emphasizing means generates a difference-emphasized video including difference frames in which differences between frames included in the second image and frames included in the third image corresponding to the frames are emphasized; The calculation means calculates an index representing a likelihood of the second image being a synthesized image based on the difference-enhanced moving image. The information processing device according to claim 3 .

5. the synthesis means includes detection means for detecting landmarks from the second image; generating the third image based on the first image and the landmarks; The information processing device according to claim 3 .

6. The determining means determines whether the second image is a synthesized image by comparing the index with a predetermined threshold value.

3. The information processing device according to claim 1 or 2.

7. a matching means for matching at least one of an object appearing in the first image and an object appearing in the second image; an authentication means for authenticating the target based on at least one of the determination result by the determination means and the matching result by the matching means; The information processing device according to claim 1 or 2, comprising:

8. Accepting input of a first image and a second image; generating a third image based on the first image and the second image; highlighting the difference between the second image and the third image; calculating an index representing a likelihood of the second image being a synthesized image based on the emphasized difference; determining whether the second image is a synthesized image according to the index; A computer-implemented information processing method.

9. On the computer, Accepting input of a first image and a second image; generating a third image based on the first image and the second image; highlighting the difference between the second image and the third image; calculating an index representing a likelihood of the second image being a synthesized image based on the emphasized difference; and determining whether the second image is a synthesized image according to the index. A computer program for executing an information processing method.