Reliability notification system

Abstract

To accurately verify the authenticity of users using deepfakes.SOLUTION: A reliability notification system includes: image acquisition means for acquiring a first image including a first user; generation means for processing the first image using deepfake to generate a second image; transmission means for transmitting the second image from the first user to a second user; calculation means for calculating a degree of reliability indicating the reliability of the second image based on results of a comparison between the first and second images; and notification means for notifying at least one of the first and second users of information regarding the reliability.SELECTED DRAWING: Figure 1

Description

[Technical Field]

[0001] TECHNICAL FIELD This disclosure relates to the technical field of trust notification systems. [Background technology]

[0002] Known systems of this type detect spoofing in remote calls. For example, Patent Document 1 discloses that when the occurrence of a specific situation is detected based on sensing data generated in a remote call made between the same accounts, spoofing is determined based on a match between stored feature information and feature information corresponding to the sensing data. [Prior art documents] [Patent documents]

[0003] [Patent Document 1] Japanese Patent Application Publication No. 2023-094428 Summary of the Invention [Problem to be solved by the invention]

[0004] There has been an increase in cases where a person's own face (e.g., a groomed face) generated by deepfake technology is used as the image displayed during remote calls. In such cases, if an account is hijacked, a technical problem arises in which someone can impersonate the person and make remote calls.

[0005] This disclosure has been made in consideration of the above-mentioned problems, and aims to provide a reliability notification system that can accurately verify whether a user using deepfake is the real person. [Means for solving the problem]

[0006] A reliability notification system according to one embodiment of the present disclosure includes an image acquisition means for acquiring a first image including a first user, a generation means for processing the first image using deepfake to generate a second image, a transmission means for transmitting the second image from the first user to a second user, a calculation means for calculating a reliability indicating the reliability of the second image based on a comparison result between the first image and the second image, and a notification means for notifying at least one of the first user and the second user of information regarding the reliability. [Brief explanation of the drawings]

[0007] [Figure 1] 1 is a block diagram showing a configuration of a reliability notification system according to a first embodiment. [Figure 2] 4 is a flowchart showing the flow of operations of the reliability notification system according to the first embodiment. [Figure 3] FIG. 10 is a block diagram showing the configuration of a reliability notification system according to a second embodiment. [Figure 4] 10 is a flowchart showing the flow of operations of the reliability notification system according to the second embodiment. DETAILED DESCRIPTION OF THE INVENTION

[0008] Hereinafter, an embodiment of a reliability notification system will be described with reference to the drawings.

[0009] First Embodiment The first reliability notification system will be described with reference to FIGS.

[0010] (System Configuration) First, the configuration of the reliability notification system according to the first embodiment will be described with reference to Fig. 1. Fig. 1 is a block diagram showing the configuration of the reliability notification system according to the first embodiment.

[0011] In Fig. 1, a reliability notification system 1 according to the first embodiment is configured to include a first terminal 10 and a second terminal 20. The first terminal 10 is a terminal used by a first user. The second terminal 20 is a terminal used by a second terminal. The first terminal 10 and the second terminal 20 may be terminals such as personal computers, smartphones, tablets, etc.

[0012] The first terminal 10 and the second terminal 20 each include a processor and a memory. A computer program may be stored in the memory. The processor, together with the memory in which the computer program is stored, may execute various processes to be performed by the reliability notification system 1. For example, the processor may execute the computer program to realize within the processor a logical function block for executing the processes to be performed by the reliability notification system 1. The first terminal 10 and the second terminal 20 may each include a camera, a microphone, a display, an input device (e.g., a mouse or a keyboard), a communication device, etc.

[0013] The first terminal 10 and the second terminal 20 are configured to be able to communicate with each other. For example, each of the first terminal 10 and the second terminal 20 may be connected to the Internet and configured to be able to send and receive various data. Furthermore, the first terminal 10 and the second terminal 20 according to this embodiment are configured to be able to perform remote calls (in other words, online calls). The remote calls may be realized by mutually sending and receiving video and audio of a first user using the first terminal 10 and video and audio of a second user using the second terminal 20.

[0014] The first terminal 10 includes, as functional blocks for realizing its functions, an image acquisition unit 101, a generation unit 102, a transmission unit 103, a reliability calculation unit 104, and a notification unit 105. The image acquisition unit 101 is a specific example of "image acquisition means." The generation unit 102 is a specific example of "generation means." The transmission unit 103 is a specific example of "transmission means." The reliability calculation unit 104 is a specific example of "calculation means." The notification unit 105 is a specific example of "notification means." Each of the image acquisition unit 101, the generation unit 102, the transmission unit 103, the reliability calculation unit 104, and the notification unit 105 may be realized by a processor or the like included in the first terminal 10. Note that these functional blocks may also be included in the second terminal 20. That is, each of the first terminal 10 and the second terminal 20 may be configured to include the above-described functional blocks.

[0015] The image acquisition unit 101 is configured to be able to acquire a first image including a first user who uses the first terminal 10. The image acquisition unit 101 may be configured to acquire the first image by taking a picture of the first user with, for example, a camera or the like mounted on the first terminal 10.

[0016] The generation unit 102 is configured to be able to process the first image acquired by the image acquisition unit 101 to generate a second image. Specifically, the generation unit 102 processes the first user included in the first image using deepfake to generate the second image. The deepfake here may replace the face of the first user included in the first image with the face of a registered user that has been registered in advance. The registered user may be, for example, a user registered as a legitimate user of the first terminal 10 (for example, the owner of the first terminal).

[0017] In principle, since it is assumed that the first user is a registered user, there would seem to be no need to use deepfake to process the face (i.e., to go to the trouble of replacing it with one's own face), but there may be cases where the first user does not want their current face to be displayed. For example, if the first user has just woken up or is not wearing makeup, the first user may not want the other party to see their current face. In such cases, if the first user processes their current face to replace their face with one that shows them in a well-groomed state, they can make a call without showing their current appearance to the other party.

[0018] The transmission unit 103 is configured to be able to transmit the second image generated by the generation unit 102 to the second terminal 20. Therefore, what is displayed on the second terminal 20 during a remote call is not the first image (i.e., a photograph of the current first generation user), but the second image (i.e., an image in which the face of the registered user has been replaced by deepfake).

[0019] The reliability calculation unit 104 is configured to be able to calculate a reliability indicating the reliability of the second image. Here, the "reliability of the second image" refers to the reliability that the face included in the second image is the face of the first user. The reliability calculation unit 104 calculates the reliability by comparing the first image before processing using deepfake with the second image after processing using deepfake. More specifically, the reliability calculation unit 104 calculates a higher reliability the higher the match rate between the face of the first user included in the first image and the face included in the second image (i.e., the replaced face of the registered user). In this way, if the first user is not a registered user, the reliability is calculated to be low, making it possible to determine impersonation by a third party from the reliability.

[0020] The notification unit 105 is configured to be able to notify information related to the reliability calculated by the reliability calculation unit 104 (hereinafter referred to as "reliability information" as appropriate). The reliability information may be information indicating the reliability itself (for example, a numerical representation of the reliability). Alternatively, the reliability information may be information indicating a processing result based on the reliability. For example, the notification unit 105 may determine that the first user is the registered user if the reliability is not less than a predetermined value, and may determine that the first user is someone other than the registered user (i.e., an impersonator) if the reliability is less than the predetermined value, and notify information indicating the determination result. In this case, the "predetermined value" may be set in advance as a threshold value for determining impersonation.

[0021] The notifying unit 105 is configured to be able to notify at least one of the first user and the second user of the reliability information. That is, the notifying unit 105 may notify the first user of the reliability information by making a notification on the first terminal 10 side, or may notify the second user of the reliability information by making a notification on the second terminal 20 side. Alternatively, the notifying unit 105 may notify the first user and the second user of the reliability information by making a notification on both the first terminal 10 and the second terminal.

[0022] (Operation flow) Next, the flow of operations of the reliability notification system 1 according to the first embodiment (specifically, operations when notifying reliability information in a remote call) will be described with reference to Fig. 2. Fig. 2 is a flowchart showing the flow of operations of the reliability notification system according to the first embodiment.

[0023] 2, when the operation of the reliability notification system 1 according to the first embodiment starts, the image acquisition unit 101 first acquires a first image including a first user who uses the first terminal 10 (step S11). Then, the generation unit 102 processes the first image acquired by the image acquisition unit 101 using deepfake to generate a second image (step S12). After that, the transmission unit 103 transmits the second image processed by the generation unit 102 to the second terminal 20 (step S13).

[0024] Next, the reliability calculation unit 104 compares the first image with the second image and calculates the reliability of the second image (step S104). Then, the notification unit 105 determines whether the reliability calculated by the reliability calculation unit 104 is less than a predetermined value (step S15).

[0025] If the reliability is not less than the predetermined value (step S15: NO), the notification unit 105 determines that the first user is the registered user and notifies the user of that fact (step S16). However, since there may be no particular inconvenience if the first user is the registered user, the notification unit 105 may omit notifying the user that the first user is the registered user.

[0026] On the other hand, if the reliability is less than the predetermined value (step S15: YES), the notification unit 105 determines that the first user is not the registered user (i.e., is an impersonator) and notifies the user of this (step S17). The notification unit 105 may issue an alarm or the like when it determines that the first user is an impersonator.

[0027] Thereafter, when the remote call ends (step S18: YES), the reliability notification system 1 ends the series of operations. On the other hand, if the remote call has not ended (step S18: NO), the reliability notification system 1 repeats the process from step S11. However, the processes of steps S15 to S17 (the processes of calculating and notifying the reliability) do not have to be performed every time a second image is generated. For example, the processes of steps S15 to S17 may be performed at predetermined intervals during the remote call.

[0028] (Technical Effects) Next, the technical effects obtained by the reliability notification system 1 according to the first embodiment will be described.

[0029] As described in FIGS. 1 and 2, in the trustworthiness notification system 1 according to the first embodiment, when a remote call is made using an image processed by deepfake, trustworthiness information regarding the processed image is notified. In this way, even if the image displayed during a remote call is processed, it is possible to accurately confirm whether the other party is the registered user. For example, if a registered user's account is hijacked, a third party may impersonate the registered user and make fraudulent remote calls. However, according to the trustworthiness notification system 1 according to this embodiment, trustworthiness information is notified as described above, making it possible to easily identify the occurrence of impersonation.

[0030] Typically, it is sufficient for the reliability information to be notified to the party receiving the second image (i.e., the second user using the second terminal 20). However, if the reliability information is notified to the party transmitting the second image (i.e., the first user using the first terminal 10), it is possible to notify, for example, a first user who is engaging in fraudulent impersonation that the system recognizes the fraud. This makes it possible to prevent the continuation of remote calls due to fraud.

[0031] Second Embodiment A reliability notification system according to the second embodiment will be described with reference to Figures 3 and 4. The second embodiment differs from the first embodiment in some configurations and operations, but other parts may be the same as the first embodiment. Therefore, the following will describe in detail the parts that differ from the first embodiment, and will omit a description of other overlapping parts as appropriate.

[0032] (System Configuration) First, the configuration of a reliability notification system according to the second embodiment will be described with reference to Fig. 3. Fig. 3 is a block diagram showing the configuration of a reliability notification system according to the second embodiment. Note that in Fig. 3, the same reference numerals are used to designate elements that are the same as those described in Fig. 1.

[0033] 3, the reliability notification system 2 according to the second embodiment is configured to include a first terminal 10 and a second terminal 20. The first terminal 10 according to the second embodiment includes, as functional blocks for realizing its functions, an image acquisition unit 101, a generation unit 102, a transmission unit 103, a reliability calculation unit 104, a notification unit 105, a biometric information acquisition unit 106, and a biometric information storage unit 107. That is, the first terminal 10 according to the second embodiment is configured to further include the biometric information acquisition unit 106 and the biometric information storage unit 107 in addition to the configuration of the first embodiment described above (see FIG. 1). The biometric information acquisition unit 106 is a specific example of "biometric information acquisition means." The biometric information storage unit 107 is a specific example of "biometric information storage means." The biometric information acquisition unit 106 may be realized by a processor or the like included in the first terminal 10. The biometric information storage means may be realized by a memory or the like included in the first terminal 10. The above-described functional blocks may be provided in the second terminal 20. That is, each of the first terminal 10 and the second terminal 20 may be configured to include the above-described functional blocks.

[0034] The biometric information acquisition unit 106 is configured to be able to acquire biometric information of the first user who uses the first terminal 10. Here, the "biometric information" is information that can identify the first user, and examples include voice and fingerprint. The biometric information acquisition unit 106 may be configured to acquire multiple types of biometric information.

[0035] The biometric information storage unit 107 is configured to be able to store biometric information of a registered user corresponding to the biometric information acquired by the biometric information acquisition unit 106. For example, if the biometric information acquisition unit 106 acquires voice, the biometric information storage unit 107 may store the voice of the registered user. Also, if the biometric information acquisition unit 106 acquires fingerprints, the biometric information storage unit 107 may store the fingerprints of the registered user.

[0036] The reliability calculation unit 104 according to the second embodiment is configured to be able to calculate the reliability based on the comparison result of the biometric information described above, in addition to the comparison result between the first image and the second image. For example, the reliability calculation unit 104 may be configured to compare the biometric information acquired from the first user (i.e., the biometric information acquired by the biometric information acquisition unit 106) with the biometric information of the registered user (i.e., the biometric information stored in the biometric information storage unit 107), and calculate a higher reliability as the matching rate between them increases.

[0037] (Operation flow) Next, the flow of operations of the reliability notification system 2 according to the second embodiment will be described with reference to Fig. 4. Fig. 4 is a flowchart showing the flow of operations of the reliability notification system according to the second embodiment. In Fig. 4, the same processes as those shown in Fig. 2 are denoted by the same reference numerals.

[0038] 4, when the operation of the reliability notification system 2 according to the second embodiment starts, the image acquisition unit 101 first acquires a first image including a first user who uses the first terminal 10 (step S11). Then, the generation unit 102 processes the first image acquired by the image acquisition unit 101 using deepfake to generate a second image (step S12). After that, the transmission unit 103 transmits the second image processed by the generation unit 102 to the second terminal 20 (step S13).

[0039] Next, the biometric information acquisition unit 106 acquires biometric information of the first user (step S21). Note that the process of step S21 may be executed before or after the processes of steps S11 to S13. That is, the biometric information acquisition unit 106 may acquire the biometric information at an earlier timing. For example, the biometric information acquisition unit 106 may acquire the biometric information at the same timing as the image acquisition unit 101 acquires the first image.

[0040] Next, the reliability calculation unit 104 calculates the reliability of the second image based on the comparison result between the first image and the second image and the comparison result of the biometric information (step S21). Then, the notification unit 105 determines whether the reliability calculated by the reliability calculation unit 104 is less than a predetermined value (step S15).

[0041] If the reliability is not less than the predetermined value (step S15: NO), the notification unit 105 determines that the first user is the registered user and notifies the user (step S16). On the other hand, if the reliability is less than the predetermined value (step S15: YES), the notification unit 105 determines that the first user is not the registered user (i.e., is an impersonator) and notifies the user (step S17).

[0042] Thereafter, when the remote call ends (step S18: YES), the reliability notification system 1 ends the series of operations. On the other hand, if the remote call has not ended (step S18: NO), the reliability notification system 1 repeats the process from step S11.

[0043] (Technical Effects) Next, the technical effects obtained by the reliability notification system 2 according to the second embodiment will be described.

[0044] 3 and 4, in the reliability notification system 2 according to the second embodiment, reliability is calculated using the comparison result of biometric information in addition to the comparison result of images. In this way, it is possible to calculate reliability with higher accuracy compared to when only the comparison result of images is used.

[0045] This disclosure is not limited to the above-described embodiments, but may be modified as appropriate within the scope of the claims and the gist or concept of the invention as can be read from the entire specification, and a reliability notification system involving such modifications is also included in the technical scope of the present invention. [Explanation of symbols]

[0046] 1,2 Reliability Notification System 10 Terminal 1 20 Terminal 2 101 Image acquisition unit 102 Generation part 103 Transmitter 104 Reliability calculation unit 105 Notification Department 106 Biometric information acquisition unit 107 Biometric information storage unit

Claims

1. image acquisition means for acquiring a first image including a first user; A generating means for generating a second image by processing the first image using deep fake; a transmitting means for transmitting the second image from the first user to a second user; a calculation means for calculating a reliability indicating the reliability of the second image based on a comparison result between the first image and the second image; a notification means for notifying at least one of the first user and the second user of information relating to the reliability; A reliability notification system comprising:

2. The deep fake replaces the face of the first user included in the first image with the face of a registered user that has been registered in advance, the calculation means calculates the reliability of the second image to be higher as the matching rate between the face included in the first image and the face included in the second image is higher; The reliability notification system of claim 1 .

3. The notification means notifies the user that the first user is impersonating the registered user when the reliability is less than a predetermined value. The reliability notification system according to claim 2 .

4. a biometric information acquiring means for acquiring biometric information of the first user; a biometric information storage means for storing biometric information of the registered user acquired in advance; Further provided with the calculation means calculates the reliability based on a comparison result between the first image and the second image as well as a comparison result between the biometric information of the first user and the biometric information of the registered user. The reliability notification system according to claim 2 or 3.

Images