Information processing system, payment control method, and program

Abstract

This invention provides an information processing system, information processing method, and program that use AI technology to prevent fraudulent payments caused by impersonation or scams during video calls. [Solution] The payment control system 1 comprises a communication unit 11 that communicates with other terminals 20 using video and audio, an imaging unit 12 that captures images of the user of the device, a reception unit 14 that receives payment processing requests during communication, an authentication unit 15 that performs biometric authentication to determine whether the user is a legitimate authorized person based on the captured video, and a control unit 16 that controls the execution of payment processing based on the authentication result. If the video to be transmitted to the other terminal is a substitute video obtained by processing or replacing the captured video, the control unit 16 performs authentication using the unprocessed captured video acquired by the imaging unit 12, and permits the execution of payment processing on the condition that the authentication is successful. It also detects AI-generated tags and digital watermarks contained in the video data of the call recipient, and if it is determined that they were artificially generated, it restricts or warns about payment processing.

Description

Technical Field

[0005] ,

[0006]

[0001] The present invention relates to an information processing system, a settlement control method, and a program.

Background Art

[0002] In recent years, video calls using smartphones, personal computers (PCs), and television receivers (hereinafter referred to as TV receivers) have become widespread and are widely established as a means of communication between remote locations. Online shopping and remittance services using video calls are also increasing, and the need to perform electronic settlements such as purchasing goods and remitting money during a call is on the rise.

[0003] With the increase in transactions via such video calls, damages such as special fraud targeting the elderly (so-called oreore fraud) and unauthorized settlements in which someone else's account is hijacked are becoming more serious. In particular, cases have been reported in which an elderly person is asked to remit money by a person posing as a family member while using the video call function of a TV receiver.

[0004] Due to the rapid progress of recent AI technology, an advanced image synthesis technology called Deepfake has emerged. Using Deepfake technology, it is technically possible to synthesize someone else's face into a video in real time and pose as a specific person during a video call. Under such circumstances, it has become extremely difficult to ensure whether the person appearing on the video call screen is the actual person, that is, the authenticity of the video, based solely on the fact that a person is shown on the screen.

[0005] In addition, with the spread of the metaverse space and virtual communication, the number of cases in which users use an avatar (virtual character) to make a video call for the purpose of protecting their privacy and self-expression is increasing. When using an avatar, only the video of the avatar is displayed to the call partner, so there is an issue of how to confirm whether the operator behind the avatar is truly the actual person, especially at the time of settlement.

[0006] Conventional technologies include identity verification systems using facial recognition. For example, there are technologies that determine whether or not someone is the person they claim to be by comparing registered facial features with facial features extracted from the input video. However, conventional facial recognition systems often lack the ability to distinguish whether the input video itself is a "real person in front of the camera" or a "forged video generated by AI." [Prior art documents] [Patent Documents]

[0007] [Patent Document 1] Japanese Patent Publication No. 2023-094428 [Overview of the project] [Problems that the invention aims to solve]

[0008] Conventional facial recognition systems can determine whether a given facial feature matches registered facial features, but they have difficulty distinguishing whether the input video itself is a recording of a real person in front of the camera or a forged video generated by AI.

[0009] Furthermore, when a user makes a video call using an avatar, the video that the other party sees is the avatar's image. Therefore, conventional facial recognition systems perform authentication processing on the avatar's image, which presents a problem as it prevents authentication based on the user's own biometric information.

[0010] Furthermore, if the other party in a video call uses AI technology to disguise their own image (for example, using deepfake video that combines the face of an acquaintance or family member), it would be difficult for the user to notice the deception, potentially leading to fraud.

[0011] The present invention has been made in view of the above problems, and aims to provide a payment control system, information processing device, payment control method, and program that can prevent impersonation fraud in video calls by reliably verifying the identity of the user even when using an avatar, and by determining whether the video of the person on the other end of the call is forged by AI or the like. [Means for solving the problem]

[0012] To solve the above problems, a payment control system according to one aspect of the present invention comprises: a communication unit that communicates with other terminals via a network using video and audio; an imaging unit that captures images of the user of the device and acquires captured images; a reception unit that receives payment processing requests during communication by the communication unit; an authentication unit that performs biometric authentication to determine whether the user is a legitimate authorized person based on the captured images; and a control unit that controls the execution of the payment processing based on the authentication result by the authentication unit.

[0013] Even if the video transmitted by the communication unit to the other terminal is a substitute video (e.g., an avatar video) obtained by processing or replacing the captured video, the control unit performs the authentication using the unprocessed captured video acquired by the imaging unit, and permits the execution of the payment process on the condition that the authentication is successful.

[0014] Furthermore, in a payment control system according to one aspect of the present invention, the control unit has an authenticity determination unit that determines whether the video data received from the other terminal contains identification information indicating that it was generated by artificial intelligence technology, or whether the optical characteristics of the video data differ from those of a natural image. If the authenticity determination unit determines that the received video data is artificially generated, the control unit restricts the execution of the payment process or outputs a warning.

[0015] Furthermore, a payment control method according to one aspect of the present invention includes a communication step in which a computer communicates with another terminal via a network using video and audio; an imaging step in which a computer captures an image of the user of its device and acquires the captured image; a reception step in which a request for payment processing is received during the communication; an authentication step in which biometric authentication is performed to determine whether the user is a legitimate authorized person based on the captured image; and a control step in which the execution of the payment processing is controlled based on the authentication result.

[0016] Furthermore, a program according to one aspect of the present invention is a program that causes a computer to execute each of the above-mentioned processes. [Effects of the Invention]

[0017] According to the present invention, when making a payment while conducting a video call, even if the user hides their face using an avatar or the like, identity verification can be performed using the unprocessed live video footage captured by the camera. This ensures reliable identity verification even when using an avatar.

[0018] Furthermore, by detecting AI-generated content in video data received from the caller, it is possible to detect impersonation scams using forged videos such as deepfakes and prevent the execution of fraudulent payment transactions.

[0019] Furthermore, by performing authentication and authenticity checks on both the user and the call recipient, a double security check is achieved, effectively preventing fraudulent money transfers and other schemes targeting the elderly through video calls. [Brief explanation of the drawing]

[0020] [Figure 1] This is a block diagram showing the overall configuration of a payment control system according to one embodiment of the present invention. [Figure 2] This is a block diagram showing the functional configuration of an information processing device according to one embodiment of the present invention. [Figure 3] This figure shows the hardware configuration of an information processing device according to one embodiment of the present invention. [Figure 4] It is a flowchart of the settlement authentication process when using an avatar. [Figure 5] It is a flowchart of the authenticity determination process of the received video. [Figure 6] It is a sequence diagram of the settlement process in the two-way authentication mode. [Figure 7] It is a diagram showing an example of screen transition related to settlement control. [Figure 8] It is a diagram showing an example of the data structure used for authenticity determination.

Embodiments for Carrying Out the Invention

[0021] Hereinafter, embodiments of the present invention will be described in detail while referring to the drawings. Note that the following embodiments are examples for explaining the present invention, and the present invention is not intended to be limited to these embodiments. The present invention can be implemented in various forms without departing from the gist thereof. <<1. Overall System Configuration>>

[0022] FIG. 1 is a block diagram showing the overall configuration of a settlement control system 1 according to an embodiment of the present invention. The settlement control system 1 includes a first terminal device 10, a second terminal device 20, a settlement server 30, an authentication server 40, and a network 50 connecting these.

[0023] The first terminal device 10 is a terminal used by a user A who intends to make a settlement (for example, a purchaser of goods). The first terminal device 10 may be, for example, a smartphone, a tablet terminal, a personal computer (PC), a television receiver (TV receiver), or an information processing device (such as a set-top box) connected to a TV receiver.

[0024] The second terminal device 20 is a terminal used by a user B who is a call partner (for example, a seller of goods or a recipient of a remittance). The second terminal device 20 may also be various terminals such as a smartphone, a PC, or a TV receiver.

[0025] The payment server 30 is a server device that performs electronic payment processing. The payment server 30 manages user account information, credit card information, etc., and performs payment processing in response to payment requests from the first terminal device 10.

[0026] The authentication server 40 is a server device that performs processing related to biometric authentication. The authentication server 40 stores the biometric information (facial features, etc.) of the user that has been registered in advance, performs authentication processing based on the captured video transmitted from the first terminal device 10, and sends the authentication result back to the first terminal device 10. The functions of the authentication server 40 may be integrated into the payment server 30, or they may be built into the first terminal device 10 itself.

[0027] Network 50 includes networks such as the Internet, mobile communication networks, and LANs (Local Area Networks), and mediates data communication between each device. <<2. Hardware Configuration of Information Processing Equipment>>

[0028] Figure 3 shows the hardware configuration of the first terminal device 10. The first terminal device 10 includes a processor 101, memory 102, storage 103, camera 104, microphone 105, speaker 106, display 107, communication interface 108, and input interface 109.

[0029] The processor 101 is a processing unit such as a CPU (Central Processing Unit), GPU (Graphics Processing Unit), or NPU (Neural Processing Unit), and it executes programs stored in the memory 102 to realize the processing of each functional unit described later.

[0030] Memory 102 is volatile memory such as RAM (Random Access Memory) and temporarily stores programs and data executed by the processor 101. Storage 103 is a non-volatile storage device such as flash memory or HDD (Hard Disk Drive) and stores the operating system, application programs, biometric information template data, etc.

[0031] Camera 104 is an imaging device equipped with a CCD (Charge Coupled Device) image sensor or a CMOS (Complementary Metal Oxide Semiconductor) image sensor, etc., and captures video of user A. Camera 104 may be an in-camera built into the terminal device or an external webcam.

[0032] Microphone 105 collects the voice of user A. Speaker 106 outputs the voice of the person on the other end of the call. Display 107 is a liquid crystal display (LCD) or organic light-emitting diode (OLED) display, etc., and displays the video of the person on the other end of the call and various UI (user interface) elements.

[0033] The communication interface 108 is an interface for data communication with other devices via the network 50, and supports communication methods such as wireless LAN (Wi-Fi), cellular communication (4G, 5G, etc.), and wired LAN.

[0034] The input interface 109 is an interface that accepts operation input from the user and includes a touch panel, physical buttons, a remote controller (remote control) receiver, etc. If the first terminal device 10 is a television receiver or an information processing device connected to a television receiver, the input interface 109 includes a remote control receiver that receives infrared signals or Bluetooth® signals from a remote control. <<3. Functional Configuration of Information Processing Devices>>

[0035] Figure 2 is a block diagram showing the functional configuration of the first terminal device 10. Functionally, the first terminal device 10 comprises a communication unit 11, an imaging unit 12, an image processing unit 13, a reception unit 14, an authentication unit 15, a control unit 16, a display unit 17, and a storage unit 18. These functional units are realized by the processor 101 executing a program stored in the memory 102.

[0036] The communication unit 11 transmits and receives data to and from the second terminal device 20, the payment server 30, and the authentication server 40 via the network 50 through the communication interface 108. The communication unit 11 enables real-time two-way communication (video call) with the second terminal device 20 using video and audio. As a communication protocol, for example, WebRTC (Web Real-Time Communication) and SIP (Session Initiation Protocol) can be used.

[0037] The imaging unit 12 controls the camera 104 to image user A and acquires captured video (video frame data). The imaging unit 12 supplies the acquired captured video to the video processing unit 13 and the authentication unit 15. An important point here is that the captured video acquired by the imaging unit 12 is raw video data (raw video data) acquired directly from the camera 104 before processing.

[0038] The video processing unit 13 processes the captured video supplied from the imaging unit 12 to generate a video to be transmitted to the second terminal device 20 via the communication unit 11. The processing performed by the video processing unit 13 includes, for example, replacement with an avatar video (a process that generates avatar movements in real time based on the facial movements and expressions of user A), background blurring, and application of a skin-beautifying filter. If no processing is performed by the video processing unit 13 (i.e., user A has selected the setting to transmit the camera video as is), the captured video is transmitted directly to the second terminal device 20 via the communication unit 11.

[0039] The reception unit 14 receives payment processing requests based on user A's input. The reception unit 14 monitors input signals from the input interface 109 (touch panel, remote control, etc.) and, upon detecting an input indicating the start of payment processing (for example, pressing the "Purchase" button on the screen, pressing the OK button on the remote control, etc.), notifies the control unit 16 of the payment processing request. The reception unit 14 can also receive payment requests from the second terminal device 20 via the communication unit 11 (for example, transmission of billing information from the seller).

[0040] The authentication unit 15 performs biometric authentication to determine whether user A is a legitimate authorized person registered in advance, based on the captured video (raw video data before processing) supplied from the imaging unit 12. The authentication unit 15 performs authentication processing using biometric authentication technologies such as facial recognition, iris recognition, or voice recognition.

[0041] In the case of facial recognition, the authentication unit 15 detects the face region of user A from the captured video and extracts facial features (for example, the relative relationship of the positions of facial landmarks, feature vectors extracted by a deep learning model, etc.) from the facial region. The authentication unit 15 compares the extracted facial features with registered facial features (templates) stored in advance in the storage unit 18, and determines that authentication is successful if the similarity is above a predetermined threshold. The authentication process may also be delegated to the authentication server 40.

[0042] A particularly important point here is that the authentication unit 15 performs authentication using the captured video (i.e., raw video data acquired directly from the camera 104) before processing by the video processing unit 13. This makes it possible to perform authentication based on the actual face image of user A, rather than the avatar image, even when user A is using an avatar for video calls.

[0043] The control unit 16 is a core functional unit that controls the execution of the payment process based on the authentication result from the authentication unit 15. The control unit 16 permits the execution of the payment process when the following conditions are met. (a) The biometric authentication of user A by the authentication unit 15 is successful. (b) The determination result by the authenticity determination unit 161, described later, satisfies the predetermined conditions (if the authenticity determination unit 161 is provided).

[0044] If the control unit 16 authorizes the execution of the payment process, it sends a payment execution request to the payment server 30. If the execution of the payment process is not authorized, the control unit 16 notifies user A via the display unit 17 that the payment has been rejected and displays a warning message if necessary. <<4. Configuration and Operation of the Authenticity Determination Unit>>

[0045] The control unit 16 may further include an authenticity determination unit 161. The authenticity determination unit 161 is a functional unit that determines whether or not video data (i.e., video of the person on the other end of the call) received from the second terminal device 20 via the communication unit 11 is artificially generated.

[0046] The authenticity determination unit 161 determines the authenticity of the received video data by, for example, one or more of the following methods. (1) Detection of AI-generated tags

[0047] Recent image generation AIs (for example, AI based on Generative Adversarial Networks (GANs), diffusion models, etc.) are increasingly adopting specifications that attach identification information (tag information) to the generated images to indicate that they are AI-generated. The authenticity determination unit 161 analyzes the metadata and attribute information contained in the received video data and detects whether or not tag information indicating that it is an AI-generated image is included. (2) Detection of digital watermark information

[0048] The authenticity determination unit 161 detects invisible digital watermark information embedded in the received video data. Some AI image generation tools have the function of embedding minute watermark information that is invisible to the human eye into the generated images. The authenticity determination unit 161 attempts to extract the digital watermark information from the received video data based on known digital watermark algorithms (for example, methods for embedding information in the frequency domain). If digital watermark information is detected, it is determined that there is a high possibility that the video data was artificially generated. (3) Analysis of optical properties

[0049] The authenticity determination unit 161 determines whether the optical characteristics of the received video data differ from those of a natural image (video captured by a real camera). Specifically, the authenticity determination unit 161 can perform the following analysis for each frame of the video data. • Detection of unnatural artifacts (flickering, blurring, etc.) at the boundary between the face and background areas. • Detection of unnatural blinking frequency and patterns • Detection that the noise pattern in the video does not match the noise characteristics inherent to the camera sensor. • Detection of inconsistencies in light reflection and shadows • Checking the consistency of the 3D structure of the face (consistency of head movement and lighting across multiple frames)

[0050] The authenticity determination unit 161 may use only one of the above determination methods, or it may use a combination of multiple determination methods. When multiple determination methods are combined, the determination results from each determination method are comprehensively evaluated (for example, if a predetermined number or more of the multiple methods determine that the result is "artificially generated", the final determination result is set to "false", or weighted scoring is performed on each determination method, etc.), and the final authenticity determination result is output.

[0051] If the authenticity determination unit 161 determines that the received video data is artificially generated, the control unit 16 performs one or more of the following processes. (a) Restrict the execution of payment processing (temporarily block payment) (b) Display a warning message to user A (for example, display a warning on display 107 that says, "Caution: The other party's video may be generated by AI"). (c) Requesting additional verification steps to execute the payment process (e.g., requesting two-factor authentication) (d) Forcibly terminate the call (e) Log any potential misuse and notify the administrator or security team. <<5. Payment Authentication Process When Using an Avatar>>

[0052] Figure 4 is a flowchart of the payment authentication process when user A is making a video call using an avatar.

[0053] In step S101, the communication unit 11 establishes a video call with the second terminal device 20 and starts two-way video and audio communication. At this point, user A has enabled the avatar function, and the video processing unit 13 converts the captured video acquired by the imaging unit 12 into an avatar video and supplies it to the communication unit 11.

[0054] In step S102, the reception unit 14 receives a payment processing request from user A. For example, a payment processing request is generated when user A presses the OK button on the remote control or taps the "Purchase" button on the screen.

[0055] In step S103, the control unit 16 instructs the imaging unit 12 to acquire raw video data from the camera 104 before processing. Here, the imaging unit 12 supplies the acquired video data directly from the camera 104 to the authentication unit 15, without going through the video processing unit 13.

[0056] In step S104, the authentication unit 15 performs biometric authentication of user A using the unprocessed captured image supplied from the imaging unit 12. The authentication unit 15 extracts user A's facial features from the captured image and compares them with a registered facial feature template.

[0057] In step S105, the control unit 16 obtains the authentication result from the authentication unit 15 and determines whether authentication was successful or not. If authentication is successful (step S105: Yes), the process proceeds to step S106. If authentication fails (step S105: No), the process proceeds to step S107.

[0058] In step S106, the control unit 16 authorizes the execution of the settlement process and sends a settlement execution request to the settlement server 30. The settlement server 30 executes the settlement process and returns the processing result to the first terminal device 10.

[0059] In step S107, the control unit 16 refuses to execute the payment process and notifies user A of the authentication failure via the display unit 17. The control unit 16 may allow authentication retries up to a predetermined number of times.

[0060] In the above flow, the transmission of avatar video continues even during the authentication process. In other words, user A's privacy remains protected while identity verification is performed in the background. Only the avatar video is displayed to the person on the other end of the call; the unprocessed live-action video is not sent to the other party. <<6. Authenticity Determination Process for Received Video>>

[0061] Figure 5 is a flowchart of the process for determining the authenticity of received video. This process is executed after the payment request is received (or continuously from the start of the call).

[0062] In step S201, the authenticity determination unit 161 acquires video data received from the second terminal device 20 via the communication unit 11.

[0063] In step S202, the authenticity determination unit 161 analyzes the metadata of the received video data and determines whether or not it contains tag information indicating that it is an AI-generated product. If tag information is detected (step S202: Yes), the process proceeds to step S206.

[0064] In step S203, the authenticity determination unit 161 performs a process to extract digital watermark information from the received video data. If digital watermark information is detected (step S203: Yes), the process proceeds to step S206.

[0065] In step S204, the authenticity determination unit 161 analyzes the optical characteristics of the received video data. If it is determined that the optical characteristics are significantly different from those of a natural image (step S204: Yes), the process proceeds to step S206.

[0066] In step S205, if no artificially generated signs are detected in any of the above determinations, the authenticity determination unit 161 determines the received video data to be "authentic" and notifies the control unit 16 of the determination result in a manner that permits the payment process.

[0067] In step S206, the authenticity determination unit 161 determines that there is a high probability that the received video data is artificially generated. Based on this determination result, the control unit 16 executes processes such as restricting payment processing, displaying a warning message, and requesting additional authentication. <<7. Two-way authentication mode>>

[0068] The payment control system 1 of this embodiment may further include a bidirectional authentication mode. In the bidirectional authentication mode, the execution of the payment process is permitted only during the period when both user A and user B are visible in the imaging unit of their respective terminal devices and both have successfully undergone biometric authentication.

[0069] Figure 6 is a sequence diagram of the payment process in bidirectional authentication mode. The first terminal device 10 performs biometric authentication for user A and requests the second terminal device 20 to perform biometric authentication for user B. The second terminal device 20 performs biometric authentication for user B and sends the authentication result back to the first terminal device 10.

[0070] The control unit 16 permits the execution of the payment process only if both the authentication result of user A and the authentication result of user B are "authentication successful". This ensures that the transaction is conducted after the identity of both the seller and the buyer has been verified.

[0071] Furthermore, the control unit 16 can continuously monitor whether both user A and user B remain visible in the imaging area of ​​their respective terminals while the payment process is being executed (for example, from the transmission of the payment request to the reception of the payment completion). If either user moves out of the imaging area's field of view (for example, leaves their seat), the control unit 16 can temporarily suspend or interrupt the ongoing payment process. <<8. Embodiment using a television receiver>>

[0072] The payment control system 1 of this embodiment can also be implemented in a form that includes a television receiver or an information processing device (set-top box, game console, etc.) connected to a television receiver.

[0073] In this case, a camera (built-in or external) is connected to the television receiver and functions as an imaging unit 12. The reception unit 14 receives payment processing requests based on input signals from a remote controller (remote control) that operates the television receiver. For example, when user A presses the OK button on the remote control on the payment screen displayed on the television receiver's display, a payment processing request is generated.

[0074] In particular, in cases where elderly people make video calls via television sets, the payment control system 1 of this embodiment greatly contributes to preventing special frauds (such as "ore-ore" fraud) targeting the elderly. For example, even if a person claiming to be a grandchild asks an elderly person to send money via video call using deepfake technology, the authenticity determination unit 161 can detect that the video of the person on the other end of the call is an AI-generated image and block or warn about the payment process. <<9. Data Structure>>

[0075] Figure 8 shows an example of a data structure used for authenticity determination. The storage unit 18 stores the following data.

[0076] (1) Biometric Information Template Table: This table stores the user ID, registered biometric information (such as facial feature vectors), and registration date and time in association with each other.

[0077] (2) AI-generated tag definition table: This table stores the patterns of tag information (metadata key name, value format, etc.) assigned by known AI image generation tools. This table is updated as new AI generation tools emerge.

[0078] (3) Digital watermark pattern table: This is a table that stores detection parameters corresponding to known digital watermark algorithms.

[0079] (4) Judgment log table: This table stores the date and time the authenticity judgment was performed, the call session ID to be judged, the judgment method, the judgment result (authentic / possibly forged), and the action taken by the control unit 16 (payment permission, payment block, warning display, etc.). <<10. Example of screen transitions>>

[0080] Figure 7 shows an example of screen transitions related to payment control.

[0081] Screen G1 is the normal screen during a video call. The majority of the screen displays the video of the other party (User B), and User A's own preview video (or avatar if an avatar is used) is displayed in the corner of the screen. Control buttons (microphone on / off, camera on / off, end call, payment button, etc.) are located at the bottom of the screen.

[0082] Screen G2 is the payment confirmation screen displayed after user A presses the payment button. It includes the payment amount, payment destination information, and the message "Authentication in progress...". While this screen is displayed, biometric authentication by the authentication unit 15 and authenticity determination by the authenticity determination unit 161 are performed in the background.

[0083] Screen G3 is the payment completion screen displayed when authentication is successful and there are no issues with the authenticity check. It displays the message "Payment completed" along with a summary of the transaction.

[0084] Screen G4 is a warning screen displayed when the authenticity determination unit 161 detects that the video of the person on the other end of the call may be AI-generated. A warning message, "Caution: The other party's video may be AI-generated. We recommend that you cancel the payment," is displayed in a large font size and a conspicuous color (e.g., red). User A can choose to "cancel the payment" or "continue anyway." <<11. Variant>>

[0085] In the above embodiment, an example was described in which the authentication unit 15 performs facial recognition, but the biometric authentication performed by the authentication unit 15 is not limited to this. For example, iris recognition, fingerprint recognition (if the terminal is equipped with a fingerprint sensor), voiceprint recognition (authentication based on voice acquired from the microphone 105), or a combination thereof (multimodal biometric authentication) may be used.

[0086] In the above embodiment, an example was described in which the authenticity determination unit 161 is built into the first terminal device 10. However, the functions of the authenticity determination unit 161 may be performed on a cloud server (e.g., an authentication server 40). In this case, the first terminal device 10 transmits the received video data to the cloud server and receives the determination result from the cloud server.

[0087] Although the above embodiment describes an example of payment processing, the present invention is not limited to payment processing and can also be applied to various important operations performed during a video call (e.g., electronic signing of a contract, permission to share confidential information, changes to account settings, etc.).

[0088] In the above embodiment, an example was described in which the video processing unit 13 replaces the image with an avatar image. However, the processing performed by the video processing unit 13 is not limited to replacement with an avatar, but may include various image processing processes such as background blurring, skin smoothing filters, and lighting correction. In any case, the authentication unit 15 performs authentication using the raw video data before processing.

[0089] In the above embodiment, a video call between a first terminal device 10 and a second terminal device 20 was described as an example, but the present invention is also applicable to group video calls involving three or more participants. In the case of a group video call, the control unit 16 can control the execution of the payment process based on the authentication results and authenticity determination results of all participants.

[0090] Furthermore, the processing of each functional unit in the above embodiment does not necessarily have to be performed in real time. For example, the determination process by the authenticity determination unit 161 may be performed once at the start of a call, and then periodically at predetermined intervals (e.g., every minute). Alternatively, it may be performed only when a payment processing request is made.

[0091] Although the functions of the payment control system 1 described in the above embodiment were described as being realized by the processor executing a program, some or all of these functions may be realized by dedicated hardware circuits (ASIC: Application Specific Integrated Circuit, FPGA: Field Programmable Gate Array, etc.).

[0092] In the embodiments described above, each functional unit realized by the processor 101 executing a program stored in the memory 102 can also be provided by recording the program on a computer-readable recording medium, or by providing the program via a network. [Explanation of Symbols]

[0093] 1. Payment control system 10. First terminal device (information processing device) 20 Second terminal device 30 Payment Server 40 Authentication Server 50 Networks 101 Processors 102 memory 103 Storage 104 Camera 105 Microphone 106 speakers 107 displays 108 Communication Interfaces 109 Input Interfaces 11 Communications Department 12 Imaging Unit 13. Video Processing Department 14 Reception Department 15. Authentication Department 16 Control Unit 161 Authenticity Determination Department 17 Display 18 Memory section

Claims

1. A communication unit that communicates with other terminals via a network using video and audio, An imaging unit that captures images of the user of the device and acquires the captured image, A receiving unit that receives payment processing requests during communication by the aforementioned communication unit, An authentication unit that performs biometric authentication to determine whether the user is a legitimate authorized person based on the captured video, The system includes a control unit that controls the execution of the settlement process based on the authentication result from the authentication unit, Even if the video transmitted by the communication unit to the other terminal is a substitute video obtained by processing or replacing the captured video, the control unit performs the authentication using the unprocessed captured video acquired by the imaging unit, and permits the execution of the payment process on the condition that the authentication is successful. The system includes an authenticity determination unit that determines whether the video data received from the aforementioned other terminal contains identification information indicating that it was generated by artificial intelligence technology, or whether the optical characteristics of the video data differ from those of a natural image. If the authenticity determination unit determines that the received video data is artificially generated, the execution of the payment process is restricted or a warning is issued. Information processing system.

2. The authenticity determination unit determines whether or not the video data is artificially generated by detecting invisible digital watermarking information or metadata embedded in the received video data. The information processing system according to claim 1.

3. The control unit permits the execution of the payment process only during the period in which both the user and the user of the other terminal are visible in the imaging unit of their respective terminals and biometric authentication of both is successful. The information processing system according to claim 1.

4. The information processing system includes a television receiver or an information processing device connected to the television receiver, The reception unit receives the payment processing request based on the input signal from the remote controller that operates the television receiver. The information processing system according to claim 1.

5. A computer-based information processing method, A communication step that performs video and audio communication with other terminals via a network, The imaging step involves capturing images of the user of the device and acquiring the captured image, A reception step that receives a payment processing request during communication via the aforementioned communication step, An authentication step is performed to determine whether the user is a legitimate authorized person based on the captured video, The set includes a control step that controls the execution of the payment process based on the authentication result from the authentication step, In the control step, even if the video to be transmitted to the other terminal is a substitute video obtained by processing or replacing the captured video, the authentication is performed using the unprocessed captured video acquired in the imaging step, and the execution of the payment process is permitted on the condition that the authentication is successful. Furthermore, the process includes an authenticity determination step that determines whether the video data received from the other terminal contains identification information indicating that it was generated by artificial intelligence technology, or whether the optical characteristics of the video data differ from those of a natural image. Furthermore, the process includes an authenticity determination step that determines whether the video data received from the other terminal contains identification information indicating that it was generated by artificial intelligence technology, or whether the optical characteristics of the video data differ from those of a natural image. If, in the authenticity determination step, it is determined that the received video data was artificially generated, the execution of the payment process is restricted or a warning is issued. Information processing methods.

6. A program for causing a computer to function as an information processing system as described in any one of Claims 1 to 4.

Images