Method for fraud detection in video

Abstract

A method of detecting fraud in video uses a code embedded in a user-side video feed. The code is a color, sequence, or other visual effect displayed on a user's screen and captured by the image sensor or camera via reflections off the user. The server-side video feed is analyzed to determine if the code is present and whether any alterations have been made. Based on the analysis, the video can be validated as authentic. The method can be performed in real-time, allowing seamless communications between users with the added benefit of user authentication.

Description

TITLEMETHOD FOR FRAUD DETECTION IN VIDEOCROSS-REFERENCE TO RELATED APPLICATIONS

[0001] This application claims the benefit under 35 U.S.C. § 119 of U.S. Provisional Application Serial No. 63 / 670,697, filed on July 12. 2024, which is incorporated herein by reference.STATEMENT REGARDING FEDERALLY SPONSORED RESEARCH

[0002] Not applicable.BACKGROUND OF THE INVENTION

[0003] The present disclosure generally relates to fraud detection and anti-fraud countermeasures. More specifically, the disclosure relates to a method of authenticating a video feed to prevent manipulation or alteration, which can be used to perpetrate fraud.

[0004] The growing sophistication of artificial intelligence (Al) driven identity fraud and deepfake videos exploits vulnerabilities in confirming both the authenticity of someone’s identity' and their true intent in real-time, particularly during events occurring via video conferencing. The lack of irrefutable proof of identity and transaction time enables bad actors to impersonate individuals, bypass security measures, and execute high-value scams. Traditional identity’ verification and know-your-customer solutions leave gaps that undermine trust in remote transactions, often requiring manual intervention and increasing operational costs for businesses. Prior methods have attempted to address this type of fraud by analyzing the user presented on the screen. For example, prior methods have attempted to determine whether a digital mask is present or if the user’s movements are natural. However, these prior systems lack the sophistication to detect new Al-driven fraud schemes.

[0005] Bad actors have manipulated video feeds and digital identification systems to commit fraud by: impersonating a company’s CFO in a video call using deepfake technology, using biometric data stolen from a government identification system, presenting a deceptive pre-recorded video as a real-time feed, and committing financial fraud on digital payment applications. Current detection schemes are one step behind the criminals exploiting these vulnerabilities. As a consequence, billions of dollars in damages, both in direct losses and increased security costs, result from this type of fraud every’ year. Further, it is estimated that use of synthetic digital identities will account for 25% of all fraud in the coming years.Therefore, it would be advantageous to develop a system and method for authenticating a person’s identity in the digital realm.BRIEF SUMMARY

[0006] According to embodiments of the present disclosure is a method for fraud detection in digital video and identification. The method provides real-time validation that a video feed being viewed is unaltered from the video captured, allowing a user to have verification that the other user’s identity is authentic. The method can detect anomalies such as identity fraud, deepfake alterations, and spoofing attempts. It protects against rapidly developing Al-driven fraud tactics and unauthorized access. Authentication is accomplished by embedding a code in the video capture process, prior to processing by a server and transmission to the viewing party. The code, comprising a color or sequence, can only be embedded through the real-world environment in which the user is capturing the video. The server-side video feed is compared to the capture stream containing the embedded code to determine if any alterations are present. The method bridges the gap between real-life and the digital realm.BRIEF DESCRIPTION OF THE SEVERAL VIEWS OF THE DRAWINGS

[0007] Fig. 1 is a flowchart of the method according to one embodiment.

[0008] Fig. 2 is a flowchart of the ‘compare videos' sub-process.

[0009] Fig. 3 is a flowchart of the ‘start detection’ sub-process.

[0010] Fig. 4 is a flowchart of the ’fake face detection’ sub-process.

[0011] Fig. 5 is a flowchart of the ‘numbers prediction’ sub-process.DETAILED DESCRIPTION

[0012] According to embodiments of the disclosure is a method 100 of providing authentication in the digital realm. In particular, the method can be used to prevent fraud that occurs when video of a user is maliciously altered. Fig. 1 shows a flowchart of the method 100, according to one embodiment. The method 100 may comprise a plurality of sub-processes or modules, such as comparing videos 201, starting detection 202. fake face detection 203, and numbers prediction 204. The method 100, including sub-processes 201 / 202 / 203 / 204, can be performed on a computer, software, hardware, application specific hardware, or a combination of the foregoing.

[0013] Referring again to Fig. 1, first a video stream is captured in real-time 101. This video can be captured from an image sensor, including the camera on a phone, tablet, orwebcam. This user video stream is also known as the frontend video. As the user video stream is captured, a code is embedded 102 through physical interaction in the user’s real -world environment. This real-world interaction is accomplished by displaying a code, such as a color, color pattern, sequence of colors, or similar visual cues, on the user’s screen or monitor. For example, if the user is capturing the video feed using their phone, the code will be played or displayed on the phone’s screen. The color or sequence will be captured in the user-side video feed due to the reflection off the user’s face, torso, glasses, and other items captured in the frame of the video. A code comprising a sequence, for example, could include a series of flashes, color changes, dimming / brightening of the monitor, or other effects that would be captured in the video feed. In one example embodiment, brightening can be considered a 1 and dimming a 0, thereby forming a binary code. The code is generated through a backend process, such as through a random number generator.

[0014] In most video applications, the user video feed is sent to a server for processing 103, such as compression of audio and video components, transmission, and decompression. As a result, for any captured video, there is a user-side feed (i.e. frontend video) and a server-side feed (i.e. web real-time communication video, or WebRTC video). The method embeds the code on the user video feed in real-life, rather than digitally on the server-side video. This method prevents digital manipulation since the code can only be embedded if a real-life user is present in front of the image sensor. Further, the code is not known by the user and can not be replicated on the server-side video feed unless it was actually captured by the image sensor.

[0015] Once the code is embedded 102 and the user video feed received on the backend or server 103, the server-side video feed (i.e. WebRTC video) is compared to the user side video 104 to determine if the code is present 104 in the web video. Analysis can include neural network frame analysis, comparing frame streams, and identifying variations. Further, the analysis of the server-side video feed 104 can include looking at the features of the user (i.e. facial landmarks) and determining whether the colors are changing as would be expected for the code being embedded on the user-side video. For example, if there were an Al-created mask positioned over the user's real face, the color reflections would not be present. Once the analysis 104 is complete, the video can be authenticated 105. Authentication 105 may be provided based on a min-diagonal analysis or Al decision making to determine if alterations were present. If no alterations are present, then the video can be authenticated 105. These process steps occur in real-time so the user and the intended viewer do not experience lag or delay during communications. The method 100 may include optional steps, such as cleaning up data, checking combinations with CRC data as a backup check, and fake face detection.

[0016] The benefits of the present method 100 are that it is currently harder to fake video in real-time. Deepfake generators (real-time or pre-rendered) typically focus on replicating faces or voices and are not dynamically embedding color-coded data into video streams. If the authentication method 100 expects specific visual signals (e.g., color markers or frame- embedded instructions) that change per session or per frame, it becomes difficult for a bad actor or fraudster to replicate such signals in real-time. Further, the method 100 is resistant to replay attacks. For example, if the embedded code encodes time-sensitive tokens, hashes, or userspecific keys, replaying an old video won’t work unless the attacker somehow knew the token for the exact moment. This feature is useful in preventing re-used or spoofed recordings. Finally, since the codes are displayed via the user’s screen or monitor, the physical image sensor and display environment are part of the authentication, making impersonation from another device more complex.

[0017] Using the method 100, there can not be 100% confidence that the received code is what was sent because it is unknown how the pattern will interact with the user's environment. For example, the lighting conditions, screen size, color of surrounding walls, and similar factors will affect the recorded output in the user’s video feed. As a result, the method 100 uses statistical analysis and confidence thresholds to make a determination if the user video feed is authentic. For example, the authentication method 100 could determine the user video feed as authentic if the analysis returned a 93% likelihood that the video was authentic. For high security applications, the threshold could be set at 98%. for example.

[0018] Security of the method 100 is increased as a result of several features. First, to fake the user video feed, not only would a fraudster have to obtain and fake the code in real-time, but they would also have to fake the environment and determine how- the code w ould displayin that environment. Further, parts of the code can be a decoy. If a fraudster intercepted the code and reproduced the decoy portions of the code, it would be an indication that the user video feed w as corrupted. Finally, there are sequences in the code, such as flashes, that show how the environment reacts to different colors, light levels, and other visual displays, providing a ‘ground truth’ for the code.

[0019] For example, the code could start with a sequence, such as a double flash. Any code transmitted prior to that double flash would be a decoy. Only three colors used in the code or sequence could provide over 68 billion combinations, depending on the number of frames captured. For example, a typical webcam can capture 24 frames per second, meaning a sequence could be transmitted in a very short period of time.

[0020] Referring again to the figures, Figs. 2-5 show various sub-processes of the method 100, according to one example embodiment. Fig. 2 shows the "comparing videos’ 201 subprocess. Here, a set of frames from the user video feed and the WebRTC video feed are compared starting with feature extraction. A set of feature vectors are created and the difference between video feeds is calculated. A statistical analysis, such as determining a minimal diagonal value, is performed. Based on this analysis, the videos are deemed the same or not. While a minimal diagonal value analysis is described, other statistical analyses can be performed. In the example provided, all frame vectors in the user video feed are compared in the server-side video. As such, it is not a comparison of one frame to another, but to the entire sequence of frames. Not only can this analysis determine if the video has been altered, but it can be used to determine if the video was pre-recorded. This comparison module 201 can also account for variation such as noise and video compression.

[0021] Fig. 3 shows the ’start detection’ sub-process 202, which is used to detect the start of the code or sequence embedded in the video feed. This detection compares features of the video, such as brightness and color information, to identify the code in the video feed. Then, the signal of the code is detected using a probability of detection. This process can be accomplished using a neural network or using other probability statistics. Next, the start signal presence is detected with the maximum probability, identifying if there is any sequence or code present.

[0022] Fig. 4 shows the ‘fake face detection’ sub-process 203. From the previous subprocess 202, it is known when the code or sequence should start in the video feed. The ‘fake face detection’ sub-process 203 determines if the brightness changes at start. If it does not, the video is fake or there is a 2D image super-imposed in the video feed. This module 203 can utilize trained models that predict how the brightness should change in response to the start of the code or sequence. This sub-process 203 can be run as a redundant check.

[0023] Fig. 5 shows the 'number prediction’ sub-process 204. This sub-process 204 uses similar logic to the ‘start detection’ module 202 to identity7numbers in the code. Once every number is detected, the sub-process 204 can determine the sequence. There can be a check sum in the code, which improves the probability of detection.

[0024] When used in this specification and claims, the terms "comprises" and "comprising" and variations thereof mean that the specified features, steps, or integers are included. The terms are not to be interpreted to exclude the presence of other features, steps or components.

[0025] The invention may also broadly consist in the parts, elements, steps, examples and / or features referred to or indicated in the specification individually or collectively in anyand all combinations of two or more said parts, elements, steps, examples and / or features. In particular, one or more features in any of the embodiments described herein may be combined with one or more features from any other embodiment(s) described herein.

[0026] Protection may be sought for any features disclosed in any one or more published documents referenced herein in combination with the present disclosure. Although certain example embodiments of the invention have been described, the scope of the appended claims is not intended to be limited solely to these embodiments. The claims are to be construed literally, purposively, and / or to encompass equivalents.

Claims

CLAIMSWhat is claimed is:

1. A method of authenticating a digital video of a user in an environment comprising: captunng a video feed of a user using an image sensor; embedding a code in the video feed, wherein the video is embedded by displaying a code in a real-world environment where the user is located; sending the video feed to a backend processor; determining if the code is present in a web video feed; and authenticating the web video feed if the code is present.

2. The method of claim 1 , wherein the code is a color.

3. The method of claim 1, wherein the code is a sequence.

4. The method of claim 1, wherein displaying a code in a real -world environment comprises: displaying the code on a monitor or screen present near the user.

5. The method of claim 1 , wherein determining if the code is present in a web video feed comprises: identifying facial landmarks of the user and determining if color changes are present based on the code embedded in the video feed.

6. The method of claim 1 performed in real-time.

7. The method of claim 1, wherein the code comprises a time-sensitive token, hash, or userspecific key.

8. The method of claim 1, further comprising: identifying a change in brightness and color in the web video feed; and determining if a start time of the change coincides with a beginning of the code.

9. The method of claim 1, wherein the code contains a decoy sequence.

10. The method of claim 1, wherein determining if the code is present in a web video feed comprises: comparing a sequence of frames in the video feed to a sequence of frames in the web video feed.

11. The method of claim 10, further comprising: extracting a set of feature vectors from each sequence of frames.

12. The method of claim 1, wherein authenticating the web video feed if the code is present further comprises: performing a statistical analysis to determine a probability that the code is present in the web video feed.

13. The method of claim 13, wherein authenticating the web video feed if the code is present occurs if the probability exceeds a threshold value.

14. The method of claim 1, further comprising: performing a fake face detection process by determining if a brightness of the web video feed changes at a beginning of the code.

15. The method of claim 1, further comprising: identifying numbers in the code; and determining a sequence based on the numbers.

16. The method of claim 15, further comprising: performing a check sum operation on the sequence.

Images