Multiple fraud type detection system and methods

Abstract

A system and method for multiple fraud type detection includes anti -injection attack system that has a layered architectural approach that uses a includes combination of different specific models to detect the attacks in combination with image processing techniques, device signals and liveness checks to detect the variety of different types of fraud attacks or repeat fraud attacks. The anti -injection attack system applies the analysis of the tools used to create deepfake, face morph and face swap attacks to define the elements of its layered architecture that can these various types of attacks.

Description

Multiple Fraud Type Detection System and MethodsBACKGROUND

[0001] The present disclosure relates to fraud detection and prevention. More specifically, the present disclosure relates to capturing image data representing a face and / or a document and capturing other information, and detecting fraud based on the captured information.

[0002] Facial recognition and comparison are one way of identifying a person and verifying a person’s identity. For example, providing a picture ID may be required to open a financial account to reduce the risk of fraud and / or to comply with laws (e.g., anti-money laundering or sanctions). As another example, an image (e.g., a selfie or video) of the user may be provided to accompany the provided picture identification (ID) and be used for a comparison, e.g., to prove the person providing the document is in fact the document holder.

[0003] Deepfake, face morph, and face swap attacks continue to grow at a rapid rate. Some reports have indicated that 20% of successful account takeover attacks this year leveraged deepfake technology. These types of attacks have increased because there are a large number of open-source and commercial tools available to create deepfake, face morph, and face swap images. For example, there are over 50 different tools for creating deepfakes. These tools are extremely easy to use, free or low cost, and often provide excellent results that are incredibly difficult for humans to spot.

[0004] Another problem with the prior art is that discovering these injection attacks is also difficult for machine learning (ML) models to detect since there are many distinct types of ML and computer vision techniques used to create them. For the detecting models to work, they have to have been trained on examples of these attacks created by the numerous iapproaches available, which is extremely challenging. The challenge is in either obtaining or creating the necessary amounts of training data needed to train effective models.SUMMARY

[0005] The techniques introduced herein overcome the deficiencies and limitations of the prior art, at least in part, with a system and method for multiple fraud type detection.

[0006] According to one aspect of the subject matter described in this disclosure, a computer-implemented method includes receiving a first image data associated with a user; performing a first detection type on the first image data to generate a first signal; performing a second detection type on the first image data to generate a second signal; generating an aggregated fraud score based upon the first signal and the second signal; and accepting the first image data as genuine based upon the aggregated fraud score.

[0007] In general, another aspect of the subject matter described in this disclosure includes a system comprising one or more processors and memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to the execution of the instructions by one or more processors, cause the one or more processors to: receive a first image data associated with a user; perform a first detection type on the first image data to generate a first signal; perform a second detection type on the first image data to generate a second signal; generate an aggregated fraud score based upon the first signal and the second signal; and accept the first image data as genuine based upon the aggregated fraud score.

[0008] Other implementations of one or more of these aspects include corresponding systems, apparatus, and computer programs, configured to perform the actions of the methods, encoded on computer storage devices.

[0009] These and other implementations may each optionally include one or more of the following features. For instance, features may also include the first detection type or the second detection type is one from a group of: a deepfake model, a face morph model, a face swap model, an unknown injection attack anomaly model, a subject and scene segmentation analyzer, an injection checker, a device risk checker, a liveness detector, a face match and face analysis subsystem, an injection attack detector, and a device risk detector. For example, in some implementations, the first detection type is a multi-frame deepfake detection model, and the second detection type is a single frame deepfake detection model, and the method further comprises performing a third detection type on the first image data to generate a third signal, wherein the third detection type is a subject and scene segmentation analysis; and wherein the aggregated fraud score is generated based upon the first signal, the second signal, and the third signal. For instance, the method further comprises performing the third detection type on the first image data further including generating a vector embedding for the first image data; accessing a matrix of vector embeddings; searching for cosine similarities between embeddings in the matrix and the generated vector embedding; and generating the third signal based on the searching for the cosine similarities. For example, the method may also include wherein the first image data is one from a group of a selfie and a video. In general, other aspects of the subject matter of this disclosure may be implemented in methods where the first image data includes a selfie image, and the method further comprises receiving a document image; generating selfie metadata from the selfie image; and generating document metadata from the document image. For example, features may also include wherein the first image data includes a selfie image and a document image, and the method further comprises performing a third detection type using the selfie image and the document image to generate a third signal, wherein the third detection type is a face match and face analysis that compares the selfie image to a holder portion of the document image andgenerates the third signal based on a match between the selfie image and the holder portion of the document image. For instance, the method may also include wherein the accepting the first image data as genuine includes comparing the aggregated fraud score to a threshold, and accepting the first image data if the aggregated fraud score satisfies the threshold. Still other implementations include performing a third detection type on the first image data to generate a third signal; and wherein the first image data is rejected as genuine based upon only the third signal.

[0010] The features and advantages described herein are not all-inclusive and many additional features and advantages will be apparent in view of the figures and description. Moreover, it should be noted that the language used in the specification has been principally selected for readability and instructional purposes and not to limit the scope of the techniques described.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The disclosure is illustrated by way of example, and not by way of limitation in the figures of the accompanying drawings in which like reference numerals are used to refer to similar elements.

[0012] Figure l is a block diagram of one example implementation of a system for multiple fraud type detection in accordance with some implementations.

[0013] Figure 2 is a block diagram of an example computing device including a multiple fraud type detector in accordance with some implementations.

[0014] Figure 3 is a block diagram of an example multiple fraud type detector in accordance with some implementations.

[0015] Figure 4A is an illustration of a first example anti-injection attack system in accordance with some implementations.

[0016] Figure 4B is an illustration of a second example anti -injection attack system in accordance with some implementations.

[0017] Figure 5 is an illustration of an example subject and scene segmentation analyzer in accordance with some implementations.

[0018] Figure 6 is a flowchart of an example method for multiple fraud type detection in accordance with some implementations.

[0019] Figure 7 is a flowchart of a second example method for multiple fraud type detection in accordance with some implementations.

[0020] Figure 8 is a flowchart of a first example method for detecting multiple fraud types in accordance with some implementations.

[0021] Figure 9 is a flowchart of a second example method for detecting multiple fraud types in accordance with some implementations.

[0022] Figure 10 is a flowchart of a third example method for detecting multiple fraud types in accordance with some implementations.

[0023] Figure 11 is a flowchart of a fourth example method for detecting multiple fraud types in accordance with some implementations.

[0024] Figure 12 is a flowchart of an example method for training models to detect multiple fraud types in accordance with some implementations.

[0025] Figure 13 is an illustration of an example set of images in which the facial image (selfie) is Al generated and the Al generated face is reproduced as the document holder image in the document, which may be detected in accordance with some implementations.

[0026] Figure 14 is an illustration of an example set of images in which two images are face morphed to produce a third image.

[0027] Figure 15 is an illustration of an example set of images used to perform face swapping.

[0028] Figure 16 is a diagram illustrating how face morphing and face swapping can be performed.

[0029] Figure 17A is an illustration of an example set of images used to show how partial image search and image segmentation is performed.

[0030] Figure 17B is an illustration of an example set of images having a similar background that can be detected using partial image search and segmentation.

[0031] Figure 18 is a block diagram of an example image-segment-based fraud detector in accordance with some implementations.

[0032] Figure 19 is a block diagram of an example subject sub-segmenter in accordance with some implementations.

[0033] Figure 20 is a block diagram of an example mathematics-based descriptor generator in accordance with some implementations.

[0034] Figure 21 A is an illustration of a first example image associated with a repeat, injection attack.

[0035] Figure 2 IB is an illustration of a second example image associated with a repeat, injection attack.

[0036] Figure 21C is an illustration of a third example image associated with a repeat, injection attack.

[0037] Figure 22 is an illustration of an example subject segment and example subject sub-segments derived from the first example image associated with a repeat fraud attack thereof in accordance with some implementations.

[0038] Figure 23 is an illustration of an example scene sub-segments derived from the first example image associated with a repeat fraud attack in accordance with some implementations.

[0039] Figure 24 is an illustration of a set of subject sub-segments and scene subsegments derived from the first example image associated with a repeat fraud attack in accordance with some implementations.

[0040] Figure 25 is a flowchart of an example method for segmented subject and scene partial image search fraud detection in accordance with some implementations.DETAILED DESCRIPTION

[0041] The present disclosure is described in the context of a multiple fraud type detection and use cases; however, it should be recognized that the multiple types of fraud detected and may vary but the architecture of the present discloser can be applied with new modules added to detect new types of fraud. In particular, the present disclosure shows a particular anti-injection architecture that can have additional fraud types detected for operation in other environments and use cases without departing from the disclosure herein. This disclosure presents a series of methods used individually or together to detect sophisticated multiple types of fraud that is presented to the verification system either directly or injected into the system.

[0042] Facial comparison, e.g., between a physical document with an image of the valid document holder and the person physically presenting the document, is a method of determining an individual’s identity. This manual task has been performed by bank tellers at a bank counter, bouncers at bars, law enforcement at traffic stops, and in countless other physical environments.

[0043] Users with nefarious intent (e.g., criminals, fraudsters, money launderers, etc.) may repeatedly attempt to trick the systems and methods used to verify documentation or identity in remote and electronic environments with much less risk of apprehension and, in some cases, little additional effort for each additional attempt. It is sometimes the case that the more times a fraudster (or other nefarious user) is able to attempt fraud, the more likely the fraudster is to eventually succeed in defeating the verification mechanisms. Therefore, detection of repeated fraudulent attempts may be used in identifying and preventing future, potentially successful, fraudulent attempts. However, criminals including fraudsters are resourceful and may not use identical instances of a document or image of a document. The terms fraudster, nefarious user, criminal are used synonymously throughout this description.

[0044] Advances in technologies have decreased the burden on nefarious users and increased the difficulty of preventing fraud in remote and electronic transactions, particularly at scale. For example, image manipulation software (e.g., Adobe’s Photoshop) has allowed users to manipulate and create different versions of documents or images quickly and easily, such as fake IDs with different images or information in the various fields such as name). The fraudster may print out or electronically submit the various versions of the fraudulent (e.g., doctored) documentation and use the various versions in a series of attempts to successfully commit fraud. For example, referring to Figure 13, an example set of images in which the facial image (selfie) 1302 is Al generated, and the Al generated face 1310 is copied and reproduced as the document holder image 1306 in the document image 1304. Image 1306 is a blowup of a portion of the image 1304 of the ID card showing the facial image in more detail. The images 1302 and 1304 can be detected, by the systems and methods described herein, as fraudulent and rejected in accordance with some implementations.

[0045] In addition, development kits and injectors may allow a fraudster to perform an injection attack. In an injection attack, the nefarious user injects a fake or manipulated facial image into a digital image stream, e.g., a digital image stream associated with document image (or document holder image) and / or selfie in an effort to defeat verification mechanisms, e.g., those verification mechanisms that may be present during a customer onboarding process. The injection may be performed by one or more of using a virtual camera, hacking the verification vendor’s application program interface (API) or software development kit (SDK), or by switching the image payload in transit. The injected image may modify a facial image (e.g., by morphing the facial features to be more similar to those in a document holder image) or replace a facial image (e.g., a face swap in which the document holder’s face overlays the nefarious user’s face). Injection attacks including deepfakes may be generated using a variety of mechanisms, e.g., generative adversarial network-based (“GAN-based”) synthetic faces, diffusion model-based synthetic faces, autoencoder-based methods, etc.

[0046] A nefarious user may also electronically modify the document holder image to match a real selfie (e.g., using photoshop or injection). A nefarious user may electronically modify a real (unseen) selfie with a face swap or face morph to match a real document holder image. For example, a nefarious user used injection to face swap, or overlay, a valid document holder’s face from a valid ID instance over his / her own face in a selfie that is submitted for comparison to the document holder’s image.

[0047] Often a nefarious user who repeatedly attempts to commit fraud does not use completely distinct documents or images of documents across his / her multiple attempts. For example, the fraudster uses an instance of a document, then modifies, the name, the date of birth, and ID number, and so on, but there will be commonalities between the attempts. Examples of commonalities may include, but are not limited to, the document’s surroundingsor background; the facial image; the issuer of the ID; the size, orientation, or position of the document in the image; etc.

[0048] The multiple fraud type detector 226 described herein addresses, at least in part, one or more of the foregoing issues and / or provides, at least in part, one or more of the aforementioned benefits. The multiple fraud type detector 226, in particular its anti-injection attack system 326 has a layered architectural approach that uses and includes combination of different specific models to detect the attacks in combination with image processing techniques, device signals and liveness checks to detect the variety of different types of fraud attacks or repeat fraud attacks. The anti -injection attack system 326 applies the analysis of the tools used to create deepfake, face morph and face swap attacks to define the elements of its layered architecture that can these various types of attacks. As an example, face morph and face swap tools most frequently map the replacement face (or face used for fraud) into the area of the original face and within the edge boundaries of that face. This leaves the neck, ears, forehead, and hair of the original subject intact or only partially modified (e.g., see Figures 14 and 15), and therefore, a characteristic that the multiple fraud type detector 226 uses to detect these fraud variants. In particular, the anti-injection attack system 326 includes a subject & scene segment analyzer 410 for searching and detecting similarities in these portions of an image and an indicator of fraud. The multiple fraud type detector 226 is particularly advantageous because it can detect multiple different types of fraud attacks. By way of example, 4 distinct types of fraud are detected using the techniques described in with reference to Figures 8-11 below. The layered architecture also includes several types of solutions, also detailed below, to detect additional varieties of attacks.

[0049] Figure 1 is a block diagram of an example system 100 for multiple fraud type detection in accordance with some implementations. As depicted, the system 100 includes a server 122 and a client device 106 coupled for electronic communication via a network 102.

[0050] The client device 106 is a computing device that includes a processor, a memory, and network communication capabilities (e.g., a communication unit). The client device 106 is coupled for electronic communication to the network 102 as illustrated by signal line 114. In some implementations, the client device 106 may send and receive data to and from other entities of the system 100 (e.g., a server 122). Examples of client devices 106 may include, but are not limited to, mobile phones (e.g., feature phones, smart phones, etc.), tablets, laptops, desktops, netbooks, portable media players, personal digital assistants, etc. In some implementations, image and data may be provided by the client device 106 (e.g., a mobile phone, tablet, or laptop) and be submitted via an API, the web, or the mobile device.

[0051] Although only a single client device 106 is shown in the example of Figure 1, there may be any number of client devices 106 depending on the implementation. The system 100 depicted in Figure 1 is provided by way of example and the system 100 and further systems contemplated by this present disclosure may include additional and / or fewer components, may combine components and / or divide one or more of the components into additional components, etc. For example, the system 100 may include any number of client devices 106, networks 102, or servers 122. The client device 106 may bet that of a user 112 and the user 112 may take, depicted by line 112, photos or images of the user 112 or documents.

[0052] The network 102 may be a conventional type, wired and / or wireless, and may have numerous different configurations including a star configuration, token ring configuration, or other configurations. For example, the network 102 may include one or more local area networks (LAN), wide area networks (WAN) (e.g., the Internet), personal area networks (PAN), public networks, private networks, virtual networks, virtual private networks, peer-to-peer networks, near field networks (e.g., Bluetooth®, NFC, etc.), cellular(e.g., 4G or 5G), and / or other interconnected data paths across which multiple devices may communicate.

[0053] The server 122 is a computing device that includes a hardware and / or virtual server that includes a processor, a memory, and network communication capabilities (e.g., a communication unit. The server 122 may be communicatively coupled to the network 102, as indicated by signal line 116. In some implementations, the server 122 may send and receive data to and from other entities of the system 100 (e.g., one or more client devices 106).

[0054] Other variations and / or combinations are also possible and contemplated. It should be understood that the system 100 illustrated in Figure 1 is representative of an example system 100 and that a variety of different system environments and configurations are contemplated and are within the scope of the present disclosure. For example, various acts and / or functionality may be moved from a server to a client, or vice versa, data may be consolidated into a single data store or further segmented into additional data stores, and some implementations may include additional or fewer computing devices, services, and / or networks, and may implement various functionality client or server-side. Furthermore, various entities of the system 100 may be integrated into a single computing device or system or divided into additional computing devices or systems, etc.

[0055] For example, as depicted, the client device 106 may optionally (as indicated by the dashed lines) include an instance of the multiple fraud type detector 226b and the server 122 may include an instance of the multiple fraud type detector 226a. However, in some implementations, the components and functionality of the multiple fraud type detector 226 may be entirely client-side (i.e., at 226b), entirely server side (i.e., at 226a), or divide among the client device 106 and server 122 (i.e., divided across 226a and 226b). For example, as described below, some implementations may use machine learning (e.g., one ormore algorithms to train one or more models), and the training and validation of the model(s) may be performed server-side at 226a and applied, during production, client side at 226b. As another example, the components and functionality of the subject and scene segment analyzer 410 may be entirely client-side, entirely server side, or divide among the client device 106 and server 122. For example, as described below, some implementations of the subject and scene segment analyzer 410 may use machine learning (e.g., one or more algorithms to train one or more models), and the training and validation of the model(s) may be performed server-side at and applied, during production, client side.

[0056] Figure 2 is a block diagram of an example computing device 200 including an instance of the multiple fraud type detector 226. The multiple fraud type detector 226 which may refer to either instance of the multiple fraud type detector 226a when the computing device 200 is a server 122, or the multiple fraud type detector 226b where the computing device 200 is a client device 106, or a combination of 226a and 226b where the functionality is divided between multiple fraud type detector 226b of the client device 106 and multiple fraud type detector 226a of the server 122. In the illustrated example, the computing device 200 includes a processor 202, a memory 204, a communication unit 208, a data store 228 and optionally a display 218.

[0057] In some implementations, the computing device 200 is a client device 106, the memory 204 stores the multiple fraud type detector 226b, and the communication unit 208 is communicatively coupled to the network 102 via signal line 114. In some implementations, the client device 106 includes at least one sensor, e.g., a camera (not shown). In another implementation, the computing device 200 is a server 122, the memory 204 stores the multiple fraud type detector 226a, and the communication unit 208 is communicatively coupled to the network 102 via signal line 116.

[0058] The processor 202 may execute software instructions by performing various input / output, logical, and / or mathematical operations. The processor 202 may have various computing architectures to process data signals including, for example, a complex instruction set computer (CISC) architecture, a reduced instruction set computer (RISC) architecture, and / or an architecture implementing a combination of instruction sets. The processor 202 may be physical and / or virtual and may include a single processing unit or a plurality of processing units and / or cores. In some implementations, the processor 202 may be capable of generating and providing electronic display signals to a display device, supporting the display of images, capturing, and transmitting images, and performing complex tasks and determinations. In some implementations, the processor 202 may be coupled to the memory 204 via the bus 206 to access data and instructions therefrom and store data therein. The bus 206 may couple the processor 202 to the other components of the computing device 200 including, for example, the memory 204, the communication unit 208.

[0059] The memory 204 may store and provide access to data for the other components of the computing device 200. The memory 204 may be included in a single computing device or distributed among a plurality of computing devices. In some implementations, the memory 204 may store instructions and / or data that may be executed by the processor 202. The instructions and / or data may include code for performing the techniques described herein. For example, in one implementation, the memory 204 may store an instance of the multiple fraud type detector 226. The memory 204 is also capable of storing other instructions and data, including, for example, an operating system, hardware drivers, web browsers, other software applications, databases, etc. The memory 204 may be coupled to the bus 206 for communication with the processor 202 and the other components of the computing device 200.

[0060] The memory 204 may include one or more non-transitory computer-usable (e.g., readable, writeable) device, a static random access memory (SRAM) device, a dynamic random access memory (DRAM) device, an embedded memory device, a discrete memory device (e.g., a PROM, FPROM, ROM), a hard disk drive, an optical disk drive (CD, DVD, Blu-ray™, etc.) mediums, which can be any tangible apparatus or device that can contain, store, communicate, or transport instructions, data, computer programs, software, code, routines, etc., for processing by or in connection with the processor 202. In some implementations, the memory 204 may include one or more of volatile memory and nonvolatile memory. The memory 204 may be a single device or may include multiple types of devices and configurations.

[0061] The communication unit 208 is hardware for receiving and transmitting data by linking the processor 202 to the network 102 and other processing systems. The communication unit 208 receives data and transmits the data via the network 102. The communication unit 208 is coupled to the bus 206. In one implementation, the communication unit 208 may include a port for direct physical connection to the network 102 or to another communication channel. For example, the computing device 200 may be the server 122, and the communication unit 208 may include an RJ45 port or similar port for wired communication with the network 102. In another implementation, the communication unit 208 may include a wireless transceiver (not shown) for exchanging data with the network 102 or any other communication channel using one or more wireless communication methods, such as IEEE 802.11, IEEE 802.16, Bluetooth® or another suitable wireless communication method.

[0062] In yet another implementation, the communication unit 208 may include a cellular communications transceiver for sending and receiving data over a cellular communications network such as via short messaging service (SMS), multimedia messagingservice (MMS), hypertext transfer protocol (HTTP), direct data connection, WAP, e-mail or another suitable type of electronic communication. In still another implementation, the communication unit 208 may include a wired port and a wireless transceiver. The communication unit 208 also provides other connections to the network 102 for distribution of files and / or media objects using standard network protocols such as TCP / IP, HTTP, HTTPS, and SMTP as will be understood to those skilled in the art.

[0063] The display 218 may include a liquid crystal display (LCD), light emitting diode (LED), touchscreen, or any other similarly equipped display device, screen, or monitor. The display 218 represents any device equipped to display electronic images and data as described herein. The display 218 is optional, and therefore, depicted with dashed lines in Figure 2.

[0064] The data storage 228 is a non-transitory memory that stores data for providing the functionality described herein. In the illustrated implementation, the data storage 228 is communicatively coupled to the bus 206. The data storage 228 stores information that is used to provide functionality as described herein. For example, the data store 228 may store an original video, images, portions of video (e.g., frames), portions of images (e.g., segments and / or subsegments), known instances of fraudulent images or segmented portions of the images, clusters of the images, descriptors associated with an image (or segment, or subsegment thereof), machine learning models, training data, and mathematical representations of images (e.g., vectors) or environment information contained therein, databases of device information, network information user information, and various other information to representations related to video and images.

[0065] It should be understood that other processors, operating systems, input devices (e.g., keyboard, mouse, one or more sensors, etc.), output devices (e.g., a speaker, display, haptic motor, etc.), and physical configurations are possible and within the scope of thedisclosure. Examples of sensors (not shown) include, but are not limited to, a microphone, a speaker, a camera, a thermal camera, a pointer sensor (e.g., a capacitive touchscreen or mouse), a gyroscope, an accelerometer, a galvanic sensor, thermocouple, heart rate monitor, breathing monitor, electroencephalogram (EEG), iris scanner, fingerprint reader, raster scanner, palm print reader, an inertial sensor, global positioning system (GPS) sensor, etc.

[0066] In some implementations, the multiple fraud type detector 226 provides the features and functionalities described below responsive to a request. For example, a request on behalf of an entity (not shown), such as a financial institution, to determine whether a user-provided document image (e.g., provided during a registration or customer onboarding) is legitimate or potentially fraudulent. As another example, a request may be by the user, such as to capture a document image and / or personal image, such as a selfie (e.g., as part of a registration or customer onboarding).Multiple Fraud Type Detector 226

[0067] Referring now to Figure 3, a block diagram of an example of multiple fraud type detector 226 is illustrated in accordance with one implementation. As illustrated in Figure 3, the multiple fraud type detector 226 may include an image receiver 322, a device information receiver 324, an anti -injection attack system 326 and a decision engine 330 including an aggregated fraud score generator 332.

[0068] The image receiver 322 is communicatively coupled to receive image data. The image receiver 322 may be steps, processes, functionalities, software executable by a processor, or a device including routines to receive image data from the client device 106 or the server device 122. For example, in some implementations, the image receiver 322 receives image data captured by a camera sensor. Examples of image data may include, but are not limited to, one or more of an image and a video. In some implementations, a receivedimage represents a document and a background, or surroundings, of that document. For example, the received image data includes an image received responsive to the user 112 being prompted to take an image of the document. For example, document image 1304 in Figure 13 is an example of a document image, which includes a background (e.g., the hand and the photo identification document as the subject). In some implementations, a received image represents a person (e.g., a selfie or photo ID) and a background, or surroundings also referred to as a scene, of that person. For example, the received image data includes an image received responsive to the user 112 being prompted to take a selfie (e.g., a single image or video clip). In some implementations, the image receiver 322 may be coupled to a client device 106 to receive image data via an API channel, a web channel, or a mobile device channel as we described below in more detail with reference to cap Figure 4B.

[0069] The image data may be “real” or “genuine” (i.e., an un-modified and true representation of the subject matter in the image), altered (e.g., using photoshop or an injection attack), or a combination thereof (e.g., a real document holder image but a modified selfie image or vice versa). The image receiver 322 makes the received image data available to one or more components of the multiple fraud type detector 226. In some implementations, the image receiver 322 communicates the received image to, or stores the received image for retrieval by, one or more other components of the multiple fraud type detector 226. More specifically, the image receiver 322 is coupled to the anti-injection attack system 326 to provide images it receives.

[0070] The document type may vary based on the implementation and use case. In some implementations, the document type may include a type associated with identification documentation. Examples of types associated with identification documentation include, but are not limited to a passport, driver’s license, government issued photo identification card, school identification, employee identification, etc. In some implementations the documenttype label may include an issuer of the identification document type. Examples issuers include, but are not limited to a country, state, province, municipality, jurisdiction, school, business, employer, or other entity. For example, a label associated with a US passport image may include a passport, documentation type label component, and a U.S. issuer label component.

[0071] The device information receiver 324 is a communicatively coupled to receive device data and other metadata from the client device 106, the network 102, and the server 122. The device information receiver 324 may be steps, processes, functionalities, software executable by a processor, or a device including routines to receive or retrieve device information and other metadata associated with the device that captured an input image. In some implementations, the device information receiver 324 may also receive other network or server information related to the image being processed for identity detection, identity verification or document verification. The device information receiver may retrieve or receive information related to both images and video. The device information receiver 324 is coupled to the server 122, the network 102 and the client device 106 to receive or retrieve information. The device information receiver 324 is coupled to send information to the antiinjection attack system 326 and the decision engine 330 as will be described in more detail below with reference to Figures 4 A and 4B.

[0072] The anti-injection attack system 326 may be steps, processes, functionalities, software executable by a processor, or a device including routines to detect multiple different types of injection attacks. The anti-injection attack system 326 is coupled to receive information from the image receiver 322 and the device information receiver 324. The antiinjection attack system 326 is coupled to provide its output to the decision engine 330, in particular, the aggregated fraud score generator 332. As noted above, the anti -injection attack system 326 has a layered architectural approach that uses and includes combination ofdifferent specific models to detect the attacks in combination with image processing techniques, device signals and liveness checks to detect the variety of different types of fraud attacks or repeat fraud attacks. One particular advantage of the anti-injection attack system 326 is that it applies multiple models to the input video or image. Additionally, the architecture is scalable and adaptable to add additional models to detect new types of fraud. The functionality of the anti-injection attack system 326 and multiple implementations will be described in more detail below with reference to Figures 4A and 4B.

[0073] The anti-injection attack system 326 determines whether a received image (e.g., a selfie or a document image) or video is a product of an injection attack. In some implementations, the anti-injection attack system 326 applies a machine learning model to detect any subtle differences (e.g., artifacts) between genuine and fraudulent (e.g., generated using an injection attack or other digital manipulation) images. These differences are often imperceptible to the human eye. In some implementations, the anti-injection attack system 326 may be used to validate one or more injection detection ML models that the antiinjection attack system 326 applies. The anti -injection attack system 326 may also be used to identify attacks that could subsequently be used to train or re-train a model. For example, if it was determined that an attack was injected and was a deepfake that the deepfake model 402 missed (failed to detect), then that image could be added to the training set of images to improve the deepfake model 402 after retraining. An example of this training is described in more detail below with reference to the method of Figure 12.

[0074] The decision engine 330 generates a signal indicating whether the image(s) are associated with any fraud type (e.g., injection based) attack. In some implementations, the decision engine 330 or supplemental systems (not shown) determines and / or initiates an action based on the signal generated by the decision engine 330. The decision engine 330 can be used to perform identity verification or ID verification. Examples of actions include,but are not limited to one or more of accept or reject the submission, contact authorities, escalate for investigation or for additional verification checks, etc. The decision engine 330 may be steps, processes, functionalities, software executable by a processor, or a device including routines to generates a signal indicating whether the image(s) are associated with any one of multiple fraud types and whether a particular action should be performed in response to the signal. The decision engine 330 can output this fraud signal along with other information (e.g., “reason codes” that indicate the reason for the rejection and type of fraud detected, for example, deepfake) to other systems (not shown) or to the server 122 for use by other fraud detection systems. In some implementations, the decision engine 330 uses machine learning, e.g., to determine one or more of the parameters / criteria and / or values used to make which decision(s) may be determined by training machine learning algorithm(s).

[0075] In some implementations, the decision engine 330 includes the aggregated fraud score generator 332 to generate the fraud signal. The aggregated fraud score generator 332 is coupled to receive one or more signals from the anti -injection attack system 326 as will be described in more detail below with reference to Figures 4A and 4B. The aggregated fraud score generator 332 may be steps, processes, functionalities, software executable by a processor, or a device including routines to generate a fraud signal. In some implementations, the fraud signal is a score between a minimum and maximum, where a score that satisfies a threshold indicates that the image provided is acceptable, and one that does not satisfy the threshold is fraudulent. The aggregated fraud score generator 332 is coupled to provide the fraud signal to other components of the decision engine 330.

[0076] For example, in some implementations, the combination of signals provided by the anti-injection attack system 326 to the aggregated fraud score generator 332 may individually be sufficient to determine the presence of fraud (e.g., in the form of an injectionbased attack) and their absence may be indicative of a lack of fraud. However, in someimplementations, the aggregated fraud score generator 332 may evaluate the combined outputs of the anti -injection attack system 326, to determine whether fraud (e.g., in the form of an injection-based attack) is present. In such implementations, the aggregated fraud score generator 332 may be able to more accurately predict the presence or absence of fraud based on a combination of outputs from the multiple signals from the anti -injection attack system 326. For example, in cases where the image(s) may have passed each of the individual evaluations (e.g., injection unlikely, no suspiciously similar pose, and no suspiciously similar background) but barely (e.g., the similarity scores were near the thresholds), the aggregated fraud score generator 332 may use the cumulative results / outputs to determine that an injection-based attack may be present and reject the request or subject the request to additional layers of scrutiny. As another example, assume that the aggregated fraud score generator 332 produces false positives (i.e., injection where no injection is present), the aggregated fraud score generator 332 may decide that one or more of the degree of dissimilarity of the backgrounds and / or poses overrides the injection detection and decide that no fraud is present.

[0077] It should be noted that, while the decision engine 330 is described above as basing the generation of a fraud signal on one or more outputs of the anti-injection attack system 326, the decision engine 330 may use other or additional signals not explicitly described herein to generate the fraud signal. For example, the decision engine 330 may check PII with external databases e.g., motor vehicle or other government databases (not shown) or evaluate other aspects of the input image and its source to determine fraud or validity.Anti -Ini ection Attack System 326

[0078] Referring now to Figure 4A, a first example anti -injection attack system 326 in accordance with some implementations is shown. Figure 4A also shows the coupling of the anti -injection attack system 326 to the aggregated fraud score generator 332. The aggregated fraud score generator 332 is also coupled to receive additional information signals from a selfie metadata extractor 430 and a document metadata extractor 432 that process images to provide additional information to the aggregated fraud score generator 332. For example, the additional information may be extracted metadata such as EXIF that is added to the image by the camera or XMP that is added images by a photo editor. The selfie metadata extractor 430 receives the same selfie image that is input into the anti-injection attack system 326. The selfie metadata extractor 430 processes the received image and generates selfie metadata associated with the image. For example, the selfie metadata may include time and location information about when the photo was taken, geolocation, device type or model, device settings, device manufacturer, camera settings, JPEG header information, etc. The selfie metadata extractor 430 outputs this selfie metadata to the aggregated fraud score generator 332. Similarly, the document metadata extractor 432 receives an image of a document, for example, an ID card, and generates document metadata associated with the image of the document it received. The document metadata extractor 432 is coupled to provide the document metadata to the aggregated fraud score generator 332. For example, the document metadata may include data time and location information about when the photo of the ID was taken, device type or model, device settings, device manufacturer, as well as camera settings. Thus, if the metadata from the selfie image is very different from the metadata from the document image, it is indicative of a fraudulent act, and this can be determined and signaled by the aggregated fraud score generator 332.

[0079] Different injection attacks may use different deepfake, face morph, and face swap generative Al techniques, and each technique may create a different set of artifacts. Depending on the implementation, the anti -injection attack system 326 may train individual models, e.g., an individual model for each of the various injection attack techniques (e.g., deepfake, face morph, and face swap generative Al techniques) or may train a model for a combination of multiple injection techniques. It should be noted that while face swap and face morph are described with reference to generative Al techniques, they may not necessarily rely on generative Al methods, and the functionality and features described herein may generate one or more models to detect artifact(s) associated with those other techniques.

[0080] In some implementations, the anti-injection attack system 326 trains one or more detection models based on a training set of images (document, selfie, or a combination thereof) including images generated using an array of popular open-source deepfake / face swap / face morph generative Al techniques including GAN-based synthetic faces, diffusion model-based synthetic faces, and auto-encoder-based methods. In some implementations, the training set does not include any production injected images (e.g., deepfakes submitted by nefarious users to defeat a verification process), at least initially. In some implementations, the models of the anti -injection attack system 326 may be retrained or use reinforced learning based on images, including deepfakes, face swaps, face morphs, etc., that are encountered in production to improve performance and / or adapt as injection attack technology advances. The anti-injection attack system 326 may train the one or more detection models using a variety of machine learning techniques, depending on the implementation and use case, including supervised learning, unsupervised learning, semi-supervised learning, etc. The varieties of supervised, semi-supervised, and unsupervised machine learning algorithms that may be used, by the anti-injection attack system 326, to train the one or more injection detection models are so numerous as to defy a complete list. Example algorithms include,but are not limited to, a decision tree; a gradient boosted tree; boosted stumps; a random forest; a support vector machine; a neural network; a recurrent neural network; a recurrent neural network; deep learning; long short-term memory; transformer; logistic regression (with regularization), linear regression (with regularization); stacking; a Markov model; Markov chain; support vector machines; and others.

[0081] In some implementations, the anti-injection attack system 326 uses a single frame (e.g., a single still image submitted by the user or a single frame from a selfie that’s video). In some implementations, the anti-injection attack system 326 applies facial detection. In some implementations, the facial detection is used in training the injection detection model, so the model concentrates on the facial features and / or immediately surrounding area, which may be where artifacts from injection are most likely to be present. In some implementations, the anti-injection attack system 326 uses video and multiple frames of images provided by the image receiver 322.

[0082] In some implementations, the anti-injection attack system 326 may train one or more detection models that is a binary classifier. For example, the anti -injection attack system 326 trains multiple binary classifier models using backbone networks like ResNet-34 or EfficientNet and the anti-injection attack system 326 applies the best performing binary classifier (as determined during validation) in production. In production, the injection detection model will be applied to image data provided by users (e.g., customers during an onboarding process) responsive to a request (e.g., for verification). In some implementations, the anti -injection attack system 326 may apply an ensemble method, e.g., by collating the inference results from multiple models to reach a conclusion.

[0083] It should be recognized that, while the foregoing example uses a binary classifier (i.e., two classes — one class associated with the presence of injection and the other class associated with an absence of injection), depending on the implementation more and / ordifferent classes may be present. For example, in some implementations an “inconclusive” class may be present. It should further be recognized that while classification is described above, in some implementations, the anti-injection attack system 326 may apply a regression model to predict a numerical or continuous value, such as a probability that injection is present.

[0084] As shown in Figure 4A, the anti-injection attack system 326 comprises a deepfake model 402, a face morphed model 404, a face swap model 406, an unknown injection attack (UIA) anomaly model 408, a subject and scene segmentation analyzer 410, an injection checker 412, a device risk checker 414, a multimodal liveness detector 420, a face match and face analysis subsystem 422, an injection attack detector 424, and a device risk detector 426. The deepfake model 402, the face morphed model 404, the face swap model 406, the UIA anomaly model 408, the subject and scene segmentation analyzer 410, the injection checker 412, the device risk checker 414, the multimodal liveness detector 420, and the face match and face analysis subsystem 422 are each coupled in parallel to receive an input image or video and provide their output to the aggregated fraud score generator 332. For example, the input image is most typically an image of a person or a selfie. The face match and face analysis subsystem 422 is also coupled to receive a document image. The injection attack detector 424 and the device risk detector 426 are coupled to receive document device information. In some implementations, this document device information is received directly. In other implementations, this document device information is the same information sent to the aggregated fraud score generator 332 that is extracted an output by the document metadata extractor 432. The injection attack detector 424 and the device risk detector 426 are also coupled to provide their output to the aggregated fraud score generator 332. The deepfake model 402, the face morphed model 404, the face swap model 406, the UIA anomaly model 408, the subject and scene segmentation analyzer 410, the injection checker412, the device risk checker 414, the multimodal liveness detector 420, and the face match and face analysis subsystem 422 may each be steps, processes, functionalities, software executable by a processor, or a device including routines to perform the operation and functionality described below for each component.

[0085] On obtaining or receiving the selfie image or video, the anti -injection attack system 326 determines whether it is a valid selfie of a live individual or a fraud attack. To determine if it is an attack, the anti -injection attack system 326 analyzes the selfie by a series of models 402, 404, and 406 trained on previously seen deepfake, face morph and face swap attack examples. The anti -injection attack system 326 also processes the selfie with the UIA anomaly model 408 to determine whether it may be an unknown injected attack. In parallel, the device is checked to see if it has been used before in fraud attacks by the injection checker 412, in particular, whether the image is an injection attack bypassing the camera is in play; and the device risk checker 414 determines if the cameras and images / videos used in the process have clean metadata. In some implementations, a fraud data store 434 stores a fraud database of previously seen fraud attacks and the images used for those fraud attacks. In such an implementation, any of the components of the injection attack system 326 may access the fraud database to check if a person, image, signature, PII real or false data has been processed before. Additionally, the selfie image and the document image are also sent to a prior fraud comparator 428. The prior fraud comparator 428 also has access to retrieve prior known fraud images, fraud assets (digital signature, hashes, etc.), image meta data, or prior fraud detection signals from the fraud data store 434. The prior fraud comparator 428 can query the fraud data store 434 for any images or fraud assets in the fraud data store 434 that match the selfie image or the document image. The prior fraud comparator 428 compares the received selfie image and the received document image to any images or fraud assets used in prior fraud attacks and signals the aggregated fraud score generator 332 if there is a similaritymatch. Effectively, the prior fraud comparator 428 accesses the fraud data store 434 or any other location that stores prior confirmed fraudulent images or data and compares the selfie image and the document image to perform a series of checks on all incoming data with previously seen fraud assets. It should be noted that the fraud data store 434 is also coupled to signal line 440 to receive images that are confirmed to be fraudulent by the aggregated fraud score generator 332.

[0086] Two further checks are performed, namely multimodal liveness check by the multimodal liveness detector 420 and subject and scene segmentation analysis by the subject and scene segmentation analyzer 410. The scores from each of these individual components 402, 404, 406, 408, 410, 412, 414, and 420 are provided as input to the aggregated fraud score generator 332. An image of the document or document scan, document device information and associated metadata used to capture the document are also used to assess an attack by the anti-injection attack system 326.

[0087] The anti-injection attack system 326 also receives document device information and a document image or document scan. In the case of the document scan, the holder image of the document scan is assessed by the face match and face analysis subsystem 422 against the selfie for a face match. The face match and face analysis subsystem 422 also analyzes the selfie and compares it to the gender, age, etc. on the document for a match. The face match and face analysis subsystem 422 outputs a signal representative score to the aggregated fraud score generator 332. The document device information is used by the injection attack detector 424 and the device risk detector 426 to determine if the device used to capture the document was used in prior fraud attacks and also if the document was injected. In some implementations, the device risk detector 426 can be a component provided by a third party. These checks and subsequent scores are also sent to the aggregated fraud score generator 332.

[0088] The deepfake model 402, the face morphed model 404, the face swap model 406 are models to detect these specific types of image generation techniques used by of images by fraudsters. Examples of such models have been described above.

[0089] The UIA anomaly model 408 is a model to detect injections attacks that have not been seen before. In some implementations, the UIA anomaly model 408 is a model to detect suspicious behavior that deviates from normal behavior. The UIA anomaly model 408 also detects threats and shows the corresponding threats when suspicious events corresponding to rules created through an attack profile are constantly occurring. The UIA anomaly model 408 may also be configured to detect malicious processes. The UIA anomaly model 408 generates a signal that can be used by the aggregate score generator as 332 in generating the accumulated score.

[0090] The subject and scene segmentation analyzer 410 processes the selfie image to detect segments of an image that have been used by fraudsters in the past. Example implementations for the subject and scene segmentation analyzer 410 will be described in more detail below with reference to Figure 5. The subject and scene segmentation analyzer 410 is particularly advantageous because it can detect similarities and portions of an image that a fraudster has repeatedly used. The subject and scene segmentation analyzer 410 is also advantageous because it reduces the computational requirements by only having to process a portion of the scene and has increased accuracy due to scene segments having a greater degree of similarity. The subject and scene segmentation analyzer 410 outputs its score to the aggregated fraud score generator 332.

[0091] The injection checker 412 receives the selfie image and checks whether the selfie image was used in an injection attack in the past as has been described above, and if so, signals the aggregated fraud score generator 332.

[0092] The device risk checker 414 receives the selfie image, extracts the metadata from the selfie image and determines whether the metadata from the selfie image indicates whether the used to take the selfie has been used in a prior fraud attack in the past as has been described above, and if so, signals the aggregated fraud score generator 332. In some implementations, the device risk checker 414 can be a component of a third-party or can use information from other large data providers or vendors, for example, LexisNexis.

[0093] The multimodal liveness detector 420 receives the selfie image and performs multimodal liveness analysis. The multimodal liveness detector 420 processes the selfie and performs two or more liveness detection checks on the received selfie image. The results of the liveness analysis by the multimodal liveness detector 420 are provided to the aggregated fraud score generator 332.

[0094] The face match and face analysis subsystem 422 is coupled to receive the selfie image and also coupled to receive the document holder image. The face match and face analysis subsystem 422 performs two or more matching operations. The face match and face analysis subsystem 422 compares the selfie to the holder portion of the document image. Based on the comparison, the face match and face analysis subsystem 422 outputs a first signal to the aggregated fraud score generator 332. The face match and face analysis subsystem 422 also processes the document image to generate document holder characteristics, for example, age, gender, height, weight, eye color, etc. and analyzes the selfie image to determine whether the characteristics of the selfie image match the document holder characteristics. Based on the comparison, the face match and face analysis subsystem 422 outputs a second signal to the aggregated fraud score generator 332.

[0095] The injection attack detector 424 receives the document device information and checks whether the document device information is associated with a past injection attack. In some implementations, the injection attack detector 424 and the injection checker412 perform the same processing merely on different input data sources, in particular, the injection attack detector 424 processes the document device information while the injection checker 412 processes the selfie image or video. In other implementations, the processing by the injection checker 412 and the injection attack detector 424 are different.

[0096] The device risk detector 426 receives the document device information, processes that metadata associated with the document, and determines whether the device is associated with the past fraudulent attempts. In some implementations, the device risk detector 426 and the device risk checker 414 perform the same processing merely on different input data sources, in particular, the device risk detector 426 processes the document device information while the device risk checker 414 checker processes the selfie image or video. In other implementations, the processing by the device risks checker 414 and the device risk detector 426 are different.

[0097] It should be understood that one particular advantage of the above-described architecture for the anti-injection attack system 326 is that it is scalable, and that additional models or other detection types may be added to process the selfie image in parallel with components 402, 404, 406, 408, 410, 412, 414, and 420. This is particularly advantageous to be able to add additional models of detection types as the fraudsters develop new types of attack. For example, although not shown in Figure 4A, the anti-injection attack system 326 may also include a pose comparator. The pose comparator determines and compares a pose between multiple images. Assume that a request is associated with a first received input image is received that is a picture of a document with a document holder image (e.g., a picture ID) and a second received input image that is a selfie, and that the person in the document holder image and selfie need to match otherwise the request is rejected (e.g., as fraudulent). However, too close of a match may be indicative of reproduction and, therefore, fraud. In some implementations, the pose comparator determines and compares a posebetween a document holder image (e.g., from a document image) and a facial image (e.g., from a selfie or video that may also be used for liveness detection). Such an example pose comparator receives image data, determines the pose (e.g., applies pose estimation) to each of the images to be compared, and compares the poses. For example, the pose comparator receives a set of associated images (e.g., responsive to a verification request that includes a selfie image and a document image), determines the pose of the document holder’s facial image (e.g., based on key points associated with various facial features), determines the pose of the face in the selfie, and compares the pose to the selfie image. In a valid instance it is very unlikely, near impossible, that the user’s pose (e.g., the pitch, roll, and yaw of the head or face and / or a facial expression) in the selfie would reproduce (i.e., be identical or nearly identical) the user’s pose in the user’s own document holder image. The pose comparator compares the poses and determines whether the pose between images satisfies a similarity threshold. The similarity threshold, when satisfied, may be indicative that the poses are sufficiently similar, which may be indicative of fraud. In some implementations, there may be multiple thresholds. For example, a first threshold of high pose similarity when satisfied may be associated with and indicative of fraud, a second threshold of moderate pose similarity, when satisfied may be associated with and indicative of inconclusiveness, and when neither the first or second threshold are satisfied it may be indicative of validity or an absence of fraud. The number of thresholds, or classification, may vary, e.g., in some implementations, there may be a single threshold (or two classes — one indicative of a suspiciously high pose similarity and another associated with non-suspicious pose similarity). In some implementations, the threshold(s) or classes may be determined using machine learning. For example, a classifier is trained to classify pairs of images (e.g., document image and selfie) into suspicious and non-suspicious classes based at least in part on theirpose similarity score. The pose comparator is merely one example of new and additional fraud detection types that may be added to the anti -injection attack system 326.

[0098] In some implementations, the deepfake model 402, the face morphed model404, the face swap model 406, the unknown injection attack (UIA) anomaly model 408, the subject and scene segmentation analyzer 410, the injection checker 412, the device risk checker 414, and the multimodal liveness detector 420 may receive and process the document image in addition to the person image or selfie. In such a case, the deepfake model 402, the face morphed model 404, the face swap model 406, the UIA anomaly model 408, the subject and scene segmentation analyzer 410, the injection checker 412, the device risk checker 414, and the multimodal liveness detector 420 each outputs two signals to the aggregated fraud score generator 332, one signal or score for the selfie image and one signal or score for the document image. This is advantageous because it increases the accuracy of the fraud detection by having both the selfie image and the document image process by each detector type.

[0099] The aggregated fraud score generator 332 receives signals from the components 402, 404, 406, 408, 410, 412, 414, 420, 422, 424, and 426 and uses those signals to generate an aggregate fraud signal on signal line 440. In one implementation, the components 402, 404, 406, 408, 410, 412, 414, 420, 422, 424, and 426 each generate a numerical score within a predefined range with one end of the range indicating fraud and the other end the range indicating the absence of fraud. The aggregated fraud score generator 332 combines those scores to produce the aggregated fraud signal. The aggregated fraud signal has a value between a minimum and maximum where a value satisfying a threshold between the minimum and maximum indicates that the selfie is an acceptable image, and a score or value that does not satisfy the threshold is a fraudulent image or an injection attack. For example, the aggregated fraud score generator 332 may be a gradient boosting machinelearning model, e.g., XGBoost, trained on hundreds of parameters to provide an overall score. The output from components 402 to 432 could be scores between 0 and 1 and used to train the gradient boosting machine learning model with these signals or could feed another aggregated model trained only on the signals from the subsystems in Figure 4. In some implementations, the numerical scores from each component 402, 404, 406, 408, 410, 412, 414, 420, 422, 424, and 426 are weighted to generate the aggregated fraud signal. In another implementation, some of the signals from the components 402, 404, 406, 408, 410, 412, 414, 420, 422, 424, and 426 input to the aggregated fraud score generator 332 are binary signals indicating the presence of fraud (false) or the absence of fraud (true). In such a case, the aggregated fraud score generator 332 includes logic to output a true signal indicating an image is acceptable or a false signal indicating the presence of fraud if any one of the components 402, 404, 406, 408, 410, 412, 414, 420, 422, 424, and 426 input a signal indicating the presence of fraud into the aggregated fraud score generator 332. In yet another implementation, the aggregated fraud score generator 332 includes logic that combines scores within a predefined range from some of the components 402, 404, 406, 408, 410, 412, 414, 420, 422, 424, and 426, and binary signals indicating the presence of fraud or the absence of fraud from other of the components 402, 404, 406, 408, 410, 412, 414, 420, 422, 424, and 426 to generate one or more aggregated fraud signals indicating the presence or absence of fraud for the selfie image submitted.

[0100] Referring now to Figure 4B, a second example anti-injection attack system 326 in accordance with some implementations is shown. In this second example, like components with the same or similar functionality have the same reference number is used in Figure 4A. Since that functionality has been described above with reference to the description of Figure 4A above, that description will not be repeated here. As shown in Figure 4B, the acquisition channels for the selfie or video image are shown in more detail.The selfie or video image may be received by an application programming interface (API) 452, via the Internet or the World Wide Web using a web browser 450, or via a mobile device 454. Each of these individual channels may capture the video or selfie image that is provided as input to the anti-injection attack system 326 with the scalable architecture. The same selfie image is also provided to the multimodal liveness detector 420 and the selfie metadata extractor 430 as has been described above. Another notable difference in this second example of the anti -injection attack system 326 is that the deepfake model 402, the face morphed model 404, and the face swap model 406 are replaced by one model or an ensemble of models. For example, Figure 4B shows them replaced by a multi-frame deepfake detection model 416 and a single frame deepfake detection model 418. In some implementations, the multi-frame deepfake detection model 416 and a single frame deepfake detection model 418 operate similar to the models described above but just operate on either multiple frames of images or single frame of an image. In some implementations, the multiframe deep fake detection model 416 is a convolutional vision transformer. The transformer has two components, namely, a Convolutional Neural Network (CNN) and a Vision Transformer (ViT). The CNN is used to extract learnable features while the ViT uses the learnable features as input and categorizes them using an attention network. The multi-frame deep fake detection model 416 is trained using a sizable deepfake video dataset. It should be understood that this is only one example approach for detecting deepfake multi-frame attacks, and that numerous other methods may be used for the multi-frame deep fake detection model 416. In some implementations, for detecting single frame deepfake images, the single frame deepfake detection model 418 may be a binary classifier based on a two-phase learning scheme. The first phase is designed to find distinguishing features for fake images. The second phase is a two-layer CNN that takes the feature encodings from the first phase and outputs a real / fake classification decision. Again, it should be understood that this is onlyone example approach for detecting deepfake single-frame attacks, and that numerous other methods may be used for the single frame deepfake detection model 418.Example Subject & Scene Segmentation Analyzer 410A

[0101] Figure 5 shows an example subject and scene segmentation analyzer 410 in accordance with some implementations. In some implementations, the subject and scene segmentation analyzer 410A comprises a segmentation system 502, a vectorization system 504, a similarity search system 506 and a decision system 508. By segmenting the selfie image into discrete components and analyzing the components individually, a direct or similarity match can be performed against previously discovered fraud selfie image attacks. Additionally, new fraud attacks can be discovered by identifying large clusters formed by similar images. Deepfake or other document creation mechanisms are now becoming more commonly seen during the ID verification process. Using images of the entire documents directly, with holder image and PII data present, creates more complications for detecting fraudulent cards. The subject and scene segmentation analyzer 410A determines “regions of interest” containing the images and PII data of the document and eliminates them from the incoming scanned document using predetermined “region of interest” masks in the “Document Assembly Object” (DAO) and uses either an ML model previously trained on both valid ID’s and fraudulent IDs to determine a fraudulent document or, alternatively, an anomaly detection method to separate valid from fraudulent documents.

[0102] The segmentation system 502 involves decomposition of an image into one or more different portions. The segmentation system 502 can be applied to either a selfie image or a document image. In some implementations, the segmentation system 502 receives images that are known to be fraudulent and decomposes them into one or more different parts or segments. Either the selfie image or the document image may be decomposed intoparticular portions that make detection of similarity computationally less expensive and more accurate. The segmentation system 502 may be steps, processes, functionalities, software executable by a processor, or a device including routines to segmenting image into one or more different portions. In some implementations, the segmentation system 502 decomposes a selfie image into a background, a head, a hairstyle / shape, a neck, and shoulders. For example, the background segment eliminates the entire subject (head, hairstyle / shape, neck, shoulders, and clothing) from the image. An example head segment eliminates the background, neck, shoulders, and clothing from the image. An example hairstyle / shape segment eliminates everything from the image except for the hairstyle / shape. An example neck segment eliminates the background, head, hairstyle / shape, and shoulders from the image. An example shoulders and clothing segment eliminates the background, head, and hairstyle / shape. The segments may also include body-skin, face-skin, facial features, etc. A different example of segmentation of an image 1702 into a background segment 1706, body segment 1714, and a face segment 1724 are shown in Figure 17A. It should be understood that any number of different segments may be applied to a given image. In one example, the segmentation system 502 is the media pipe selfie segmenter. Based on the attributes of fraudulent images that have been submitted, those images can be analyzed to determine which segments are repeatedly used with a high frequency of similarity. The segmentation system 502 is coupled to provide the segmentation once defined to the vectorization system 504.

[0103] The vectorization system 504 receives the segmentation of an image, e.g., the segmented parts of the image from the segmentation system 502. The vectorization system 504 passes one or more of the segmented parts of the image through a neural network to generate one or more vector embeddings. In some implementations, vectorization system 504 may each be steps, processes, functionalities, software executable by a processor, or adevice including routines to generate the vector embedding. In some implementations, the vectorization system 504 can be obtained on training for the task specifically or by transfer learning from other domains, to obtain a vector embedding. The vectorization system 504 sends the vector embedding to the similarity search system 506. It should be understood that the vectorization system 504 may receive many segmentations for many different images and for each of them generates a different vector embedding. The set of vector embeddings can be provided to the similarity search system 506.

[0104] The similarity search system 506 searches for the similarity checks of the vector embedding of the query scan against an index of embeddings from previous transactions. The similarity search system 506 may each be steps, processes, functionalities, software executable by a processor, or a device including routines to perform the searches for similarity checks based on the vector embedding. In some embodiments, the similarity search system 506 searches for cosine similarities between embeddings in a N-by-N matrix. In one example, the similarity search system 506 identifies similar scans based on using the status. In another example, the similarity search system 506 performs a blind search to determine whether there are a predefined Y number of items above an X similarity threshold. The results of the similarity search system 506 are provided to the decision system 508.

[0105] The decision system 508 receives the search results from the similarity search system 506 and generates a decision on whether the scanned image is fraudulent or not. The decision system 508 may each be steps, processes, functionalities, software executable by a processor, or a device including routines to generate the decision on whether a scan image is fraudulent. In some implementations, the decision system 508 uses two different modes to determine whether an input image is fraudulent. In a first mode, the decision system 508 determines whether the input image has matched a previously discovered fraudulent case. For example, if the vector embedding for the scanned image matches a vector embedding fora prior image known to be fraudulent, the input image is identified as being fraudulent. If so, the input image is identified as fraudulent. In a second mode, the decision system 508 determines whether the input image has matched a large number of prior scans, and it fulfills certain requirements, e.g., a large ratio of the segmented portion of the whole image matches similar segments of the images known to be fraudulent. For example, if the vector embedding for the scanned image matches a predefined number of vector embedding for images that are known to be fraudulent, the input image is identified as being fraudulent.Example Subject & Scene Segmentation Analyzer 41 OB

[0106] Often a nefarious user who repeatedly attempts to commit fraud does not use completely distinct documents, images of documents, or backgrounds across his / her multiple attempts. For example, the fraudster uses an instance of a document, then modifies, the name, the date of birth, and ID number, and so on, but there will be commonalities between the attempts. Examples of commonalities may include, but are not limited to, the document’s surroundings or background; the facial image; the issuer of the ID; the size, orientation, or position of the document in the image; etc. As another example, the fraudster may provide selfies of himself / herself swapping or morphing his / her face to those of various valid document holders, but there will be commonalities between the attempts. Examples of commonalities may include, but are not limited to, clothing, hair, head outline, backgrounds, objects in the background, etc.

[0107] The systems and methods described herein may at least partially address one or more of a number of challenges. Deepfakes, face swaps, and face morphs may become increasingly difficult to detect and the algorithms for generating them improve making detection more difficult. Automatic identification of prior images with commonalities, where the commonalities may be similarities rather than identicalities, using computer vision is non-trivial and differs from a human’s mental process. For example, it may be trivial for a human to determine “similar” subject matter between two images; however, reliable detection of similarity (not identicality) with even modest changes (e.g., a slight change in the cropping between images, slight change in the angle or position of the camera, change in the angle or position of an object between images, change in the lighting conditions, etc.) is non-trivial in the field of computer vision and may be at least partially addressed by the systems and methods herein. Focusing on an image segment (e.g., the scene, also referred to herein as the surroundings or background, and the subject (e.g., a person or document) or sub-segment thereof (e.g., a person’s hair, or an object in the background) is non-trivial in the field of computer vision and may at least be partially addressed by the methods and systems herein. The amount of image data that needs to be processed is generally increasing (e.g., as the number of transactions associated with images for authentication increases, the resolution of those images increases, etc.). Accordingly, how to efficiently process an image received for authentication, also referred to herein as an image under test, compare the image under test to prior images, and maintain data about prior images for comparison is a challenge and may be partially addressed by the system and methods herein.

[0108] Referring now to Figure 18, a block diagram of an example subject and scene segment analyzer 41 OB is illustrated in accordance with one implementation. As illustrated in Figure 18, the subject and scene segment analyzer 41 OB may include an image segmenter 1824, a descriptor generator 1828, a similar segment look-up engine 1830, a decision engine 1832, and a descriptor retention engine 1834.

[0109] For clarity and convenience, an example repeat attack is referenced frequently herein. Referring to Figures 21 A-21C a set of example images 2102, 2104, and 2106 are illustrated. The set of images were produced using an injection attack (e.g., deepfake, face morph, or face swap); accordingly, the faces differ, but the images share a number ofcommonalities. Assume for the purposes of our example, that image 2102 is the image under test (i.e., the image presently being evaluated to determine whether the image is genuine) and was received by the image receiver 322. Also, assume that images 2104 and 2106 were received by the image receiver 322 at an earlier time and / or in association with a prior transaction.

[0110] In some implementations, the image segmenter 1824 divides an image into a plurality of segments. In some implementations, the image segmenter 1824 divides the image into a first segment associated with the subject and a second segment associated with a scene, also referred to herein as a background or surroundings. In the example use cases discussed herein, frequently refer to use cases in which the subject includes a person (e.g., in the context of a selfie image), a document (e.g., a check), or both (e.g., a photo identification document).[OHl] For example, the selfie image 2102 of Figure 21 may be sub-segmented into a subject segment represented by segment 2202 in Figure 22, and a scene segment (not shown) representing surroundings, i.e., the remaining portion of the selfie image 2102 (i.e., absent the subject segment 2202).

[0112] In some implementations, one or more of the segments may be subsegmented. In some implementations, the plurality of segments includes one or more subsegments. As illustrated in Figure 18, in some implementations, the image segmenter 1824 includes a subject sub-segmenter 1842 and a scene sub-segmenter 1844.

[0113] The subject sub-segmenter 1842 generates one or more subject sub-segments. Examples of subject sub-segments where the segment represents a document as its subject may include, but are not limited to, document holder information visible on the document (e.g., name, address, date of birth, demographics, such as gender, weight, height, eye color, hair color, etc.), a document identifier (account number, routing number, driver’s licensenumber, passport number, etc.), a document security feature (e.g., document holder image, ghost image, laser perforations, hologram, raised text, etc.) Examples of sub-segments where the segment represents a person (e.g., in a selfie image) may include, but are not limited to, the person’s head, the person’s hair, the person’s neck, the person’s torso, the person’s clothes, and the person’s hand(s), etc. These may also be examples of sub-sub-segments, e.g., where such segments are extracted from a person represented in document holder image sub-segment.

[0114] For example, referring to Figure 22, from subject segment 2202, the subject sub-segmenter 1842 generates (illustrated by line 2210) sub-segment 2220 that represents the person’s hair, generates (illustrated by line 2212) sub-segment 2222 that represents the person’s face and neck, and generates (illustrated by line 2214) sub-segment 2224 that represents the person’s clothes. It should be recognized that these are merely example segments, and other segments and different segments exist. For example, in some implementations, the subject sub-segmenter 1842 may generate as a segment (not shown) or sub-segment of segment 2222 (not shown) representing the person’s neck (e.g., the portion of 2222 below the jawline) separately.

[0115] Referring now to Figure 19, a block diagram illustrating an example subject sub-segmenter 1842 is illustrated in accordance with some implementations. In the illustrated implementation, the subject sub-segmenter 1842 includes a silhouette segment determiner 1902 to determine the outline of the head, a face segm enter 1904 to determine a segment representing the face, a hair segment determiner 1906 to determine a segment representing the subject’s hair (or scalp in that case of baldness), upper torso segment determiner 1908 to determine the user’s upper torso (e.g., the portion below either the jawline or neckline that’s in frame), a neck segment determiner 1910 to determine a segment representing the neck (e.g., between the jawline and clothing neckline), and a clothingsegment determiner 1912 for determining a segment representing the subject’s clothing. However, other or different (sub-)segment determiners may be present or used. For example, one or more of 1902-1912 may optionally be omitted, depending on the implementation, and / or a hat segment determiner (not shown) may be present for determining a segment associated with headwear, whether fashionable, functional, or religious in nature.

[0116] The scene sub-segmenter 1844 generates one or more scene sub-segments.Examples of scene sub-segments where the segment represents a scene (e.g., in a selfie image or document image) may include, but are not limited to, hand(s) holding a document, furniture (desk, chairs, etc.), appliances, surface treatments (e.g., wood paneling, brickwork, wood grains, etc.), decor (e.g., vases, window treatments, light fixtures, mirrors, etc.), windows, vegetation (e.g., house plants, trees, etc.), electronics, cabling, clutter, or other objects in the background.

[0117] For example, referring to Figure 23, the scene sub-segmenter 1844 generates a set of sub-segments from the background segment (not shown). In the illustrated example of Figure 23, a window sub-segment 2302, furniture sub-segments 2304, 2306, 2308, shelf subsegment 2310, the ceramics with plant matter decor sub-segments 2312 and 2314, the bottle sub-segment 2318 and the background of the shelf behind that bottle sub-segment 2216, a faucet segment 2320, brickwork sub-segments 2322 and 2324 representing the brickwork above and below, respectively, the shelf of sub-segment 2310, are shown. However, it should be recognized that the scene sub-segments illustrated in Figure 23 are non-exhaustive and that additional or different sub-segments may be generated. For example, referring now to Figure 24, image 2400 illustrates an image similar to 2102 (e.g., there is a container on furniture under the window not present in 2102) with thickened line weight outlines added to represent an example of the sub-segments that may be generated from image 2102 in context. Sub-segments analogous to those illustrated in Figure 22 and 23 are labeled. However, asshown in Figure 23 sub-segmentation may vary based on the implementation and use case. For example, the ceramics with plant matter decor sub-segments 2312 and 2314 of Figure 23 are each (sub)divided into a plant matter sub-segment and a ceramic sub-segment in Figure 24, and the window segment 2302 is divided into individual sub-segments for each pane and the frame in Figure 24.

[0118] In some implementations, the image segmenter 1824 converts the image under test into the edge domain, determines the perimeter of the segments, and determines the perimeter of sub-segments. In some implementations, the image segmenter 1824 or component thereof (e.g., the subject sub-segmenter 1842 or the scene sub-segmenter 1844) may perform contour extraction. For example, the image segmenter 1824 applies a contour finding method to extract the outline of the subject segment 1002 from an edge detected version (not shown) of image 2102, and the subject sub-segmenter 1842 extracts sub-segment 2222 representing the head and neck from an edge detected version (not shown) of segment 2202 or directly from an edge detected version (not shown) of image 2102, depending on the implementation.

[0119] In some implementations, the image segmenter 1824 and / or components thereof may make segments and / or sub-segments available to one or more components of the subject and scene segment analyzer 410B. In some implementations, the image segmenter 1824 or components thereof communicate the segments and / or subsegments, or stores the segments and / or sub-segments, and one or more other components of the subject and scene segment analyzer 410B, e.g., the descriptor generator 1828, is coupled to obtain the segment(s) and / or sub-segment(s).

[0120] The descriptor generator 1828 generates one or more descriptors. In some implementations, the descriptor generator 1828 converts an image or portion thereof (e.g., animage segment or sub-segment) into a feature vector representing one or more features associated with that image or portion thereof.

[0121] In some implementations, the one or more descriptors may include a descriptor associated with the image as a whole, a segment, or a sub-segment. In some implementations, the type of descriptor may vary based on one or more of the implementations, use case, and portion of the image (e.g., image, segment, sub-segment, sub- sub-segment, etc.). For example, in some implementations, the descriptor may include, or be based on, a mathematical representation (e.g., a geometric feature), a textual representation (e.g., a textual description of the content of the segment), or both (e.g., a geometric feature and textual descriptor of the same segment). In some implementations, multiple, different descriptors may be associated with the same portion of the image. For example, in some implementations, the descriptor generator may generate a textual description of clothing subsegment 2224, such as “blue, button-up, collared shirt,” and a mathematical representation of the clothing sub-segment 2224, such as a vector embedding based on a one or more geometric features (e.g., perimeter, shape, compactness, an area size, etc.).

[0122] As illustrated in Figure 18, in some implementations, the descriptor generator 1828 may include a mathematics-based descriptor generator 1852 and a text-based descriptor generator 1854. However, in some implementations, one of the mathematics-based descriptor generator 1852 (or a component thereof) and the text-based descriptor generator 1854 may optionally be omitted and / or other descriptor generators may be present without departing from the description herein. Additionally, while a single mathematics-based descriptor generator 1852 and a single text-based image descriptor generator 1854 are illustrated in Figure 18, multiple different instances of each, or both, may be present and specialized to generate a descriptor for a particular segment or sub-segment content (e.g., one to textually describe the subjects clothes, another to textually describe the person’s face andhair, another to describe the perimeter of the person’s head or hair, etc.); descriptors of different features of a common sub-segment type (e.g., a pose, facial features, and a face mesh generated from the face sub-segment).

[0123] The mathematics-based descriptor generator 1852 generates a mathematical representation associated with an image segment or image sub-segment (e.g., derived from an image under test). The plurality of mathematical representations may include mathematical representations of the image under test as a whole, a segment of the image under test, and / or one or more sub-segments (or sub-sub-segments) thereof.

[0124] In some implementations, the mathematics-based descriptor generator 1852 generates a mathematical representation by determining a set of parameters and then converting the parameters into a vector. Examples of parameters include, but are not limited to, one or more of color parameters, another domain representation, and geometric feature parameters. Examples of color may include color histograms or statistical representations of one or more color histograms (e.g., mean, mode, standard deviation, min, mac, etc. for each RGB color channel, etc.). Examples of another domain representation may include, but are not limited to a representation in the frequency domain, edge domain, color domain (e.g. a color distribution), etc. Examples of geometric features include, but are not limited to, a shape, an area size, a compactness, a perimeter, etc.

[0125] In some implementations, the mathematics-based descriptor generator 1852 generates a mathematical representation by obtaining an embedding, such as a vector or other embedding. For example, the mathematics-based descriptor generator 1852 trains, validates, and applies a neural network, and a sub-segment of the image under test is input into that neural network, such as a convolutional neural network (CNN), transformer(s), generative adversarial network (GAN), Vision Large Language Model, etc., and an embedding is output. In some implementations, the output is a semantic embedding optimized for the task ofretrieving similar (sub-)segments. In some implementations, the encoding is generated using a topological method, such as persistent homology (e.g., to create a persistent barcode).

[0126] In some implementations, the mathematics-based descriptor generator 1852 generates one or more embeddings. In some implementations, the mathematics-based image descriptor generator 1852 trains a model in an attribute-aware fashion, e.g., using a transformer architecture that is customized to put emphasis on more informative parts of the image. For example, the mathematics-based descriptor generator 1852 generates a set of attention maps highlighting informative parts of an image. Then, the mathematics-based descriptor generator 1852 uses a feature diffusion module to iteratively inject attribute information into global feature maps, thereby suppressing unnecessary noise and inferring attribute-aware representations. Then, the mathematics-based descriptor generator 1852 uses a feature aggregation module to exploit mutual information for aggregating attribute characteristics from different images, thereby enhancing the representation capabilities of feature embedding.

[0127] In some implementations, the informative parts of the image may be used to determine the segments or sub-segments for which descriptor generator 1828 or component thereof generates a descriptor. For example, assume the clothing and hair are selected as more important attributes, in some implementations, a generated descriptor of the subject segment may emphasize those features in the encoding. As another example, assume that in the face certain points are not modified by an injection attack method (e.g., the space between the fraudster’s eyes, or the width and location relative to the eyes of the fraudster’s mouth, or other); in some implementations, the descriptor generator 1828 may generate an embedding that emphasizes those points / features, which may improve the results of a subsequent similarity -based search by the similar segment look-up engine 1830.

[0128] In some implementations, the mathematics-based descriptor is contentdependent. For example, the mathematics-based descriptor generator 1852 generates a description of a sub-segment representing the subject’s hair differently than the mathematicsbased descriptor generator 1852 generates a description of a person’s face.

[0129] It should be recognized that while a variety of descriptors have been described, and one or more of the descriptors and how a particular descriptor is generated may vary based on the implementation, use case, sub-segment content, etc., the descriptor for a given type of sub-segment (e.g., a vectorization based on the perimeter of a sub-segment representing person’s hair) remains consistent between a sub-segment of an image under test and that type of sub-segment, also referred to as the corresponding sub-segments (i.e., hair sub-segments in the current example), associated with prior transactions. Otherwise, the similar segment look-up engine 1830 would not return results or meaningful results. For example, querying prior hair sub-segments represented by an N-dimension vectorization of the perimeter of those prior hair sub-segments for those most similar to an M-dimension vectorization of the perimeter (or an embedding) of the hair sub-segment under test is unlikely to yield accurate or useable results.

[0130] In some implementations, the mathematics-based descriptor generator 1852 is content dependent. Figure 20 illustrates an example of a mathematics-based descriptor generator 1852 that generates one or more mathematics-based descriptors based on the content of the sub-segment. In the illustrated implementation, the mathematics-based descriptor generator 1852 includes a perimeter generator 2002, a facial feature generator 2004, a face mesh generator 2006, and a pose generator 2008. However, other or different descriptors are within the scope of this disclosure and may be included. For example, semantic descriptors may also be used. Additionally, one or more of the perimeter generator2002, a facial feature generator 2004, a face mesh generator 2006, and a pose generator 2008 may optionally be omitted depending on the implementation.

[0131] In some implementations, perimeter generator 2002 determines a perimeter associated with a sub-segment, e.g., the perimeter, also occasionally referred to as the “outline,” of the sub-segment itself. In some implementations, the perimeter generator 2002 determines the perimeter using one or more of a linear piecewise approximation and splines. In some implementations, the perimeter generator 2002 obtains the contour of the outline of the sub-segment, obtains a set parameter indicating the maximum length of a line segment, and fits line segments to contour using a method such as the Douglas-Peucker algorithm to reduce the number of points while maintaining (at least roughly) the shape. In some implementations, the perimeter generator 2002 may then calculate the slope and intercept (or use another representation such as start and end points) for each linear segment and combine those into a vector representing that portion (e.g., sub-segment) of the image. In some implementations, the perimeter, or another geometric, descriptor may be obtained using a machine learning model (e.g., a convolutional neural network, transformers, Vision LLM, etc.). In some implementations, this vector may be searched by the similar segment look-up engine 1830 to return a result set or refine an earlier result set.

[0132] In some implementations, the perimeter generator 2002 may be applied to any or all sub-segments, as all sub-segments are associated with a perimeter. For example, the perimeter generator 2002 may generate, or derive, descriptors for one or more of the scene segment, the subject segment, one or more sub-segments associated with the scene, such as for sub-segments 2302, 2304, 2306, 2308, 2310, 2312, 2314, 2316, 2318, 2320, 2322, and 2324, one or more sub-segments associated with the subject such as sub-segments 2220, 2222, and 2224. In some implementations, the perimeter generator 2002 may be applied differently to different sub-segments, e.g., based on content or other characteristics (e.g. areaor other size). For example, a smaller maximum length in the linear piecewise approximation may be applied to a hair sub-segment than a background object in order to preserve more detail in the description of the hair sub-segment. As another example, the maximum length of a segment in the linear piecewise approximation may vary based on segment size, e.g., to preserve detail in relatively small background objects.

[0133] The perimeter descriptor may be an efficient and useful means for computer- based identification of a commonality that may be indicative of fraud and may be overlooked by one or more of a fraudster and / or the tools the fraudster employs in repeated fraud attacks. For example, in some injection attacks, such as a face swap or face morph, the injected face is mapped (e.g., in real-time in the case of a video) to the fraudster’s face. However, the modifications are limited to the facial, so one or more of the fraudster’s hair, the outline of the fraudster’s head (e.g., hair, ears, etc.), the fraudster’s neck, shoulders, and clothes may be unmanipulated by the injection attack. The objects in the background may also be unmanipulated by the injection attack. The tools (e.g., for injection attacks in the present example) available to a fraudster allow a fraudster to easily and rapidly make multiple attempts. Therefore, when the fraudster makes repeated fraudulent attempts (e.g., using injection attacks), little time may have passed or they are doing it from a consistent location, so one or more of objects in the background, the fraudster’s clothes, the fraudster’s hair style, the fraudster’s head outline, etc. may have changed little between attempts.

[0134] In some implementations, mathematics-based descriptor generator 1852 is content dependent and provides a sub-segment representing a face to one or more of a facial feature generator 2004, a face mesh generator 2006, and a pose generator 2008.

[0135] In some implementations, the facial feature generator 2004 obtains a description of one or more facial features. In some implementations, the facial feature generator 2004 performs linear piecewise approximation to obtain major features of the face,such as the eyes, the nose, and the mouth. In some implementations, the facial feature generator 2004 applies splines or linear piecewise approximation to model the face using a description based on edge loops. In some implementations, the facial feature generator 2004 includes a facial recognition model and obtains an embedding from a facial recognition model for the face sub-segment. In some implementations, the facial feature generator 2004 obtains the facial feature description from one or more machine learning models (e.g., CNN, transformers, Vision LLM). In some implementations, the areas around facial key points may be treated as a sub-sub-segment and embedded, e.g., by a neural network (e.g., convolutional neural network, transformers, Vision LLM, etc.), as description(s) of the facial feature(s) and subsequently searched.

[0136] In some implementations, the face mesh generator 2006 obtains a description based on a face mesh describing facial landmarks. The face mesh generator 2006 may generate a facial mesh then generate a vector based on some or all of the landmarks therein. In some implementations, the face mesh generator 2006 generates a description of a reduced set of landmarks from the face mesh. In some implementations, the reduced set of landmarks may be described through persistent homology. For example, assume that in an injection attack, aspects of the fraudster’s face are consistent across fraud attempts (e.g., the ratio of the distance between the eyes to the width of the mouth and / or the distance between the eyes and the mouth) because the injection attack is mapping the fraud victim’s facial features to the fraudster’s facial features, and these aspects remain consistent. In some implementations, the face mesh generator 2006 generates a vector based on these ratios or landmarks associated therewith.

[0137] In some implementations, the pose generator 2008 generates a description of the facial pose (e.g., pitch, roll, and yaw). In some implementations, the pose generator 2008 determines the pose from the landmarks associated with the facial mesh generated by the facemesh generator 2006. It should be noted that, a near-identical pose between genuine images is highly unlikely, so when the results of a look-up by the similar segment look-up engine 1830 returns pose descriptors, the decision engine 1832 may determine whether the pose descriptors indicate that the pose vectors are “too similar,” which may be indicative of fraud, e.g., copy-pasting of the selfie photo into the document holder facial image.

[0138] The text-based descriptor generator 1854 generates a computer-generated, text-based descriptor associated with a portion of the image under test. For example, in some implementations, the text-based descriptor generator 1854 generates a textual description of one or more of the scene, the subject, or a portion thereof. Referring to example image 2102, in some implementations, the text-based descriptor generator 1854 generates a text-based description of one or more of the background (e.g., “indoors, kitchen, brickwork”), the subject (e.g., “Gender: Male, Age: Middle age, Hair: Dark, Short, Straight, Hat: none, Skin Tone: Tan”), or a portion thereof (e.g., describing the subject’s clothing as “Color: Light Blue, Sleeve Length: Long, Style: Button-up, Collared: Yes), etc.

[0139] In some implementations, the text-based descriptor is a string. In some implementations, the text / string may be converted into vector, embedded or otherwise converted into mathematics-based representation of the text, e.g., so that the converted, textbased descriptor may be used by the similar segment look-up engine 18180 in a manner analogous to one or more of the other mathematics-based representations described above.

[0140] In some implementations, the text-based image descriptor generator 1854 uses a large language model (LLM) or a vision large language model (Vision LLM). In some implementations, an image under test or a portion thereof (e.g., one or more of scene segment, subject segment, and a sub-segment thereof) is fed into the Vision LLM, attributes are extracted from the output text, and used for subsequent searching by the similar segment look-up engine 18180 against other images (or portions thereof).

[0141] In some implementations, the descriptor generator 1828 and / or one or more (sub)component thereof may make a set of descriptors available to one or more components of the subject and scene segment analyzer 41 OB. In some implementations, the descriptor generator 1828 or components thereof communicate the segments and / or subsegments, or stores the segments and / or sub-segments, and one or more other components of the subject and scene segment analyzer 410B, e.g., the similar segment look-up engine 1830, is coupled to obtain one or more descriptors.

[0142] In some implementations, the similar segment look-up engine 1830 determines, for a queried descriptor, or set of descriptors, associated with an image under test a result set that includes other descriptors associated with other images that are determined to be most similar to the queried descriptor, or set of descriptors. This determination is also sometimes referred to herein as “search,” “look-up,” or similar. In some implementations, the other images may include other images associated with a transaction under test, other images associated with other, prior, and / or distinct transactions, or both.

[0143] In some implementations, the other images may include other images associated with a transaction under test. For example, the similar segment look-up engine 1830 searches a mathematics-based descriptor associated with a selfie image under test (e.g., a descriptor of the pose in a selfie image) against a set of analogous mathematics-based descriptors including an analogous mathematics-based descriptor from a second image associated with the transaction under test (e.g., a descriptor of the pose in the document holder image). In some implementations, the other images may be associated with prior transactions distinct from the transaction under test. For example, a mathematics-based descriptor associated with a selfie image under test (e.g., a vector based on the perimeter of the hair sub-segment) is searched against a set of analogous mathematics-based descriptors(e.g., vectors based on the perimeter of the hair sub-segment in other selfies) associated with other selfie images associated with prior transactions.

[0144] In some implementations, the similar segment look-up engine 1830 determines, for a set of descriptors, associated with an image under test a composite result set. In some implementations, the similar segment look-up engine 1830 determines a composite result set based on a composite query. For example, multiple descriptors (e.g., area, shape, frequency distribution in 2D, color distribution, etc.) may be combined into a single query, such as by vector concatenation where the multiple descriptors are vectors. In some implementations, the similar segment look-up engine 1830 iteratively and additively determines a composite result set by performing multiple searches and combines the results. For example, the similar segment look-up engine 1830 performs multiple searches each based on various descriptors associated with the image under test and combining / aggregating the result sets and eliminating duplicates. In some implementations, the similar segment look-up engine 1830 iteratively determines a composite result set filtering / refining an initial result set based on one or more searches using other descriptor(s). For example, the similar segment look-up engine 1830 obtains a first result based on a first descriptor and performs a second search among the second descriptors associated with transactions associated with the first result set using a second descriptor under test.

[0145] In some implementations, the similar segment look-up engine 1830 may perform a match-based look-up for a descriptor to identify a set of exact matches. For example, the similar segment look-up engine 1830 determines whether a textual descriptor of clothing sub-segment 2224 matches textual descriptors of other clothing sub-segments associated with prior transactions.

[0146] In some implementations, the similar segment look-up engine 1830 may perform a similarity -based look-up to identify a set of descriptors most similar to the querieddescriptor(s) under test. In some implementations, the similar segment look-up engine 1830 uses a vector-based similarity to identify a set of results. For example, the similar segment look-up engine 1830 obtains a descriptor that is a vector or vectorizes descriptor (e.g., a textual descriptor or other embedding) and uses the resulting vectorized descriptor under test in the look-up (e.g., as the query or portion thereof).

[0147] In some implementations, the similar segment look-up engine 1830 evaluates the cosine similarity between the descriptor under test (e.g., a vector associated with a subsegment of the image under test) to analogous descriptors (e.g., other vectors associated with another image associated with the transaction under test and / or other transaction, where the other vectors are analogous at least in part because they were derived using the same method and / or analogous content, e.g., hair sub-segment). Cosine similarity is merely one example of a similarity -based look-up method that may be used by the similar segment look-up engine 1830. Other examples that may be used without departing from the disclosure herein include, but are not limited to, Euclidean distance, Manhattan distance, dot product, Minkowski distance, etc.

[0148] In some implementations, the segments and sub-segments that serve as the basis of the search may vary in scope (e.g., the portion of the or subject document) and may not necessarily be distinct. For example, in some implementations, similar segment look-up engine 1830 may perform searches at multiple levels of granularity, such as a first similaritybased look-up based on the subject segment 2202 and at least one similarity -based look-up based on a sub-segment 2220, 2222, 2224 thereof. As another example, in some implementations, the similar segment look-up engine 1830 may perform searches at the most granular layer of segmentation (e.g., hair sub-segment 2220 in the example of Figure 22; a hair sub-sub-segment in a document holder image sub-segment in a document segment of a document image, etc.); etc.

[0149] In some implementations, the variation in granularity / scope may facilitate detection of different types of fraud. For example, when a fraudster modifies small portions of a document (e.g., changing only a name or DOB) a vector descriptor, by the look-up engine 1808, of the document may surface other doctored document images, while a look-up of descriptors associated with sub-segments representing only modified parts (e.g., the name or DOB portions) may be too dissimilar to appear in a result set. In some implementations, a high degree of similarity at a high-level, but low similarity for a more granular part may be determined by the decision engine 1832, described below, as indicative of fraud and / or what portion(s), segments, sub-segments, etc., are fraudulent. Highly similar segments may be used to align the images of the matched transactions in order to perform a difference, in the visual domain or in another space such as color or frequency domain, that may be determined by the decision engine 1832 as indicative of fraud. For example, in case of a face swap, the matched segments may be the hair, the upper body, which are used to align the matched images; by subtracting one image to the other, these highly similar segments are removed and the remainder consist of the face area and possible patches of the background. In some implementations, such difference operation can be performed in the respective embedding space of the segments, thereby highlighting abnormal remainders in a subset of the embedding spaces that can be determined by the decision engine 1832 as indicative of fraud.

[0150] In some implementations, the similar segment look-up engine 1830 may perform a look-up for a reduced set of transaction parts, i.e., a set of transaction parts of interest. In some implementations, the reduced set may be a subset of descriptors associated with an image under test that are determined to be most highly predictive / indicative of fraud (e.g., based on a feature reduction analysis). In some implementations, the reduced set may be used as an initial filter. For example, in some implementations, a first reduced set is searched, the results are used by the decision engine 1832, then based on an initial decisionby the decision engine 1832, additional descriptors may be one or more of generated, searched, and the results analyzed.

[0151] In some implementations, the similar segment look-up engine 1830 may subject a result set to one or more thresholds. For example, in some implementations, the similar segment look-up engine 1830 may require that a minimum measure of similarity threshold descriptor under test and another descriptor associated with another transaction or the transaction under test be satisfied for inclusion of the prior transaction part in the result set. As another example, in some implementations, the similar segment look-up engine 1830 may subject the result set to a maximum, X, number of results, so that only the X most similar descriptors are returned in the result set.

[0152] In some implementations, the result set(s) include a measure of similarity. For example, the percent similarity between the vector describing a hair segment under test to the most similar hair segments in other images.

[0153] In some implementations, the most similar descriptors in the result set(s) are associated segments or sub-segments of one or more underlying prior transactions, occasionally referred to herein as matched transactions. It should be noted that the result set(s) (e.g. a composite result set) may identify matched transactions representing repeated fraud attacks based on commonalities despite variations in the scene (e.g., changes in background clutter), the subject (e.g., having shifted in his / her chair between fraud attempts, or swapped / morphed into a new face), or both.

[0154] In some implementations, the result set(s) are supplemented to include information about the underlying matched transactions. For example, other descriptors associated with the matched transactions, ratio of face-to-background or subject-to- background, metadata (e.g., IP address, a prior decision regarding whether a matching transaction was fraudulent or non-fraudulent, device information, etc.), or other information.

[0155] In some implementations, the similar segment look-up engine 1830 may provide an improved user interface or overcome technical challenges. For example, in some implementations, the similar segment look-up engine 1830 may generate and present the result set(s) to a user. As another example, in some implementations, prior to training the decision engine 1832, the similar segment look-up engine 1830 may generate and present various result set(s) for training data based on image data associated with prior transactions. Obtaining good data, e.g., correctly labeled data, for training is a challenge in the field of machine learning. The result sets generated for prior transactions from the training data as under test may surface repeated fraud attempts in the training data that were missed or not correctly labeled as such. In some implementations, the result set(s) of the similar segment look-up engine 1830 be provided to a labeling tool (not shown) and enhance the mining of results used by the decision engine 1832 in training.

[0156] The similar segment look-up engine 1830 is communicatively coupled with other components of the subject and scene segment analyzer 410B (or components thereof) to provide the result set(s) describing most similar descriptors and / or other data associated with matched transactions thereto. In some implementations, the similar segment look-up engine 1830 may store the result set(s) in the memory 204 (or other data storage such as 228) for retrieval. In some implementations, the similar segment look-up engine 1830 may send the result set(s) to one or more other components of the subject and scene segment analyzer 410B (or components thereof). For example, in some implementations, the similar segment lookup engine 1830 may send the result set(s) to the decision engine 1832.

[0157] The decision engine 1832 generates a signal indicating whether the image(s) associated with a transaction under test are associated with any fraud type (e.g., injection based) attack. In some implementations, the decision engine 1832 or supplemental systems (not shown) determines and / or initiates an action based on the signal generated by thedecision engine 1832. The decision engine 1832 can be used to perform identity verification or ID verification. Examples of actions include, but are not limited to one or more of accept or reject the submission, contact authorities, escalate for investigation or for additional verification checks, etc.

[0158] In some implementations, the action initiated by the decision engine 1832 may or may not be limited to the transaction under test. In some implementations, the decision engine 1832 may take action with respect to a prior, matching transaction. For example, the decision engine 1832 may determine that a prior, matching transaction that was previously determined to be non-fraudulent was erroneous and may initiate a remedial action (e.g., freezing an account, contacting authorities, etc.).

[0159] The decision engine 1832 may be steps, processes, functionalities, software executable by a processor, or a device including routines to generates a signal indicating whether the image(s) are associated with any one of multiple fraud types and whether a particular action should be performed in response to the signal. The decision engine 1832 can output this fraud signal along with other information (e.g., “reason codes” that indicate the reason for the rejection and type of fraud detected, for example, deepfake) to other systems (not shown) or to the server 122 for use by other fraud detection systems.

[0160] In some implementations, the decision engine 1832 may include one or more heuristics used to make decision(s). For example, when the similarity between a pose descriptor of a selfie under test and a document under test is above a certain similarity threshold, decline as fraudulent (e.g., because the face from one was copy -pasted to the other), or when the descriptors associated with the outline of the head and hair exceed a threshold similarity, but descriptors associated with the face are dissimilar, decline as fraudulent (e.g., because that’s indicative of a face morph, face swap, or a deepfake generatedface copied into the subject’s head outline, such as the face portion 1310 in imaged 1304 and 1306 of Figure 13).

[0161] In some implementations, the decision engine 1832 uses machine learning, e.g., to determine one or more of the parameters / criteria and / or values used to make decision(s) The machine learning technique(s) used by the decision engine 1832 to train the one or more machine learning models may vary depending on the implementation and use case, and may include supervised learning, unsupervised learning, semi-supervised learning, etc. The varieties of supervised, semi-supervised, and unsupervised machine learning algorithms that may be used to train the one or more decision models are so numerous as to defy a complete list. Example algorithms include, but are not limited to, tree-based learning algorithms such as random forests or gradient boosted trees; neural network-based methods such as feedforward networks or transformers; support vector machines; and others. In some implementations, ensemble learning techniques, i.e. combining or "stacking" multiple models together to obtain a better performant model, may be used.

[0162] In some implementations, the decision engine 1832 trains and uses a plurality of decision models (not shown). For example, a first model, or set of models, used to analyze a document image under test and a second model or set of models for analyzing selfie images under test; a first model, or set of models, for detecting face swaps and a second model, or set of models, for detecting face morphs; etc., which may or may not rely on a common set of descriptors.

[0163] Depending on the implementation and use case, the segmentation, description generation, search, and analysis scene and subject portions of an image under test may be done together, individually (e.g., alone, in series, or in parallel). In some implementations, the scene-based determiner 1862 determines, based at least in part on the result set(s) associated with the scene, whether the image under test is indicative of fraud. In someimplementations, the subject-based determiner 1864 determines, based at least in part on the result set(s) associated with the subject, whether the image under test is indicative of fraud. In some implementations, the composite fraud score generator 1866 may aggregate a set of model outputs and generate the signal indicating whether the image(s) are fraudulent based on the aggregated result. For example, the composite fraud score generator 1866 aggregates the output of the scene-based determiner 1862 and the subject-based determiner 1864 and optionally other outputs (e.g., liveness detection results, other consistency checks, etc.) to determine whether the image and / or transaction is indicative of fraud.

[0164] In some implementations, the descriptor retention engine 1834 manages the data store 228. In some implementations, the data store 228 stores descriptors for look-up by the similar segment look-up engine 1830. In some implementations, the data associated with the transaction under test is added, by the descriptor retention engine 1834 to the data store, as new prior transaction data.

[0165] In some implementations, descriptor retention engine 1834 may retain the received input image under test (e.g., a selfie or document image), and a set of descriptors associated therewith (i.e., the descriptors describing the image, a segment, a sub-segment, sub-sub-segment, etc., of that image), but may not retain intermediary data, such as the segments or sub-segments themselves. The descriptors are relatively efficient data structures, which may be used for look-up. Retention of the received, prior input images may be used as updates are made (e.g., as the segmentation and / or descriptor generation varies, so that prior images may be (re)segmented or (re)sub-segmented, and / or updated descriptions may be generated in association therewith).

[0166] In some implementations, the descriptor retention engine 1834 manages the data store 228 to facilitate more efficient operation including look-up. In someimplementations, the more efficient operation is based on one or more of indexing and curation (e.g., in a graph data store to leverage pattern association).

[0167] In some implementations, the descriptor retention engine 1834 curates at least a portion of the data store 228. In some implementations, the curation is time-based, decision-based, or a combination thereof. For example, in some implementations, the descriptor retention engine 1834 may maintain descriptors associated with recent transactions. In some implementations, the recent transactions are transactions within a rolling, fixed time period (e.g., within the last 14 days, 180 days, 210 days, 18 months, 6 months, one-year, or other time period). In some implementations, the fixed time period is based on an analysis of fraud patterns. For example, assume that fraudsters usually attempt fraud in batches (e.g., a large number of fraudulent attempts in a short period of time) or in a trickle (e.g., a few attempts every other week). In such a case, the optimum time period may be just long enough to capture a few instances of the typical trickle-type fraudster (e.g., a few months). Setting the time period much beyond that period may have diminishing returns, as the transaction parts data is likely to include at least one if not multiple instance of prior fraudulent transactions, and result in one or more of slower look-ups, larger graphs, and may yield little in improved prediction accuracy at least in some implementations. In some implementations, the recent transactions are the X most recent transactions (e.g., the last 100k, IM, or other number of transactions). As another example, in some implementations, descriptors and images associated with a transaction determined to be fraudulent is retained or retained longer (i.e., a combination of decision-based and time-based curation).

[0168] In some implementations, the descriptor retention engine 1834 indexes the descriptors for more efficient look-up. In some implementations, the indexing incorporates information about nearest or most similar neighbors. For example, in some implementations, the descriptor retention engine 1834 uses Hierarchical Navigable Small World (HNSW) forindex construction and HNSW is used by the similar segment look-up engine 1830 during look-up to obtain result sets. As another example, in some implementations, the descriptor retention engine 1834 uses K-means clustering as an index approach.

[0169] As another example, the systems, methods, features, and functionalities described herein may provide decisions quickly and use less resources (e.g., money, time, or computational resources such as memory, processors cycles, electricity, I / Os, etc.). The use of descriptors may use relatively little memory, may be efficiently processed / searched, the search may enable similarity-based searching by a machine (which is difficult in computer vision), etc.

[0170] In some implementations, the systems, methods, features and functionalities described herein may detect fraudulent activity that may not have been detected by a human (e.g., looking at a transaction in isolation) and / or identify prior transactions that were misidentified, whether by human or computer. For example, a sophisticated user of photo editing may be able to modify a document so that the modification / anomaly is indistinguishable to a human eye, but the systems, methods, features and functionalities described herein may, in some implementations, identify such modifications, if not directly, based on similarity to other transactions (or more specifically one or more descriptors derived from an image associated therewith). As another example, one or more previously successful instances of repeated fraud may be identified based on a transaction under test, and remedial action may be taken.

[0171] It should be noted that, while the foregoing description describes, the subject and scene segmentation analyzers 410A and 410B in the context of the multiple fraud type detection system of Figures 1-4B and multiple fraud type detection may have advantages, the scene segmentation analyzer(s) 410A and / or 410B may be implemented in the context of a stand-alone system and method for fraud detection in some implementations. For example,an implementation where a determination by the decision engine 1832 (or one or more of its components) and based on subject and scene segmentation is solely determinative of an action (e.g., whether to accept or reject) to take with regard to a request.Example Methods

[0172] Figures 6-12 and 25 are flowcharts of example methods 600, 700, 800, 900, 1000, 1100, 1200, and 2500 that may, in accordance with some implementations, be performed by the multiple fraud type detector 226 described above with reference to Figures 1-5 and 18-20. The methods 600, 700, 800, 900, 1000, 1100, 1200, and 2500 of Figures 6-12 and 25, respectively, are provided for illustrative purposes, and many variations exist and are within the scope of the disclosure herein.

[0173] Figure 6 shows an example general method 600 for multiple fraud type detection in accordance with some implementations. The method 600 begins by receiving 602 one or more images associated with the user request. As noted above, this could be a selfie alone, a selfie image as well as a document image, or it could be a plurality of images of either type, or it could be a video. The method 600 continues by receiving 604 device information associated with the user request. For example, this may be the metadata associated with the device that captured the image(s) in block 602. The method 600 continues to detect 606 a first fraud type using the received image or information. For example, this may be any one of the fraud detection types identified above in the antiinjection attack system 326 shown in Figure 4A above. More specifically, the first fraud type may be any detection for one of: a deepfake, a face morph, a face swap, an unidentified injection attack anomaly, an injection attack, a device attack, a partial image search, multimodal liveness, or a face match. The method 600 continues by detecting 608 a second type of fraud using the received image information. The second type of fraud may similarlybe any one of the fraud detection types identified above in the anti -injection attack system 326 shown in Figure 4A above, so long as it is not the same fraud type used in block 606. It should be understood that the method 600 of the present disclosure advantageously provides the ability to detect a plurality of different fraud detection types. The method 600 may continue to detect any number of additional fraud types identified by block 610 in Figure 6. It should be understood that n may be any number greater than 2. Also, the additional fraud types are different than the fraud types detected in blocks 606 and 608. Block 610 in figure 6 is depicted with dashed lines indicating that it and any number of additional fraud type detection steps are optional. Based on each of the fraud types detected in block 606 to block 610, confidence scores are generated for each block, and the method 600 sends the confidence scores for aggregation in block 612. In block 614, the method 600 accepts or rejects the image received in block 602 as genuine based on the combined score generated in block 612. For example, if the aggregated score in block 612 satisfies the threshold, then the image is accepted as genuine whereas if the aggregated score does not satisfy the threshold, the image is rejected as fraudulent.

[0174] Figure 7 shows a second example method 700 for multiple fraud type detection in accordance with some implementations. The method 700 of Figure 7 illustrates a process similar to the architecture of the anti-injection attack system 326 shown in Figure 4A. In particular, the method 700 performs a plurality of checks for different types of fraud in parallel.

[0175] A first parallel process of the method 700 begins in block 702 by receiving an image associated with the user request. For example, this may be a selfie or video. Next, the method 700 analyzes 704 the image with the first model. Next, the method 700 analyzes 706 the image with the second model. This process continues with this any number of additional models analyzing the received image. Then the method 700 analyzes 708 the image with annth model. In some implementations, this first parallel process provides an analysis block for each model that only requires the selfie image or video. For example, the first through nth models may include the deepfake model, the face morph model, the face swap model, the UIA anomaly model, the subject and scene sequence analysis or model, an injection checking model, a device risk checking model, and a multimodal liveness detector. This part of the parallel process completes with the blocks 704 to 708 sending 710 confidence scores for aggregation.

[0176] A second parallel process of method 700 begins in block 720 by receiving a selfie, a document image and device data. Next, the method 700 performs 722 a face match and face analysis using the selfie and the document image. The method 700 continues to check 724 for prior injection attacks using the device data received in block 720. Next, the method 700 checks 726 for device risk using the device data received in block 720. The method 700 continues by generating metadata from the selfie image and the document image and performing 728 metadata checks. The method 700 completes this second parallel process by sending 730 the confidence scores generated in blocks 722, 724, 726, and 728 for aggregation.

[0177] In some implementations, the image analysis performed in blocks 704, 706, 708, 722, 724, 726, 728, and 730 may be performed in several parallel processes in contrast to only the 2 parallel processes depicted in Figure 7.

[0178] The method 700 continues by aggregating 712 the confidence scores from each detector and other check from both parallel processes into an aggregated score. Then the method 700 accepts 714 the images genuine if the aggregated confidence score satisfies a threshold or rejects the images as fraudulent if the aggregated score does not satisfy the threshold. The aggregated score satisfies the threshold, then the image is classified asgenuine. If the aggregated score does not satisfy the threshold, then the image is classified as fraudulent.

[0179] Figure 8 shows a first example method 800 for detecting multiple fraud types in accordance with some implementations. In the example method 800 shown in Figure 8, the anti -injection attack system 326 is used to detect instances where a document holder image is used to create a selfie face with a face swap or morphed selfie face. The method 800 begins by generating 802 a modified image by eliminating the face or other parts of the image. If this is a repeat fraud that was previously detected either automatically or via the customer, the face can be eliminated from the image and the background used to identify an identical match or a high similarity match from a previous attack with the same or similar background. Similarly, the head, the head and neck, the head and neck and shoulders may also be eliminated to improve the similarity match. For example, the subject and scene segmentation analyzer 410 may be used to perform this block. The method 800 proceeds to compare 804 the modified image to identify a high similarity match from a prior attack. It should be noted that one advantage of the present disclosure is that the image need not be identical but can have a similarity above a predefined threshold. In some implementations, this block 804 is performed by the similarity search system 506 of the subject and scene segmentation analyzer 410. The method 800 continues to perform 806 face morph or face swap detection for identification. For example, this block 806 may be performed by analysis of the received image by the face morphed model 404 and / or the face swap model 406. Either or both of these models 404, 406 may be a deep learning model adapted for identification. This approach still works even if you do not know the model / approach that created the face. In some implementations, a camera injection detection software solution can also be used. Next, the method 800 examines 808 the header information of the image for tampering. For example, the examining the image header information may revealtampering using the tool such as Adobe Photoshop. The method 800 continues by comparing 810 information differences between document capture and user image capture. In particular, capture device information can be used to determine whether the document image and the selfie image were captured with the same device. For example, the impurities in the silicon of the camera between document capture and image capture. The method 800 continues by using 812 a model to detect identical user poses. The model may detect instances when the holder image of the document is a straight copy onto the selfie. Next, the method 800 accepts or rejects 814 the image as genuine based on the analysis performed above in blocks 804, 806, 808, 810 and 812.

[0180] Figure 9 shows a second example method 900 for detecting multiple fraud types in accordance with some implementations. In this second example, the method 900 is detecting instances where the fraudster has used a selfie image to create a document holder image. The method 900 detects 902 manipulation of the holder image by detecting the abuse of a security feature. One example of this is shown below Figure 13 in which the fraudster abused the document security feature(s) e.g., microprint omitted in areas of the face of the user where the image was swapped out. The method 900 continues by detecting 904 an identical prior expression of the user in an image using a detection model. Next, the method 900 detects 906 and identical prior pose with a detection model. The pose and expression identical to the pose and expression used in a prior image can be detected using a detection module. The method 900 continues by accepting or rejecting 908 the image as genuine based on the evaluations done in blocks 902, 904, and 906.

[0181] Figure 10 shows a third example method 1000 for detecting multiple fraud types in accordance with some implementations. In this third example, the method 1000 is used to detect instances when the fraudster has generated deepfake on both the document holder image and the selfie. The method 1000 begins by detecting 1002 manipulation of theholder image by detecting abuse of the security feature on the holder image. Next, the method 1000 detects 1004 identical prior expressions or poses with a detection model. These may be localized facial expression manipulation. This detection 1004 can be applied to both the selfie and the document image. The method 1000 continues by detecting 1006 physical traits of the user (e.g., humans) to detect deepfake. For example, fraud detection models can be combined with software that creates a video sequence that can react to different prompts, e.g., to move towards the camera, or away from the camera. The model is then used to detect physical human traits such as eye blinking or artificial blinking patterns in deepfakes. Again, this detection 1006 may be applied to both the selfie and the document image. The method 1000 continues by accepting or rejecting 1008 the image as genuine based on the evaluations done in blocks 1002, 1004 and 1006.

[0182] It should be understood that the rejection of any images in these example detection methods 800, 900, 1000, and 1100 may be used to augment the information used by the detection models. Often fraudsters perform repeated attacks and reuse document images and selfie images with just a subset of the PII fields changed. In this case, the subject and scene segmentation analyzer 410 (partial image search) can use the background and document position to increase the accuracy of detecting fraudulent documents. The additional optional steps 1010, 1012, and 1014 depicted in Figure 10 illustrate one example of how that information can be augmented. Once an image is rejected in block 1008, the method 1000 may continue to perform 1010 a partial image search on a subset a background and document labels. Then the method 1000 confirms 1012 that the subset of background and document labels are fraudulent. The method 1000 then adds 1014 the subset a background in document labels to a database of fraudulent images.

[0183] Figure 11 shows a fourth example method 1100 for detecting multiple fraud types in document images in accordance with some implementations. The method 1100 firstreceives a document image. The method 1100 creates 1102 one or more modified images by masking out personal identifiable information (PII) data from images that have been determined to be repeatedly used by fraudsters. For example, ID documents repeatedly used by fraudsters would have the PII masked out. By masking the PII data out, it produces a detecting model free of noise added by the PII data making it more accurate in the detection of fraudulent documents. Next, the method 1100 trains or creates 1104 models using the masked images generated in block 1102. The method 1100 continues by applying 1106 a deepfake detection model to the document image. For example, the deepfake model 402 as described above can be applied to the document image. Then the method 1100 applies 1108 a copy manipulation detection model to the document image. For example, a model developed on synthetically generated ID cards can be used for this step. In another example, the detector masks out all the PII data on incoming document image and compares it using either a model or unsupervised anomaly detection method from previously obtained valid documents of this type. The valid document abstraction to be used for comparison is held in the document assembly object. The method 1100 continues by comparing 1110 a modified image with an anomaly detection model. Then the method 1100 accepts or rejects 1112 the image as genuine based on the evaluations done in blocks 1106, 1108, and 1110.

[0184] Figure 12 shows an example method 1200 for training models to detect multiple fraud types in accordance with some implementations. In order to train the deep learning model, a critical part is to create training datasets that mimic the popular attack patterns in public and in production. The present disclosure advantageously generates training data from for the deepfake detection model is meticulously curated from two primary sources, each serving a distinct purpose. Training multi-frame (video) deepfake detection models requires a dataset or datasets of considerable size and variety that are generally deepfake videos. Several datasets from Meta, Google, and others can be used for suchtraining. The goal of method 1200 is to build a detection model that generalizes better to unseen deepfakes and different datasets helps with this issue. Training for a single frame deepfake detection model can follow the same process but also a different training method can be used. For example, model parsing can be used which is a reverse engineering approach which uncovers the unique patterns behind the Al model used. The method 1200 begins by receiving 1202 instances of images confirmed to be deepfakes. For example, authentic instances of deepfakes encountered in diverse real-world scenarios can be retrieved from the data store 228. Then the method 1200 generates 1204 production-mined data points from the image instances received in block 1202. In some implementations, these production-mined data points are generated by automated processes or software that searches and analyzes the image instances in order to identify patterns and extract useful information. These automated processes or software implement workflows (e.g., deepfake detector, repeated fraud match, partial image search, selfie subject and scene segmentation, etc.). For example, a single frame-based face swap / face morph can be identified from an API channel or real-time deepfake video injection via virtual camera. In other implementations, the production-mined data points are obtained from human audits of the image instances. In yet other implementations, the production-mined data points are combination of data points generated automatically and data points generated by human audits. These production-mined data points are invaluable, capturing the nuances and intricacies of genuine deepfake scenarios observed in various productions. Next, the method 1200 generates 1206 a first portion of the training data set from the production-mined data points. In this implementation, a second portion of the training data set is self-generated data. The selfgenerated data set includes a first part of single frame deep fakes and the second part of video-based defects. The method 1200 continues by using deepfake generation tools to produce 1208 a first part of a self-generated data set including user images or selfies. Forexample, there are tools for generating single frame deepfakes and tools available for generating deepfake videos. The output from these tools can be used as part of the dataset for the respective models training. This block 1208 uses both open-source and commercial tools to create digitally manipulated selfies. This block 1208 involves manipulating one or a pair of images to produce a single, high-fidelity output image. For example, the face swap / face morph techniques may be employed to generate the single frame deep fakes. Next, the method 1200 uses deep fake generation tools on video to create deepfake-infused content. These video-based deep fakes are the second part of the self- generated data set. Block 1210 involves manipulating real-time or recorded videos to create deepfake-infused content. It should be noted that the self-generated data set produced by block 1208 and 1210 are advantageously designed to replicate and expand upon the characteristics found in the detection of fraudulent images in production. In particular, the self-generated data set intentionally mirrors the images encountered in the real world. Next, the method 1200 combines 1212 the first portion of the training data set (production mined data points) with the second portion of the training data set (first and second parts of self-generated data) to create the final training data set. This is ensured to be a comprehensive and representative training data set. Next, the method 1200 uses the training data set to train 1214 a new model. The method 1200 may optionally augment 1216 the training data set. For example, the method 1200 may adaptively use additional tools and images to augment 1214 the training data set. The multiple fraud type detector 226 advantageously modifies and updates selfgenerated dataset continuously or periodically to match images observed during operation / production because of the dynamic nature of the deepfake landscape. This allows the training data set to be modified to mirror the ever-evolving characteristics of deepfakes encountered in the real world.

[0185] Figure 13 shows an example of a process 1300 used to generate a deepfake. One image 1302 is an artificial intelligence (Al) generated facial image (selfie). As has been noted above, this type of deepfake image can be easily generated using a variety of software tools that have now become widely available. The fraudster copies a portion (the face of the selfie) of the image 1302 onto an image of an ID card (not shown) to generate image 1304. For example, the fraudster can simply copy or photoshop a portion of the Al generated facial image 1302 onto an image of the ID card to produce image 1304. This image 1304 can be detected by the anti-injection attack system 326 described above as a fraudulent document and reject it. Image 1306 is a blowup of a portion of the image 1304 of the ID card showing the facial image in more detail. In particular, as shown in image 1306, the background portion 1308 of the image 1306 has a microprint security feature across the image as expected. However, the portion 1310 of the image 1306 that the fraudster has copied from 1302 can be seen as not having the microprint security feature on the majority of the face. A valid ID would have the microprint covering the entire face of the person in the holder image, but in this case, the microprint is absence in the center of the holder image face up to the middle of the person’s forehead. In some implementations, the subject and scene segmentation analyzer 410 is able to detect the absence of the microprint security feature in the cardholder image 1304. In other implementations, a modeling approach may be used instead of a similarity search. For example, a CNN or anomaly detection model could detect the absence of microprint located in the face region. It should be noted that the subject and scene segmentation analyzer 410 may also detect common portions of the image (e.g., background) that show that the hand and the card position are identical in many attacks.Since the hand and card position never change in multiple fraudster submissions, the subject and scene segmentation analyzer 410 using a partial image search can detect it is repeat fraud.

[0186] Figure 14 shows an example set of images 1402, 1404 and 1406 in which two images 1402 and 1404 are face morphed 1400 to produce a third image 1406. Figure 14 shows how a nefarious user has morphed an image of the nefarious user (not shown) to make his facial features more similar to those of various, valid document holder’s facial image and the resulting deepfake image 1406. Comparing Figure 14 to Figure 15 illustrates the difference between a face morph and a face swap is shown. In these two examples, the same base images 1402, 1404 were used to generate the resulting images 1406, 1506. In the case of a face swap (Figure 15), if you were to mask out everything but the central face oval 1514 of the image, you would immediately recognize it as the face from image 1404. In the case of a face morph (Figure 14), it is an image 1406 with elements of both of the entire heads of images 1402 and 1404. The edges (for example hair of image 1402) make it hard to see this when you are looking at the entire morphed image 1406. Since the user in image 1404 is follicle-challenged compared to the user in image 1402, the morphing algorithm splits the difference and gives the resulting image 1406 with a much shorter hair cut smoothing the difference between the two entire heads in images 1402 and 1404. You can see the difference when images 1406 and 1506 viewed side by side. However, another face morphing algorithm may morph a more limited set of features (e.g., may leave the silhouette of the head or the hair unmodified.

[0187] Figure 15 shows an example set of images 1402, 1404 and 1506 used to perform the process 1500 of face swapping. In this example, the fraudster begins with the first image 1402 and a second image 1404. As shown, the first image 1402 includes a background portion 1508 and a face portion 1510. Similarly, the second image 1404 includes a background portion 1512 and a face portion 1514. The fraudster uses the background portion 1508 of the first image 1402 and combines it with the face portion 1514 of the second image 1404 to generate the third new image 1506. For example, the fraudster couldphotoshop the face portion 1514 onto image 1402 to generate the new image 1506. This illustrates how simple creating a new image is given the software tools that are readily available.

[0188] Figure 16 is a diagram illustrating how face morphing and face swapping can be combined into a process 1600 for yet another way to generate a deepfake image. The process begins with face landmark detection 1602 from a first image 1604 and a second image 1606. The first image 1604 includes a face area 1608, face 1. The second image 1606 also includes a face area 1610, face 2. Both these areas are detected for future use. The process 1600 applies a target mask 1612 to the frame, in this case, the Web camera frame with image 1604. Then the process 1600 proceeds to generate a morph 1614 of face 1 1608 and face 2 1610. The process 1600 then applies the mask 1612 to the second image 1606 and replaces the mask 1612 with the morph 1614 generated from phase 1 and phase 2. By understanding how the tools available work to create deepfakes, face morphs, and face swaps, the anti -injection attack system 326 of the present disclosure advantageously can include in its architecture any detection techniques used to determine the new anomalies as they are created by the fraudsters. As an example, this face morph technique changes only the center face area and the forehead, hairline, neck, etc. are left unchanged and can be matched by a similarity match if this is one of a repeat attack using the same scene and subject but with a face change.

[0189] Referring now to Figure 17A, an example set of images 1702, 1704, 1710, and 1720 are used to show how partial image search and image segmentation 1700 is performed. It should be understood that Figurel7A is merely one example of a way in which an image may be segmented, and there are a variety of other ways or features upon which to segment the image. This is the segmentation that is done prior to the partial image search. The subject and scene segmentation analyzer 410 advantageously allows a subject to besegmented in any number of different ways to determine the portions of the subject that may be repeated and are real and valuable in identifying them. The segmentation also allows identification of areas which should not be identical and repeated in an image. Image 1702 is an example of a beginning image. In this example, the beginning image 1702 is segmented into three different portions: a background, the body, and the face. The second image 1704 shows the segmentation of the background 1706 versus the remainder of the image 1708. The third image 1710 shows the segmentation of the body 1714 portion of the subject versus the remainder of the image 1712. Finally, the fourth image 1720 illustrates the face 1724 portion of the image versus the remainder of the image 1722. It should be understood that the example of Figure 17A is merely one example of the many possibilities for segmenting subject and scene to improve similarity matches.

[0190] Figure 17B shows an example set of images 1752, 1754, 1756, and 1758 having similar backgrounds that can be detected using partial image search and segmentation. The images 1752, 1754, 1756, and 1758 show examples of facial images (e.g., selfies) with similar, or identical backgrounds. This set of images 1752, 1754, 1756, and 1758 shows the strength of the subject and scene segmentation analyzer 410 because even though the backgrounds 1760a, 1760b, 1760c, and 1760d change slightly image to image, a similarity match is still found by the subject and scene segmentation analyzer 410. In each of the images 1752, 1754, 1756, and 1758, the faces 1762a, 1762b, 1762c, and 1762d are very different as the fraudster has swapped in different faces into versions of the base image. In other implementations, the fraudster may perform injection on the same selfie or may generate a series of injected selfies (i.e., selfies generated using injection serially during a single session so that the nefarious user’s background, clothes, etc. are similar, but may not be identical). The subject and scene segmentation analyzer 410 advantageously identifies repeated fraudulent attempts that use similar, but not identical images, which havehistorically presented a challenge for prior art detection systems. For example, existing methods using hashes may determine identicality, but not similarity that does not rise to the level of identicality. It should be noted that since the face is the only significant change in the images, the head shape and hair line and outline of the head or other portions are identical so they can also be segmented out, measured and a representation generated; and the representation used in a similarity match as another fraud signal in addition to background similarity or other portions of similarity.

[0191] Figure 25 shows an example general method 2500 for segmented subject and scene partial image search fraud detection in accordance with some implementations. The method 2500 begins by receiving, at block 2502, image data representing a subject and a scene, which may be associated with a transaction. As noted above, this could be a selfie alone, a selfie image as well as a document image, or it could be a plurality of images of either type, or it could be a video. The method 2500 continues by segmenting, at block 2504, the image data into the subject segment(s) and the scene segment(s). At block 2506, one or both of the subject and scene segments are sub-segmented. However, in some implementations, the sub-segments obtained from block 2506 may be obtained directly without first segmenting the image into a subject and scene at block 2504. The method 2500 continues by generating, at block 2508, a set of descriptors associated with at least some of the sub-segments generated at block 2506. As noted above, the composition of the descriptors may vary depending on the implementation and use case, e.g., the variations are so numerous as to defy a complete listing but examples include, but are not limited to: one or more of at least one mathematics-based descriptor and at least one text-based descriptor; one or more of at least one descriptor associated with the subject and at least one descriptor associated with the scene; a set of descriptors associated with / derived from the subject’s hair, head shape and upper torso, etc. The method 2500 continues by obtaining 2510 a set ofcomposite search results associated with other images with subsegment(s) most similar to those of the first image, as determined by their respective descriptors. At block 2512, a determination is made, using a first machine learning model, whether the first image data is likely fraudulent based on the composite search result.

[0192] The method 2500 of Figures 25 is provided for illustrative purposes, and many variations exist and are within the scope of the disclosure herein. For example, block 2512, may provide an initial decision (e.g., inconclusive, likely fraudulent, first pass: failed or passed, etc.) and one or more of blocks 2504-2512 may be repeated (e.g., to generate additional sub-segments or encodings and associated search results) to come to a final conclusion, identify the type of fraud (e.g., injection attack), identify the fraudulent portion(s) of the image under test, etc. In some implementations, the orchestration of such repeated, or layered, may be controlled by the value of a fraud score, the factors of decision, or a combination thereof. The orchestration system may rely on heuristics, machine learning or LLM-based agents depending on the implementation.Other Considerations

[0193] It should be understood that the above-described examples are provided by way of illustration and not limitation and that numerous additional use cases are contemplated and encompassed by the present disclosure. In the above description, for purposes of explanation, numerous specific details are set forth in order to provide a thorough understanding of the present disclosure. However, it should be understood that the technology described herein may be practiced without these specific details. Further, various systems, devices, and structures are shown in block diagram form in order to avoid obscuring the description. For instance, various implementations are described as having particular hardware, software, and user interfaces. However, the present disclosure applies to any typeof computing device that can receive data and commands, and to any peripheral devices providing services.

[0194] Reference in the specification to “one implementation” or “an implementation” or “some implementations” means that a particular feature, structure, or characteristic described in connection with the implementation is included in at least one implementation. The appearances of the phrase “in some implementations” in various places in the specification are not necessarily all referring to the same implementations.

[0195] In some instances, various implementations may be presented herein in terms of algorithms and symbolic representations of operations on data bits within a computer memory. An algorithm is here, and generally, conceived to be a self-consi stent set of operations leading to a desired result. The operations are those requiring physical manipulations of physical quantities. Usually, though not necessarily, these quantities take the form of electrical or magnetic signals capable of being stored, transferred, combined, compared, and otherwise manipulated. It has proven convenient at times, principally for reasons of common usage, to refer to these signals as bits, values, elements, symbols, characters, terms, numbers, or the like.

[0196] It should be borne in mind, however, that all of these and similar terms are to be associated with the appropriate physical quantities and are merely convenient labels applied to these quantities. Unless specifically stated otherwise as apparent from the following discussion, it is appreciated that throughout this disclosure, discussions utilizing terms including “processing,” “computing,” “calculating,” “determining,” “displaying,” or the like, refer to the action and processes of a computer system, or similar electronic computing device, that manipulates and transforms data represented as physical (electronic) quantities within the computer system’s registers and memories into other data similarlyrepresented as physical quantities within the computer system memories or registers or other such information storage, transmission or display devices.

[0197] Various implementations described herein may relate to an apparatus for performing the operations herein. This apparatus may be specially constructed for the required purposes, or it may comprise a general-purpose computer selectively activated or reconfigured by a computer program stored in the computer. Such a computer program may be stored in a computer readable storage medium, including, but is not limited to, any type of disk including floppy disks, optical disks, CD-ROMs, and magnetic disks, read-only memories (ROMs), random access memories (RAMs), EPROMs, EEPROMs, magnetic or optical cards, flash memories including USB keys with non-volatile memory or any type of media suitable for storing electronic instructions, each coupled to a computer system bus.

[0198] The technology described herein can take the form of a hardware implementation, a software implementation, or implementations containing both hardware and software elements. For instance, the technology may be implemented in software, which includes but is not limited to firmware, resident software, microcode, etc. Furthermore, the technology can take the form of a computer program product accessible from a computer- usable or computer-readable medium providing program code for use by or in connection with a computer or any instruction execution system. For the purposes of this description, a computer-usable or computer-readable medium can be any non-transitory storage apparatus that can contain, store, communicate, propagate, or transport the program for use by or in connection with the instruction execution system, apparatus, or device.

[0199] A data processing system suitable for storing and / or executing program code may include at least one processor coupled directly or indirectly to memory elements through a system bus. The memory elements can include local memory employed during actual execution of the program code, bulk storage, and cache memories that provide temporarystorage of at least some program code in order to reduce the number of times code must be retrieved from bulk storage during execution. Input / output or I / O devices (including but not limited to keyboards, displays, pointing devices, etc.) can be coupled to the system either directly or through intervening I / O controllers.

[0200] Network adapters may also be coupled to the system to enable the data processing system to become coupled to other data processing systems, storage devices, remote printers, etc., through intervening private and / or public networks. Wireless (e.g., WiFi™) transceivers, Ethernet adapters, and modems, are just a few examples of network adapters. The private and public networks may have any number of configurations and / or topologies. Data may be transmitted between these devices via the networks using a variety of different communication protocols including, for example, various Internet layer, transport layer, or application layer protocols. For example, data may be transmitted via the networks using transmission control protocol / Internet protocol (TCP / IP), user datagram protocol (UDP), transmission control protocol (TCP), hypertext transfer protocol (HTTP), secure hypertext transfer protocol (HTTPS), dynamic adaptive streaming over HTTP (DASH), realtime streaming protocol (RTSP), real-time transport protocol (RTP) and the real-time transport control protocol (RTCP), voice over Internet protocol (VOIP), file transfer protocol (FTP), WebSocket (WS), wireless access protocol (WAP), various messaging protocols (SMS, MMS, XMS, IMAP, SMTP, POP, WebDAV, etc ), or other known protocols.

[0201] Finally, the structure, algorithms, and / or interfaces presented herein are not inherently related to any particular computer or other apparatus. Various general-purpose systems may be used with programs in accordance with the teachings herein, or it may prove convenient to construct more specialized apparatus to perform the required method blocks. The required structure for a variety of these systems will appear from the description above. In addition, the specification is not described with reference to any particular programminglanguage. It will be appreciated that a variety of programming languages may be used to implement the teachings of the specification as described herein.

[0202] The foregoing description has been presented for the purposes of illustration and description. It is not intended to be exhaustive or to limit the specification to the precise form disclosed. Many modifications and variations are possible in light of the above teaching. It is intended that the scope of the disclosure be limited not by this detailed description, but rather by the claims of this application. As should be understood by those familiar with the art, the specification may be embodied in other specific forms without departing from the spirit or essential characteristics thereof. Likewise, the particular naming and division of the modules, routines, features, attributes, methodologies, and other aspects are not mandatory or significant, and the mechanisms that implement the specification or its features may have different names, divisions and / or formats.

[0203] Furthermore, the modules, routines, features, attributes, methodologies, engines, and other aspects of the disclosure can be implemented as software, hardware, firmware, or any combination of the foregoing. Also, wherever an element, an example of which is a module, of the specification is implemented as software, the element can be implemented as a standalone program, as part of a larger program, as a plurality of separate programs, as a statically or dynamically linked library, as a kernel loadable module, as a device driver, and / or in every and any other way known now or in the future. Additionally, the disclosure is in no way limited to implementation in any specific programming language, or for any specific operating system or environment. Accordingly, the disclosure is intended to be illustrative, but not limiting, of the scope of the subject matter set forth in the following claims.

Claims

WHAT IS CLAIMED IS:

1. A computer-implemented method comprising: receiving a first image data associated with a user; performing a first detection type on the first image data to generate a first signal; performing a second detection type on the first image data to generate a second signal; generating an aggregated fraud score based upon the first signal and the second signal; and accepting the first image data as genuine based upon the aggregated fraud score.

2. The computer-implemented method of claim 1 further comprising: rejecting the first image data as fraudulent based upon the aggregated fraud score.

3. The computer-implemented method of claim 1 wherein the first detection type is one from a group of: a deepfake model, a face morph model, a face swap model, an unknown injection attack anomaly model, a subject and scene segmentation analyzer, an injection checker, a device risk checker, a liveness detector, a face match and face analysis subsystem, an injection attack detector and a device risk detector.

4. The computer-implemented method of claim 1 wherein the second detection type is one from a group of: a deepfake model, a face morph model, a face swap model, an unknown injection attack anomaly model, a subject and scene segmentation analyzer, an injection checker, a device risk checker, a liveness detector, a face match and face analysis subsystem, an injection attack detector and a device risk detector.

5. The computer-implemented method of claim 1 wherein the first detection type is a multi-frame deepfake detection model, and the second detection type is a single frame deepfake detection model, and wherein the method further comprises: performing a third detection type on the first image data to generate a third signal, wherein the third detection type is a subject and scene segmentation analysis; and wherein the aggregated fraud score is generated based upon the first signal, the second signal, and the third signal.

6. The computer-implemented method of claim 5 wherein performing the third detection type on the first image data further comprising: generating a vector embedding for the first image data; accessing a matrix of vector embeddings; searching for cosine similarities between embeddings in the matrix and the generated vector embedding; and generating the third signal based on the searching for the cosine similarities.

7. The computer-implemented method of claim 1 wherein the first image data is one from a group of a selfie and a video.

8. The computer-implemented method of claim 1 wherein the first image data includes a selfie image, and the method further comprises: receiving a document image; performing a first detection type on the document image to generate a third signal; performing a second detection type on the document image to generate a fourth signal; andwherein the generating the aggregated fraud score is also based on third signal and the fourth signal.

9. The computer-implemented method of claim 1 wherein the first image data includes a selfie image, and the method further comprises: receiving a document image; extracting selfie metadata from the selfie image; extracting document metadata from the document image; and wherein the generating the aggregated fraud score is also based on the selfie metadata and the document metadata.

10. The computer-implemented method of claim 1 wherein the first image data includes a selfie image and a document image, and the method further comprises performing a third detection type using the selfie image and the document image to generate a third signal, wherein the third detection type is a face match and face analysis that compares the selfie image to a holder portion of the document image and generates the third signal based on a match between the selfie image and the holder portion of the document image.

11. The computer-implemented method of claim 1 wherein the accepting the first image data as genuine includes comparing the aggregated fraud score to a threshold, and accepting the first image data if the aggregated fraud score satisfies the threshold.

12. The computer-implemented method of claim 1, further comprising: performing a third detection type on the first image data to generate a third signal; and wherein the first image data is rejected as genuine based upon only the third signal.

13. A system comprising:one or more processors; and a memory operably coupled with the one or more processors, wherein the memory stores instructions that, in response to execution of the instructions by one or more processors, cause the one or more processors to: receive a first image data associated with a user; perform a first detection type on the first image data to generate a first signal; perform a second detection type on the first image data to generate a second signal; generate an aggregated fraud score based upon the first signal and the second signal; and accept the first image data as genuine based upon the aggregated fraud score.

14. The system of claim 13, wherein the instructions cause the one or more processors to reject the first image data as fraudulent based upon the aggregated fraud score.

15. The system of claim 13, wherein the first detection type is one from a group of: a deepfake model, face morph model, a face swap model, an unknown injection attack anomaly model, a subject and scene segmentation analyzer, an injection checker, a device risk checker, a liveness detector, a face match and face analysis subsystem, an injection attack detector, and a device risk detector.

16. The system of claim 13, wherein the second detection type is one from a group of: a deepfake model, face morph model, a face swap model, an unknown injection attack anomaly model, a subject and scene segmentation analyzer, an injection checker, a device risk checker, a liveness detector, a face match and face analysis subsystem, an injection attack detector and a device risk detector.

17. The system of claim 13, wherein the first detection type is a multi-frame deepfake detection model, and the second detection type is a single frame deepfake detection model, and wherein the instructions cause the one or more processors to: perform a third detection type on the first image data to generate a third signal, wherein the third detection type is a subject and scene segmentation analysis; and wherein the aggregated fraud score is generated based upon the first signal, the second signal, and the third signal.

18. The system of claim 14, wherein the instructions cause the one or more processors to: generate a vector embedding for the first image data; access a matrix of vector embeddings; search for cosine similarities between embeddings in the matrix and the generated vector embedding; and generate a third signal based on the search for cosine similarities.

19. The system of claim 13, wherein the first image data is one from a group of a selfie and a video.

20. The system of claim 13, wherein the first image data includes a selfie image, and the instructions cause the one or more processors to: receive a document image; perform a first detection type on the document image to generate a third signal; perform a second detection type on the document image to generate a fourth signal; andwherein the generation of the aggregated fraud score is also based on third signal and the fourth signal.

21. The system of claim 13, wherein the first image data includes a selfie image, and the instructions cause the one or more processors to: receive a document image; extract selfie metadata from the selfie image; and extract document metadata from the document image, and wherein the generation of the aggregated fraud score is also based on the selfie metadata and the document metadata.

22. The system of claim 13 wherein the first image data includes a selfie image and a document image, and wherein the instructions cause the one or more processors to perform a third detection type using the selfie image and the document image to generate a third signal, wherein the third detection type is a face match and face analysis that compares the selfie image to a holder portion of the document image and generates the third signal based on a match between the selfie image and the holder portion of the document image.

23. The system of claim 13 wherein the acceptance the first image data as genuine includes comparing the aggregated fraud score to a threshold, and accepting the first image data if the aggregated fraud score satisfies the threshold.

24. The system of claim 13, wherein the instructions cause the one or more processors to: perform a third detection type on the first image data to generate a third signal; and wherein the first image data is rejected as genuine based upon only the third signal.

Images