Detection of deepfake documents using custom neural networks

Abstract

A method to verify an identity document using a machine learning model. The method may comprise receiving an image of the identity document by a computing system, extracting, the image from the identity document, preprocessing, the image prior to providing the image to the machine learning model and providing the preprocessed image to the machine learning model as input. The machine learning model may analyze the input to detect whether the image was synthetically generated. The machine learning model may comprise a convolution layer configured to extract spatial features from the image; a batch normalization layer configured to stabilize a feature map; and a dense layer for classification. An output of the machine learning model may be generated. The output may be an image verification score indicating a likelihood that the image was synthetically generated. Based on the score, an authentication method or a denial message may be generated.

Description

Attorney Docket No.: 096923-1458070 Client Ref: EFX-195WODETECTION OF DEEPFAKE DOCUMENTS USING CUSTOM NEURAL NETWORKSTECHNICAL FIELD

[0001] The present disclosure relates generally to identity verification and document authentication. More specifically, but not by way of limitation, this disclosure relates to systems and methods that leverage artificial intelligence (Al) for detecting deepfake-generated images in identity documents.BACKGROUND

[0002] A deepfake is a type of realistic-looking media that has been created using artificial intelligence (Al) to appear real or otherwise authentic. For example, deepfakes can be videos, photos, or audio recordings that replace or alter a person's face or body with someone else. They may also be entirely synthetically created faces or personas, without being based on a real person. The rise of deepfakes, particularly in identity documents, presents significant security challenges across industries, including banking, government services, and online platforms that rely on secure identity verification.

[0003] With the increasing sophistication of artificial intelligence, particularly in the generation of deepfake images, traditional identity verification methods are becoming less effective. Fraudsters can manipulate images in identity documents to such a degree that visual inspection, software, and even experts are unable to reliably detect forgeries.SUMMARY

[0004] Various aspects of the present disclosure provide a system and methods for enhancing identity verification by employing artificial intelligence to detect deepfake manipulations in identity documents. In some examples, the methods may include extraction and validation of personally identifiable information (PII) from the identity document. The system can receive a request to validate an identity document. In some aspects, the system may access an analysis module, which may combine a deep learning-based model (e.g., a convolutional neural network (CNN) such as VGG16) with other analysis techniques, such as optical character recognition (OCR to ensure accurate detection of manipulated images contained within an identity document (e.g., a deepfake image), reliable extraction of key information from various types of identity documents,1US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO and a comparison of those features to produce a metric related to the identity document. The system can determine a risk indicator related to the provided identity document by using the trained machine learning or the analysis module, or any combination thereof. The system can transmit, to a remote computing device, a responsive message comprising at least the risk indicator for use in controlling access to one or more interactive computing environments.

[0005] Aspects of the disclosed technology include a system of one or more computers that can be configured to perform particular operations or actions by virtue of having software, firmware, hardware, or a combination of them installed on the system that in operation causes or cause the system to perform the actions. One or more computer programs can be configured to perform particular operations or actions by virtue of including instructions that, when executed by data processing apparatus, cause the apparatus to perform the actions.

[0006] One aspect of the disclosed technology includes verifying an identity document using a machine learning model. The verification may include, receiving by the processor, an identity document. An image may be extracted from the identity document. The extracted image may be preprocessed prior to providing the image to the machine learning model. The preprocessed image may be provided to the machine learning model as input. The machine learning model may analyze the input to detect whether the image was synthetically generated. The machine learning model may include a convolution layer configured to extract spatial features from the image, a batch normalization layer configured to stabilize a feature map, and a dense layer configured to output a probability related to the image. The verification may include generating, as an output of the machine learning model, an image verification score indicating a likelihood that the image was synthetically generated. The verification may include upon determining that the image verification score is higher than a predefined threshold, generating an authentication message configured to cause a receiving device to take an action based on receipt of the authentication message. The verification may include upon determining that the image verification score is lower than the predefined threshold, generating a denial message configured to cause the receiving device to take an alternative action based on the failure of authentication.

[0007] Aspects of the disclosed technology may include one or more of the following features.The method where the synthetically generated image is a deepfake generation. Preprocessing the image may include resizing the image, normalizing pixel values of the image, performing data 2US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO augmentation on the image, adjusting brightness and contrast of the image, or any combination thereof. The method may include: extracting, by the processor, personally identifiable information (pii) from the image using optical character recognition; validating, by the processor, the extracted pii against a trusted external data source to generate a pii validity score; and calculating, by the processor, a final verification score by combining the image verification score and the pii validity score based on predefined weights. The final verification score is compared against a predefined threshold to determine whether the identity document is genuine. The method may include providing the final verification score to a third-party computing system. The method may include generating an image authenticity score by comparing the image of the identity document with a reference image of an individual associated with the identity document. Implementations of the described techniques may include hardware, a method or process, or computer software on a computer-accessible medium.

[0008] Additional aspects of the disclosed technology include corresponding computer systems, apparatus, and computer programs recorded on one or more computer storage devices. One general aspect includes a system for verifying an identity document. The system may include a processor. The system may include a memory storing instructions that, when executed by the processor, cause the system to: receive, by a processor, an image of the identity document by a computing system; extract, by the processor, the image from the identity document; preprocess, by the processor, the image prior to providing the image to the a machine learning model; provide, by the processor, the preprocessed image to the machine learning model as input; analyze, using the machine learning model, the input to detect whether the image was synthetically generated, the machine learning model may include: a convolution layer configured to extract spatial features from the image; a batch normalization layer configured to stabilize a feature map; and a dense layer configured to output a probability related to the image. The system may include generate, as an output of the machine learning model, an image verification score indicating a likelihood that the image was synthetically generated; and upon determining that the image verification score is higher than a predefined threshold, generating an authentication message configured to cause a receiving device to take an action based on receipt of the authentication message; or upon determining that the image verification score is lower than the predefined threshold, generating a3US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO denial message configured to cause the receiving device to take an alternative action based on the failure of authentication.

[0009] This summary is not intended to identify key or essential features of the claimed subject matter, nor is it intended to be used in isolation to determine the scope of the claimed subject matter. The subject matter should be understood by reference to appropriate portions of the entire specification, any or all drawings, and each claim.

[0010] The foregoing, together with other features and examples, will become more apparent upon referring to the following specification, claims, and accompanying drawings.BRIEF DESCRIPTION OF THE DRAWINGS

[0011] FIG. 1 illustrates an overview of a verification system architecture, according to some aspects of the present disclosure.

[0012] FIG. 2 depicts a deepfake detection model architecture, according to some aspects of the present disclosure.

[0013] FIG. 3 is a flowchart of a training process for a deepfake detection model, according to some aspects of the present disclosure.

[0014] FIG. 4 is a flowchart of an end-to-end document verification process, including score generation and final output, according to some aspects of the present disclosure.

[0015] FIG. 5 illustrates an example computing system, according to some aspects of the present disclosure.DETAILED DESCRIPTION OF THE INVENTION

[0016] Certain aspects and examples of the present disclosure relate to systems and methods that leverage artificial intelligence (Al) for detecting deepfake-generated images in identity documents. The described systems and methods address challenges associated with Al-powered deepfake detection models with advanced PII extraction and scoring mechanisms to improve the overall reliability and security of identity verification processes. The present disclosure allows for verification of identity documents through the use of machine learning models, particularly deep learning techniques. This system is designed to detect and prevent fraud by identifying manipulated images, including those generated using deepfake technologies, and accurately4US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO extracting and validating personally identifiable information (PII) from various types of identity documents.

[0017] According to some aspects of the disclosed technology, the system generates multiple scores during the verification process that may be based on different characteristics or parts of the identity document. This may include, for example, an image authenticity score, an image verification score, a PII readability score, and a PII validity score. These scores may be combined to produce a final metric for the identity document, which may reflect the overall authenticity of the identity document. The final metric may be used to make automated decisions regarding the acceptance or rejection of the document. For example, access to a computing system, to a secure database, or an authentication message may all be determined and / or generated based on the final metric generated related to the document.

[0018] The system may process a submitted identity document by first applying a series of image preprocessing steps, such as rotation, normalization, and scaling, to ensure consistency across inputs. A deepfake detection engine, which may be powered by a Visual Geometry Group 16 (VGG-16 or VGG16) based CNN, may be used to identify any signs of image manipulation. VGG-16 may refer to a convolutional neural network that is 16 layers deep. In addition, an OCR module extracts PII from the document, including data fields such as name, address, date of birth, and identification number. The extracted PII is further validated through comparison with authoritative datasets and the application of error- detection algorithms.

[0019] The disclosed methods improve the accuracy and efficiency of identity verification by addressing the challenges posed by deepfake forgeries and inconsistencies in document formats. One aspect of the disclosed technology and its improvement over conventional techniques is the ability to detect sophisticated manipulations, such as deepfake forgeries, which are increasingly difficult for traditional verification methods to identify.

[0020] The disclosed system may include several components which may work to enhance the accuracy and security of the document verification process. The system may include a preprocessing engine which may extract images, and normalize input images (e.g., by applying transformations such as rotation, scaling, and color adjustments to ensure consistent input data for the system). A detection engine or a machine learning engine, which may include a neural network5US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO(e.g., a CNN based on a VGG16 architecture), may be used to identifying forged or manipulated content within the document. This engine may be specifically trained to recognize subtle anomalies in document images that may indicate manipulation. The detection engine may analyze pixel-level and structural features in the document image, identifying patterns that may suggest the use of AI- generated content, deepfake content, or other image-editing techniques. By leveraging a deep learning model trained on large datasets, the system can adapt to evolving methods of image manipulation, offering a significant advantage over traditional rule-based or heuristic methods.

[0021] Another improvement is the system’s ability to extract PII from a wide variety of document formats and types. An information extraction module may be powered by an optical character recognition (OCR) engine designed to handle inconsistencies in document layout, language, and quality. This module may extract key fields, such as name, date of birth, identification number, and other relevant data, ensuring that even poorly formatted or low-quality documents can be accurately processed. The extracted PII may be validated using a combination of reference databases, algorithms to detect formatting errors, and consistency checks to ensure that the information aligns with known patterns or trusted data sources.

[0022] The scoring mechanism employed by the system may further enhance its robustness and allow for improved decision-making capabilities. The system generates multiple scores during the document verification process, including a metric which assesses the likelihood of image manipulation, and a data integrity score which evaluates the accuracy and completeness of the extracted PII. Additional scores, such as a document quality score, may be generated to account for the clarity of the document image. These scores are then combined into a final metric (e.g., a final document score), which may provide a holistic evaluation of the document’s authenticity. In some examples, the weights given to the various scores may be tuned and / or adjusted based on the type of document and / or determined by machine learning techniques.

[0023] As another advantage, the disclosed systems and methods may also address the need for scalability and efficiency in high-volume environments. By integrating a machine learning model that has been trained using K-fold cross-validation and other techniques to reduce overfitting, the system ensures reliable performance across various document types, lighting conditions, and image qualities. This allows the system to process large numbers of documents quickly and accurately, minimizing the need for manual review and reducing operational overhead.6US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO

[0024] As another advantage, the system offers numerous technical advantages over conventional solutions. As one example, one or more iterations or versions of the trained machine learning model may be trained to enable the system to adapt to new types of image manipulations as they emerge. Thus, the system may be dynamic and remain effective even as additional fraudulent techniques evolve. The use of multiple scoring metrics may provide a more nuanced assessment of document authenticity, reducing the risk of false positives and negatives. As another example, the systems modular design may allow for one or more components of the system to be easily integrated and / or included with existing identity verification workflows, thus making it highly adaptable to a range of industries, including finance, government, healthcare, and online services.

[0025] The system’s architecture also facilitates the efficient handling of sensitive data, ensuring that PII is securely processed and stored in compliance with data protection regulations. By processing documents in real-time and generating a final metric, the system enables rapid decision-making, improving both the user experience and operational efficiency. Moreover, the ability to perform batch processing or handle real-time requests makes the system suitable for high- demand environments where speed and accuracy are paramount.

[0026] In overview, the disclosed system provides significant advancements in identity verification by combining Al-powered image manipulation detection, robust PII extraction, and comprehensive scoring mechanisms. These improvements result in a system that may be capable of handling sophisticated forgeries in documents while maintaining high levels of accuracy and efficiency. This may make the disclosed technology a valuable solution for industries requiring secure and reliable identity verification.

[0027] The verification system can generate a risk indicator for one or more systems using the model that is deployed in the production environment and that is configured to service requests for risk indicators which may be based on the detection of deepfakes. The system can then transmit the risk indicator (e.g., a final score) to a remote computing system. In some examples, this may be the system from which the risk indicator was requested. The risk indicator can be used to control access of the requestor entity to an interactive computing environment. For example, the risk indicator can be included in a responsive message to the request for evaluating the requestor entity such that the responsive message can be used to allow, challenge, or deny access to the requestor7US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO entity and / or a related entity. For example, if the risk indicator is below a predefined threshold, a request by the requestor entity to access the interactive computing environment may be automatically denied or flagged for manual review. Certain aspects described herein, which can include dynamically selecting a model for serving requests for risk indicators and providing a responsive message indicating a risk associated with a requestor entity, can improve at least the technical fields of controlling interactions between computing environments, access control for a computing environment, or a combination thereof. Additionally or alternatively, by using the techniques described herein, a verification system may provide legitimate access to the interactive computing environment more efficiently and using fewer computing resources compared to other risk assessment systems or techniques.

[0028] Certain aspects described herein, which can include generating one or more risk indicators associated with target entities and providing a responsive message using the risk indicator, can improve at least the technical fields of controlling interactions between computing environments, access control for a computing environment, or a combination thereof. For instance, by generating and transmitting the responsive message, the verification system can cause access to a computing system to be controlled more accurately. The risk indicator may be used to better predict whether the requestor entity requesting access is legitimate, and using the risk indicator may yield fewer malicious interactions than if the responsive message is not used. Further, the verification system leverages distinctive components of the risk indicator to create a robust and easily implemented framework.

[0029] These illustrative examples are given to introduce the reader to the general subject matter discussed here and are not intended to limit the scope of the disclosed concepts. The following sections describe various additional features and examples with reference to the drawings in which like numerals indicate like elements, and directional descriptions are used to describe the illustrative examples but, like the illustrative examples, should not be used to limit the present disclosure.I. EXAMPLE ENVIRONMENT FOR VERIFYING AN IDENTITY DOCUMENT

[0030] FIG. 1 illustrates an overview of a verification system architecture, according to some aspects of the present disclosure. Illustrated in FIG. 1 is the architecture 100, which may include a8US2008 30588218 1Attorney Docket No.: 096923-1458070 Client Ref: EFX-195WO verification system 102, databases 116, client computing systems 124, user computing systems 122, and third-party computing systems 126, which may be in data communication through a public data network 128. Additional components of architecture 100 are discussed below.

[0031] The verification system 102 may process and verify identity documents using artificial intelligence models, according to some aspects of the present disclosure. FIG. 1 illustrates hardware and software components of the verification system 102, which may be a specialized or general-purpose computing system capable of performing document analysis, deepfake detection, and personally identifiable information (PII) extraction. The verification system 102 may include a model training server 104 which may be used to train machine learning models (e.g., models that may detect document manipulations and verify the authenticity of the submitted identity documents).

[0032] The model training server 104 may include one or more processing devices configured to execute a model training and inference application, which includes multiple modules responsible for preprocessing, training, inference, and verification tasks. These modules may be included in a model training and verification application 106 contained on the model training server. The aforementioned modules may include but are not limited to, a data preprocessing module 108, a PII extraction module 110, a scoring engine 112, and a machine learning engine 114. Additionally, the verification system 102 may include or interact with any of the other components described in the following figures, such as the deepfake detection model architecture described in FIG. 2. Additionally, while not shown as separate modules, the model training and verification application 106 may allow for the training, update, monitoring, and modification of trained and / or untrained machine learning models.

[0033] The data preprocessing module 108 may be responsible for receiving raw input data from external systems, such as user computing systems 122 or client computing systems 124. These computing systems may provide, for example, digital images of identity documents (e.g., a scanned copy or photograph). The data preprocessing module 108 may perform various preprocessing tasks, including scaling, rotation, color normalization, and other image transformations. These tasks may ensure that the document image is normalized and suitable for analysis by the machine learning models used in the verification process.9US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO

[0034] The PII module 110 may utilize optical character recognition (OCR) techniques to extract relevant information from the document, such as the person’s name, date of birth, identification number, and address. The PII module 110 may process PII data, identifies key fields, and may also ensure the accuracy of the extracted information by comparing it to known data sources. As one example, this extracted data may then be validated against external databases or reference datasets stored in databases 116, ensuring that the data extracted from the document is accurate and matches trusted sources.

[0035] The scoring engine 112 may generate multiple scores to assess the authenticity of the document and the reliability of the extracted data. It may also receive and / or consider outputs from other components (e.g., the machine learning engine 114). These scores may include, without limitations, a manipulation detection metric, which may indicate the likelihood that the document has been digitally altered or contains deepfake elements, and a data integrity metric, which may indicate the accuracy and completeness of the PII fields extracted from the document. The system may also generate a document quality score, which may assess the clarity and legibility of the document image. In some examples, such as in a digital document, a document quality score may examine digital markers on the digital identity document. These individual scores are combined into a final metric by the scoring engine 112, which determines the overall authenticity of the document and whether it should be accepted, flagged for manual review, or rejected. Various scores, sub-scores, and metrics may be generated, including variations of the named scores herein. Certain scores may be referred in multiple names.

[0036] The machine learning engine 114 may contain one or more algorithms and / or trained machine learning models. The inference engine 114 may generates predictions or classifications using the trained model. The machine learning engine 114 may contain one or more of the trained machine learning models that have been trained using the methods described herein. This may include the neural works described herein with respect to FIG. 2.

[0037] The model training server 104 may communicate with a data repository 118 via a network 130. The data repository 118 may store a variety of datasets, including training datasets 120 and model output data 132. The training datasets 120 may include of identity document images and related data used to train the machine learning models employed by the verification system 102 or components thereof (e.g., the machine learning engine 114). For example, the training 10US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO datasets may include images of known forged or manipulated documents, which help the system learn to recognize different types of fraudulent activity or fraudulent documents. Model output data 132 may include the results generated by the machine learning models, such as the predictions, classifications, and scores produced during the verification process.

[0038] The verification system 102 can perform additional tasks related to the ongoing validation and deployment of the trained machine learning models. For example, the verification system 102 may receive new datasets from external sources such as databases 116 or user computing systems 112, which provide new document images for analysis. These external systems can feed real-time data and / or live data into the verification system 102 for immediate processing and inference. The system may also update its models by retraining them using new data, ensuring that they remain effective in detecting the latest forms of document manipulation.

[0039] The verification system 102 may interact with external systems via the public data network 128. These external systems include client computing systems 124 and user computing systems 122, which may submit identity documents for verification. Third-party computing systems 126, such as financial institutions, regulatory bodies, law enforcement, governments, investigators, or other external organizations, may also request verification results from the verification system 102. For example, a third-party system computing system 124 may request verification of a customer’s identity document during the account onboarding process or for compliance with regulatory requirements. The verification system 102 may also return verification results, which may include a detailed breakdown of the scores and the final verification decision.

[0040] The verification system 102 may also supports multiple modes of operation, including batch processing and real-time document verification. It can process a large volume of document submissions, making it suitable for high-demand environments such as financial services, healthcare, or government agencies. In some cases, the system may process a document in real time and immediately return the verification result to the submitting system, such as a user computing system 122 or client computing system 124.

[0041] In some aspects the verification system 102 may allow external systems to request additional verification or reanalysis of a previously submitted document. For example, if new data becomes available or if the initial verification result is inconclusive, a third-party computing11US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO system 126 may request a secondary analysis of the same document and / or an additional verification or identification document. The system may also allow for manual review by a human operator in cases where the verification scores indicate uncertainty or a need for further investigation.

[0042] The databases 116 may contain trusted reference data, which may include government records, credit bureau data, or other authoritative sources. The verification system 102 may compare the PII extracted from the document against the data stored in these databases, ensuring that the extracted data is accurate and legitimate. The system’s ability to cross-check extracted data against multiple sources reduces the likelihood of fraud or errors in the verification process. In some examples, the verification system 102 may compare the data against the data repository 118.

[0043] Communications between the verification system 102 and internal and / or external systems may occur over secure networks, such as the public data network 128 or over the network 130. These networks may include a combination of wired and wireless connections, enabling the verification system 102 to securely transmit data and results across multiple systems and locations. For example, data may be transmitted between the verification system 102 and the data repository 118 or other network-attached storage units, which store large datasets for training and inference purposes. The network infrastructure supports robust and scalable processing, allowing the system to handle complex verification tasks efficiently.

[0044] Storage within the verification system 102 or attached storage units may include primary storage directly accessible by the processors within the model training server 104, as well as secondary or auxiliary storage for long-term data retention. This storage infrastructure may include various types of devices, such as hard drives, solid-state drives, optical storage, or cloudbased storage solutions. Data is stored in non-transitory, machine-readable media, ensuring that information is retained securely and can be accessed for later analysis or audit purposes. The data repository 118 or other network-attached storage units within the operating environment may store a variety of different types of data organized in various ways and from multiple sources. The network-attached storage unit may include storage beyond the primary storage located within the model training server 104 that may be directly accessible by the processors therein. In some aspects, the network-attached storage unit may include secondary, tertiary, or auxiliary storage, 12US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO such as large hard drives, servers, and virtual memory, among other types of suitable storage. Storage devices may include portable or non-portable storage devices, optical storage devices, and various other mediums capable of storing and containing data. A machine-readable storage medium or computer-readable storage medium may include a non-transitory medium in which data can be stored and that does not include carrier waves or transitory electronic signals. Examples of a non-transitory medium may include, for example, a magnetic disk or tape, optical storage media such as a compact disk or digital versatile disk, flash memory, memory devices, or other suitable media.

[0045] In some aspects, the model training server 104 can perform additional tasks related to the validation and deployment of the machine learning model, using the modules within the model training and verification application 106. The server may receive external datasets from databases 116, which feed new data into the model for ongoing training or real-time analysis. The databases 116 might store external data sources such as documents, records, or other relevant data that can be used to enhance the model's performance.

[0046] Furthermore, the verification system 102 can communicate with various other computing systems. The other computing systems can include user computing systems 122, such as smartphones, personal computers, etc., client computing systems 124, and other suitable computing systems. While FIG. 1 illustrates that the verification system 102 and the client computing systems 124 are separate systems, the verification system 102 and the client computing systems 124 can be one system. For example, the verification system 102 can be a part of the client computing systems 124, or vice versa. The client computing systems 124 may deploy the model in a model deployment environment, such as an online platform or service where the model generates predictions based on new data and / or data stored in the databases 116. The verification system 102 may communicate with various other computing systems via one or more data networks, such as the public data network 128. These communications could involve transferring data for further processing, receiving updates to the model, or delivering the model's outputs to end-users or other systems. The network might include a combination of wired and wireless connections, supporting a robust and scalable infrastructure for large-scale data processing and machine learning tasks.13US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO

[0047] In some examples, the client computing system 124 may include other computing resources associated therewith (e.g., not shown in FIG. 1), such as server computers hosting and managing virtual machine instances for providing cloud computing services, server computers hosting and managing online storage resources for users, server computers for providing database services, and others. The interaction between the user computing system 122, the client computing system 124, and the verification system 102 , or any suitable sub-combination thereof may be performed through graphical user interfaces, such as the user interface, presented by the verification system 102, the client computing system 124, other suitable computing systems of the computing environment 100, or any suitable combination thereof. The graphical user interfaces can be presented to the user computing system 122. Application programming interface (API) calls, web service calls, or other suitable techniques can be used to facilitate interaction between any suitable combination or sub-combination of the client computing system 124, the user computing system 122, and the verification system 102.

[0048] A user computing system 122 can include any computing device or other communication device that can be operated by a user or entity, such as the user entity, which may include a consumer or a customer. The user computing system 122 can include one or more computing devices such as laptops, smartphones, and other personal computing devices. A user computing system 122 can include executable instructions stored in one or more non- transitory computer-readable media. The user computing system 122 can additionally include one or more processing devices configured to execute program code to perform various operations. In various examples, the user computing system 122 can allow a user to access certain online services or other suitable products, services, or computing resources from a requestor entity, such as the client computing system 124, to engage in mobile commerce with the client computing system 124, to obtain controlled access to electronic content, such as the third-party computing systems 126, hosted by the client computing system 124, etc.

[0049] In a simplified example, the system illustrated in FIG. 1 can configure the verification system 102 to be used for generating predictions based on information from the client computing systems 124, the user computing systems 122, or third-party computing systems 126. The model training server 104 can retrieve data sources associated with one or more of the computing systems in response to a request to perform a prediction or to re-train a machine learning model. The data14US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO sources may, for example, be retrieved from databases 116 or received via other suitable computing systems. The databases 116 can store, for example, data sources such as articles or other publications periodically scraped from the Internet. The model training server 104 can determine a risk indicator associated with the requestor entity by extracting and information provided by the requestor entity. The model training server 104 can transmit a data point, or any inference derived therefrom, to the client computing system 124 as an output prediction.

[0050] Each client computing system 124 may include one or more devices such as individual servers or groups of servers operating in a distributed manner. A client computing system 124 can include any computing device or group of computing devices operated by a seller, lender, or other suitable entity that can provide products or services. The client computing system 124 can include one or more server devices. The one or more server devices can include or can otherwise access one or more non-transitory computer-readable media.

[0051] The client computing system 124 can further include one or more processing devices that can be capable of providing a third-party computing systems 126, such as a user interface, etc., that can perform various operations. The third-party computing systems 126 can include executable instructions stored in one or more non-transitory computer-readable media. The instructions providing the third-party computing systems 126 can configure one or more processing devices to perform the various operations. In some aspects, the executable instructions for the third-party computing systems 126 can include instructions that provide one or more graphical interfaces. The graphical interfaces can be used by a user computing system 122 to access various functions of the third-party computing systems 126. For instance, the third-party computing systems 126 may transmit data to and receive data, such as via the graphical interface, from a user computing system 122 to shift between different states of the third-party computing systems 126, where the different states allow one or more electronic interactions between the user computing system 122 and the client computing system 124 to be performed.

[0052] A user computing system 122 can be used to request access to the third-party computing systems 126 provided by the client computing system 124. The client computing system 124 can submit a request, such as in response to a request made by the user computing system 122 to access the third-party computing systems 126, for risk assessment to the verification system 102 and can selectively grant or deny access to various electronic functions based on risk 15US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO assessment performed by the verification system 102. Based on the request, or continuously or substantially contemporaneously, the verification system 102 can determine one or more risk signals or risk indicators for data (e.g., identification documents) associated with the requestor entity, which may submit or may have submitted the request via the user computing system 122. The verification system 102, the client computing system 124, or a combination thereof can determine whether to grant the access request of the user computing system 122 to certain features of the third-party computing systems 126. The verification system 102, the client computing system 124, or a combination thereof can use the risk indicator for other suitable purposes such as identifying a manipulated identity, controlling a real-world interaction, and the like.

[0053] In a simplified example, the system illustrated in FIG. 1 can configure the risk assessment server 104 to be used for controlling access to the third-party computing systems 126. The risk assessment server 104 can retrieve data associated with the requestor entity in response to a request to access the third-party computing systems 126. The data may, for example, be retrieved based on identity information (e.g., information collected by the client computing system 124 via a user interface provided to the user computing system 122) provided by the client computing system 124 or received via other suitable computing systems. The risk assessment server 104 can retrieve the data associated with the requestor entity from the data repository 112. The risk assessment server 104 can determine a risk indicator associated with the requestor entity using a model deployed in a production environment of the risk assessment server 104. The risk assessment server 104 can transmit the risk indicator, or any inference derived therefrom, to the client computing system 124 for use in controlling access to the third-party computing systems 126.

[0054] The risk indicator associated with the requestor entity, or any suitable score or comparison determined therefrom, can be used, for example by the verification system 102, the client computing system 124, etc., to determine whether the risk associated with the requestor entity accessing a good or a service provided by the client computing system 124 using exceeds a threshold, thereby granting, challenging, or denying access by the requestor entity to the third- party computing systems 126. For example, if the verification system 102 determines that the risk indicator indicates that risk associated with the identification document is lower than a threshold value, then the client computing system 124 associated with the service provider can generate or16US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO otherwise provide access permission to the user computing system 122 that requested the access. The access permission can include, for example, cryptographic keys used to generate valid access credentials or decryption keys used to decrypt access credentials. The client computing system 124 can also allocate resources to the requestor entity and provide a dedicated web address for the allocated resources to the user computing system 122, for example, by adding the user computing system 122 in the access permission. With the obtained access credentials or the dedicated web address, the user computing system 122 can establish a secure network connection to the third- party computing systems 126 hosted by the client computing system 124 and access the resources via invoking API calls, web service calls, HTTP requests, other suitable mechanisms or techniques, etc.

[0055] In some examples, the verification system 102 may determine whether to grant, challenge, or deny the access request made by the user computing system 122 for accessing the third-party computing systems 126. For example, based on the risk indicator associated with the requestor entity, the verification system 102 can determine that the requestor entity is a legitimate entity that made the access request and may authenticate the request. In other examples, the verification system 102 can challenge or deny the access attempt if the verification system 102 determines that the requestor entity may not be a legitimate entity.

[0056] Each communication within the computing environment 100 may occur over one or more data networks, such as a public data network 128, a network 130 such as a private data network, or some combination thereof. A data network may include one or more of a variety of different types of networks, including a wireless network, a wired network, or a combination of a wired and wireless network. Examples of suitable networks include the Internet, a personal area network, a local area network (“LAN”), a wide area network (“WAN”), or a wireless local area network (“WLAN”). A wireless network may include a wireless interface or a combination of wireless interfaces. A wired network may include a wired interface. The wired or wireless networks may be implemented using routers, access points, bridges, gateways, or the like, to connect devices in the data network.

[0057] While FIG. 1 illustrates a specific configuration of devices and systems, it should be understood that variations may exist depending on the implementation. For example, the verification system 102 may be distributed across multiple servers or data centers to provide 17US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO redundancy and fault tolerance. Additionally, while the model training server 104 is depicted as a standalone component, it may also be integrated into the client computing systems 124 or operated as part of a cloud-based service, depending on the deployment scenario.II. EXAMPLE MACHINE LEARNING ARCHITECTURE FOR DEEPFAKE DETECTION MODELS

[0058] FIG. 2 depicts a deepfake detection model architecture, according to some aspects of the present disclosure. Depicted in FIG. 2 is architecture 200. In the example provided with respect to FIG. 2, architecture 200 may contain an input layer 210, base model layers 220, custom layers 230, and an output layer 240. In some examples, architecture 200 may be based on an VGG16- based neural network. The architecture 200 can be implemented using any of the components as described above with reference to FIG. 1. The architecture 200 depicts a structure of a neural network, which may, once trained be put into a production environment (e.g., within the machine learning engine 114).

[0059] In some examples, architecture 200 may include and / or be formed from a neural network. Neural networks may be machine learning models which may be designed to simulate the structure of the human brain. A neural network may include of interconnected layers of neurons. Each neuron may process inputs by applying mathematical functions, passing the results to the next layer, and so forth, until a final layer or set of neurons is reached. This final layer or final neuron may contain an output (e.g., a score, a risk indicator, a probability) which may ultimately be used in making decisions and / or predictions.

[0060] In architecture 200, the neurons are arranged in layers. In a neural network, neurons are responsible for receiving input, performing calculations, and passing output to the next layer. In some examples, each layer may learn progressively abstract features of an image. For example, in a neural network tasked with detecting deepfake manipulations in document images, early neurons might detect sharp edges or straight lines, while neurons in deeper layers may identify more sophisticated patterns like unnatural blending, irregularities in font spacing, or pixel-level inconsistencies introduced by image manipulations. These patterns would be harder for the human eye to detect, but neural networks can learn to identify them after being trained on large datasets. As an example, in architecture 200, neurons in earlier layers may detect simple patterns, such as18US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO edges or textures, while neurons in deeper layers may capture more complex features, such as shapes, objects, or anomalies. As the network progresses, these neurons learn to combine simple features into more complex ones, allowing the model to understand the intricate details that indicate deepfake manipulations. In some examples, neurons may be specialized to detect specific types of manipulations, such as color inconsistencies or unnatural blending, making them useful in deepfake detection tasks.

[0061] Additionally, when implementing neural networks in systems like the verification system 102 or architecture 100, speed and efficiency may be desirable. For example, this may be especially true when the system needs to operate in real-time, such as in financial or government sectors where delays in document verification can introduce significant security risks and / or cause operational inefficiencies. While in some examples neural networks may require substantial computational resources, particularly when dealing with high-resolution images or large numbers of documents, techniques, including those described herein may increase the efficiency of a neural network and decrease the processing time. In some examples, neural networks are optimized to minimize processing times while maintaining accuracy. Techniques like transfer learning, dimensionality reduction, and hardware accelerators (such as GPUs) may be employed to enhance speed and efficiency in the verification process.

[0062] The input layer 210 of architecture 200 may be the preprocessed image or document provided by various systems, such as for example, user computing systems 122, client computing systems 124, and third-party computing systems 126. This input may come in the form of digital images of identity documents, such as a scanned image of a passport or a photograph of a driver’s license. In other examples, digital documents may be provided.

[0063] At or prior to the input layer 210, preprocessing of the input information may be performed. Preprocessing steps may be used to prepare the input for analysis by the neural network. As an example, preprocessing may include resizing the image to the appropriate dimensions expected by the network. For instance, a fixed pixel size (e.g., 224 x 224 pixels) may be chosen based on the type, configuration, processing power, and complexity of the neutral network. The pre-processing may also be based on the neural layers which are downstream to the input layer 210. For a VGG16 model, the aforementioned pixel size may be balanced to allow for efficiency processing. Other transformations, such as normalization, can ensure that pixel values 19US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO are within a consistent range, allowing the neural network to process the image effectively. These preprocessing steps ensure uniformity across inputs, helping the model to generalize better during training and inference.

[0064] The preprocessed image may be fed from the input layer 210 into the base layers 220. The base layers 220 may include and / or be related to a convolutional neural network (CNN) such as VGG16. VGG16 may be useful in image classification, and may consist of 16 layers, including convolutional layers that extract hierarchical features from the input image. VGG-16 may contain 16 weight layers, including 13 convolutional layers and 3 fully connected layers. Convolutional layers act like filters that scan the image for patterns, detecting things like edges, textures, and more complex shapes. In some examples, the base layers 220 may contain variants of VGG-16, including different types of layers and / or frozen / unfrozen layers.

[0065] In the example provided in FIG. 2, for detection of deepfakes within identification documents, base layers 220 may leverage transfer learning, where a portion of the base layers and / or machine learning model is pre-trained on a large dataset and later fine-tuned for specific tasks. In the example provided, the top four layers (layers 222) of base layers 220 may be unfrozen to allow fine-tuning during training, enabling the model to learn task-specific patterns, such as manipulations in document images. The remaining layers (layers 221) may be “frozen” or be configured to not be changed to retain their pre-trained knowledge of general image features. “Unfrozen” layers may refer to layers which can be tuned, modify, or updated with weights.

[0066] In other examples, the base model may be another architecture, such as ResNet or MobileNet, which may provide their respective strengths. ResNet may enable much deeper networks with its skip connections, which help mitigate the vanishing gradient problem during training. MobileNet can be used in environments with limited computational resources, such as mobile or embedded systems. Thus, the technology may be applicable to various computing environments with differing computational power.

[0067] After feature extraction, the data may be passed through a series of custom layers 230. These are discussed in turn below.

[0068] After feature extraction, the data may be passed through a series of custom layers 230. The first layer may be the Global Average Pooling 2D layer 231 (referred herein as layer 231).20US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WOLayer 231 is useful for dimensionality reduction. Dimensionality reduction is the process of reducing the number of input variables in the model. Dimensionality reduction helps simplify the model by focusing only on the most important features, discarding unnecessary or redundant information. In the context of neural networks, dimensionality reduction may be useful in preventing overfitting, where the model becomes too specialized to the training data and fails to generalize to new, unseen data. Thus, this layer may allow for the generalization to various types of deepfakes and allow the model to be updated to recognize new methods of deepfakes.

[0069] Reducing dimensionality also decreases the computational load on a machine learning model. Large feature maps from convolutional layers are computationally expensive or computationally intensive to process, especially in real-time applications where efficiency may be crucial. The use of layer 231 allows the machine learning model to compress the output of the convolutional layers into smaller, more manageable forms, while retaining significant features of the image. In some examples, other forms of dimensionality reduction, such as max pooling or average pooling, may be used, depending on the architecture of the network.

[0070] Following dimensionality reduction, the features may be passed to a dense layer 232 (referred herein to as layer 232). Layer 232 may apply a fully connected operation to aggregate the information learned by the previous layers. In some examples, a Rectified Linear Unit (ReLU) activation function may be used or applied at this layer, which may introduce non-linearity into the machine learning model. Without non-linearity, the model would behave as a simple linear function, limiting its ability to learn complex patterns within the data.

[0071] The ReLU activation function may be particularly effective for deep neural networks as it allows the model to converge quickly during training, mitigating the issue of vanishing gradients, where gradients become too small to update weights effectively. In some examples, other activation functions, such as Leaky ReLU or Exponential Linear Unit (ELU), may be used depending on the specific task requirements.

[0072] The output from layer 232 may be passed through a batch normalization layer 233 (referred herein to as layer 233). Batch normalization is a technique that may standardizes the output of each layer, ensuring that the input to the next layer has a mean of zero and a standard deviation of one. By doing so, batch normalization may address a training problem referred to as21US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO an internal covariate shift problem, in which the distribution of inputs to each layer changes during training. This stabilization allows the network to converge more quickly, improving training efficiency and model performance.

[0073] Batch normalization may also act as a form of regularization (which may be similar to dropout), by adding a slight randomness and / or noise during training. This addition of noise may make the model less sensitive to small changes in the training data. In some examples, batch normalization may be applied after any layer where large changes in data distribution are expected. This process may be particularly useful in deep neural networks, where small imbalances in data can compound across layers and lead to issues such as vanishing or exploding gradients.

[0074] Moving across the neural network, the next layer may be a dropout layer 234 (referred herein as a layer 234). This layer may improve generalization and may prevent overfitting. At this layer, a random percentage of the neurons may be deactivated during each iteration of training of the machine learning model. This may force the network to learn more redundant, robust features, as it cannot rely on any particular subset of neurons to make predictions. This in turn may prevent overweighting of any subset of neurons or neural pathways within the neural network being trained. As one example, a dropout rate of 0.5 may be used, meaning 50% of the neurons are deactivated in each forward pass. The specific dropout rate may be set heuristically.

[0075] In the process of making the neural network more robust to variations in the data, dropout ensures that the model can generalize to new, unseen examples. This may be particularly important in applications like deepfake detection, where subtle manipulations can vary significantly from one example to another. In other examples, the dropout rate may be adjusted based on the size of the network and the complexity of the task, with larger networks requiring a higher dropout rate to avoid overfitting.

[0076] The final layer in the custom layers 230 may be the second dense layer (referred to herein as layer 235). This layer may use a sigmoid activation function to output a probability score. The sigmoid function may transform the input of the layer 235 into a value between 0 and 1, representing the likelihood that the input image has been manipulated. The value between 0 and 1 may be a probability. This probability score from layer 235 may be passed to the output layer 240, where it can be interpreted as the model’s prediction regarding the authenticity of the document.22US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO

[0077] The output layer 240 may be the final output of the neural network. The output layer 240 may ultimately represent a rating, score, metric, value, and / or probability related to a deepfake detection. The output layer 240 may translate and / or contain more than one types of probability. For example, the output 240 may indicate the type, algorithm, and / or nature of the deepfake and / or deepfake techniques used. It may also indicate when it is or is not sure about a particular output.

[0078] Output 240 may represent the deepfake detection score, which may be passed to one or more other components. For example, the output may be passed to the scoring engine 112. In some examples, the scoring engine 112 may combine this score with other scores (e.g., a data integrity score, a PII score, etc.), to generate a final verification score or other final metric. This final metric may be used by the machine learning engine 114, or other components of the verification system 102 to determine whether the document is authentic or requires further investigation.

[0079] During training of a neural network, each neuron in a given layer may be connected to every neuron in the following layer, and the strength of these connections is determined by weights. During training, the model adjusts these weights to minimize the error between the predicted output and the actual label. The process of updating the weights may be referred to as backpropagation. A person of skill in the art will appreciate various training methods for neural networks and other machine learning models.

[0080] Additionally, the training process for the machine learning model may involve K-fold cross-validation. K-fold cross-validation is a technique that may be used to assess how well the model generalizes to new data. Cross-validation is especially important in tasks like deepfake detection, where the data can be highly variable. By using cross-validation, the model can avoid overfitting to any particular subset of the training data and can better generalize to unseen manipulations.

[0081] In K-fold cross-validation, the dataset may be split into K subsets, or “folds.” The model is trained on K minus 1 of these folds. The performance of the model may be then tested on the remaining subset or fold. This process may be repeated K times, with each fold being used as the validation set once. This ensures that the model is trained and validated on every part of the dataset, providing a more robust evaluation of its performance. The splitting and randomness of the K subsets may be performed by other computers and / or based on other heuristic considerations.23US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WOIn some examples, other validation techniques, such as leave-one-out cross-validation or stratified K-fold cross-validation, may be used depending on the size and distribution of the dataset.III. TECHNIQUES FOR TRAINING A MACHINE LEARNING MODEL

[0082] FIG. 3 is a flowchart showing the training process for the deepfake detection model, according to some aspects of the present disclosure. Illustrated in FIG. 3 is method 300. Method 300 may be used for model training and data preprocessing, according to some aspects of the present disclosure. One or more computing devices (e.g., the user computing systems 122, client computing systems 124, the model training server 104) implement operations depicted in FIG. 2 by executing suitable program code (e.g., the model training and inference application 106). For illustrative purposes, the method 300 is described with reference to certain examples depicted in the figures. Other implementations, however, are possible. While the blocks of the method 300 are described in the temporal order below for illustrative purposes, it may be appreciated that the blocks can occur in any order and some blocks may occur simultaneously.

[0083] At block 310, data may be collected, and input preparation may occur. In this block, the system may gather training data, which can include both genuine and manipulated document images. This data may originate from a variety of sources, such as for example, user computing systems 122, client computing systems 124, and third-party computing systems 126. Additional sources, such as external databases or third-party systems, may be used to augment the dataset. In some examples, the resolution of the training data, the types of deepfakes generated, and the tags may be varied to ensure a robust training dataset. Further, in some examples, ensuring high-quality input data is crucial, as low-resolution images or poorly formatted documents may negatively affect the model’s ability to detect subtle manipulations. In some examples, input validation may be performed to ensure that only images of sufficient quality are used. At this step, metadata about the type of identity document may also be provided (e.g., passport, driver’s license, military identification, etc.). This metadata may be used to fine tune or modify the input and / or output the machine learning model.

[0084] At block 320, data preprocessing may be applied to the input. Preprocessing may assist with preparing the dataset of training. At this block, images may be resized to fit the input dimensions expected by the neural network. This resizing may ensure uniformity across the24US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO dataset, allowing the model to learn patterns more effectively. Next, normalization may be performed on the input images, which may adjust pixel values such that they fall within a standardized range. This step helps prevent skewed inputs from affecting the learning process. For example, extreme variations in lighting or contrast between documents may distort the patterns learned by the model, reducing its ability to detect manipulations accurately. This preprocessing step may ensure that certain types of images (or images of identities) are not processed by the trained machine learning model.

[0085] Preprocessing may also include image augmentation techniques, such as rotating, flipping, or cropping the images. Augmentation may increase the diversity of the training data, making the model more robust to variations in real-world input. This is especially important for deepfake detection, where subtle changes in angles, lighting, or alignment may alter the appearance of a document. By augmenting the data, the model learns to detect manipulations across a wider range of conditions, improving its generalization to unseen data. The robustness of the model in detecting deepfakes may be directly linked to the quality and diversity of the data used during preprocessing.

[0086] At block 330, the preprocessed data may be split into training, validation, and test sets. These setts may be included in the training datasets 120 referred to in FIG. 1. Splitting the data is important for assessing how well the model generalizes to new, unseen examples. Techniques, including K-fold cross-validation may be used to split the data into multiple types of test and / or validation sets. Stratified splitting may also be used, ensuring that both manipulated and genuine documents are well-represented in each subset.

[0087] At block 340, the model may be initialized. Block 340 may involve model initialization. In some examples, this may include initializing a pre-trained model (e.g., VGG16) or variations thereof. Certain layers may be unfrozen for fine-tuning such as in architecture 200. This may allow the model to adjust its weights during training, tailoring its ability to detect deepfakes in document images. In some examples, base weights may be assigned during model initialization.

[0088] At block 350, the training of one or more machine learning model may take place. During training, in the example of a neural network, the model’s weights are changed to be better suited and more accurate at detecting an output task by using techniques like backpropagation and25US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO gradient descent. The model learns to classify images as genuine or manipulated by minimizing a loss function, such as binary cross-entropy, which measures the error between the predicted output and the true label. In some examples, regularization techniques like dropout and batch normalization may be applied to prevent overfitting. Dropout may involve deactivating a portion of neurons during each forward pass, ensuring that the model learns generalized patterns rather than relying too heavily on specific neurons. Batch normalization may be applied to ensure that each layer receives well-scaled inputs, stabilizing the training process and accelerating convergence. A person of skill in the art will appreciate that other variations of training may be possible depending on requirements, convergence, and computational power.

[0089] At block 360, the trained machine learning model may be validated. Validation may be used to monitor the model’s performance and / or fine-tuning its hyperparameters, such as the learning rate or batch size. The model’s performance on the validation set can reveal whether adjustments to these parameters are necessary. In some examples, early stopping may be employed to prevent the model from continuing to train once its performance on the validation set stops improving. Early stopping helps mitigate the risk of overfitting by halting training when the model reaches an optimal point, ensuring that further training does not degrade performance in terms of its generalization to multiple types of datasets.

[0090] At block 370, additional evaluation may be performed. This block may be performed on unseen data. At this block, various metrics may be calculated, such as accuracy, precision, recall, and an Fl -score. Accuracy may measure the percentage of correct predictions, while precision may measure the proportion of true positives out of all positive predictions, reflecting how often the model correctly identifies manipulations. Recall may measure the proportion of true positives out of all actual positives, which may indicate how well the model captures all instances of manipulation. The Fl -score provides a harmonic mean of precision and recall, offering a balanced metric for assessing the model’s effectiveness. In some examples, a confusion matrix may be generated, providing a breakdown of true positives, true negatives, false positives, and false negatives, offering deeper insights into the model’s behavior and areas for improvement.26US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WOIV. TECHNIQUES FOR PERFORMING DOCUMENT VERIFICATION

[0091] FIG. 4 shows the end-to-end document verification process, including score generation and final output, according to some aspects of the present disclosure. FIG. 4 illustrates method 400. Method 400 may begin with the submission of an input document and ends with the verification output, which is returned may returned to a client, third-party system, a user, and / or provide and / or deny access to certain data. One or more computing devices (e.g., the user computing systems 122, client computing systems 124, the model training server 104) implement operations depicted in FIG. 2 by executing suitable program code (e.g., the model training and inference application 106). For illustrative purposes, the method 400 is described with reference to certain examples depicted in the figures. Other implementations, however, are possible. While the blocks of the method 400 are described in the temporal order below for illustrative purposes, it may be appreciated that the blocks can occur in any order and some blocks may occur simultaneously.

[0092] At block 410, an identity document may be submitted for verification. The user computing system 122 or the client computing system 124 may submit a document, such as a scanned image or digital photo of an identity document, like a passport, driver’s license, or another government-issued ID. This submission may mark the beginning of the verification flow.

[0093] At block 420, preprocessing of the document may take place. This may include extracting an image from the document. The image may be preprocessed to resizing the image to match the expected input dimensions required by the machine learning model (e.g., 224 x 224 pixels), normalizing the pixel values to ensure uniformity in lighting and contrast, and applying data augmentation to create variations in the input data. These preprocessing steps help the model handle a range of real-world document variations and ensure that the image is formatted appropriately for subsequent stages of analysis. In other examples, the pre-processing may include identifying metadata about the user of the document, other images of the user which may be present in a database for comparison, the type of document, the expected features of a document, the age of the identity document, the rarity of the type of document (e.g., passport versus state ID), or other features anticipated in the document. These other features may be used to generate other scores related to the document.27US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO

[0094] At block 430, the preprocessed document may be passed through the machine learning model, where a score is generated. This may include a trained version of the model described in FIG. 2. The model may use and / or extract important features, such as edges, textures, and spatial patterns, to detect any signs of manipulation or deepfake alterations. This process may produce a score (e.g., an image verification score), which may be a measure, metric, and / or probability of the likelihood of manipulation in the document.

[0095] At block 440, the system may extract and / or obtain other textual and / or visual information from the identity document. For example, an OCR technique such as Easy OCR may be used to extract textual information from the document. EasyOCR may apply optical character recognition to extract structured data, such as names, dates, and addresses, from the image. Since the extracted text from identity documents often lacks context, this block also may allow for the extracting and / or processing structured text. Traditional natural language processing techniques may struggle at processing structured text. The text is extracted for further analysis, such as for example, regex-based detection of Personally Identifiable Information (PII) fields.

[0096] At block 450, PII may be detected and validated. This may include a number of processes or sub-blocks which may be performed in any order. For example, regular expressions (regex) may be used to detect specific PII fields, such as Name, ID Number, Date of Birth, and Address, from the extracted text. This regex-based detection may ensure that the system can accurately extract relevant information from the structured text. Once extracted, the PII data may be validated against trusted external and / or internal databases to verify its legitimacy, generating a PII Validity Score. A PII readability score may also be calculated to assess the clarity of the extracted information. These scores contribute to the overall verification process by ensuring the accuracy and validity of the personal data present in the document.

[0097] At block 460, the authenticity of the document image may be checked. An image authenticity score may be generated by comparing the document image with a reference image of the person (e.g., comparing a driver’s license photo to a selfie). This may ensure that the document is not only legitimate (e.g., not generated as a deepfake) but also matches the individual submitting it. This step may thus allow for verifying that the identity document corresponds to the correct individual. In some examples, this step may occur only after it has been verified that the document is not a deepfake.28US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO

[0098] At block 470, the system may calculate a final score (e.g., a Final Document Score (FDS)) by combining the various scores generated throughout the process. This score may be based on weights which may be assigned to each score based on their importance. For instance, the image verification score and PII validity score may carry more weight in determining document legitimacy than the PII readability score. The final document score is calculated using the following formula:FDS=(Wiv -Siv )+(Wpr -Spr )+(Wpv -Spv )+(Wia -Sia) (Equation 1) where:Wiv and Siv are the weight and score for Image Verification respectively,WPr and Spr are the weight and score for PII Readability respectively,WPv and Spv are the weight and score for PII Validity respectively, andWia and Sia are the weight and score for Image Authenticity respectively.

[0099] The final document score may be compared to a predefined threshold. If the score exceeds and / or is under the threshold (e.g., 0.7), the document is flagged as suspicious and may require further review. Otherwise, the document is considered genuine and can proceed through the system. In some examples, the final document score may be overridden or not considered when a certain element of the score is 0. For example, no final document score may be calculated if the image verification score is 0.

[0100] At block 480, the output of the verification process may be provided and / or used to perform an action. For example, the final document score and / or the corresponding decision may be sent back to the user computing system 122 or the client computing system 124. The output may include details on the verification result, such as the scores used to reach the decision, and any recommendations for further action, such as submitting additional documents or undergoing manual review. In some examples, a credit profile of a user may be provided once the identify is verified. In other examples, access to a resource may be provided once the identify and / or the identity document is considered verified. In other examples, a sliding scale may be used to determine the amount of resources to which a user and / or entity is provided access to. This block may include transmitting, to a remote computing device (e.g., the client computing device 124), a29US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO responsive message comprising at a risk indicator which may be based on the final document score for use in controlling access to one or more interactive computing environments. For example, the risk indicator can be used in controlling an interaction involving a requestor entity or access of the requestor entity to a restricted system (e.g., the third-party computing system 126).V. EXAMPLE OF COMPUTING SYSTEM

[0101] Any suitable computing system or group of computing systems can be used to perform the operations for the techniques described herein. For example, FIG. 5 is a block diagram depicting an example of a computing device 500, which can be used to implement the risk assessment server 104. The computing device 500 can include various devices for communicating with other devices in the computing environment 100, as described with respect to FIG. 1. The computing device 500 can include various devices for performing one or more operations, such as risk assessment operations, described above with respect to FIGs. 1-4.

[0102] The computing device 500 can include a processor 502 that can be communicatively coupled to a memory 504. The processor 502 can execute computer-executable program code stored in the memory 504, can access information stored in the memory 504, or both. Program code may include machine-executable instructions that may represent a procedure, a function, a subprogram, a program, a routine, a subroutine, a module, a software package, a class, or any combination of instructions, data structures, or program statements. A code segment may be coupled to another code segment or a hardware circuit by passing or receiving information, data, arguments, parameters, or memory contents. Information, arguments, parameters, data, etc., may be passed, forwarded, or transmitted via any suitable means including memory sharing, message passing, token passing, network transmission, among others.

[0103] Examples of a processor 502 can include a microprocessor, an application-specific integrated circuit (ASIC), a field-programmable gate array (FPGA), or any other suitable processing device. The processor 502 can include any suitable number of processing devices, including one. The processor 502 can include or communicate with a memory 504. The memory 504 can store program code that, when executed by the processor 502, causes the processor 502 to perform the operations described herein.30US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO

[0104] The memory 504 can include any suitable non-transitory computer-readable medium. The computer-readable medium can include any electronic, optical, magnetic, or other storage device capable of providing a processor with computer-readable program code or other program code. Non-limiting examples of a computer-readable medium can include a magnetic disk, memory chip, optical storage, flash memory, storage class memory, ROM, RAM, an ASIC, magnetic storage, or any other medium from which a computer processor can read and execute program code. The program code may include processor-specific program code generated by a compiler or an interpreter from code written in any suitable computer-programming language. Examples of suitable programming language can include Hadoop, C, C++, C#, Visual Basic, Java, Python, Perl, JavaScript, ActionScript, etc.

[0105] The computing device 500 may also include a number of external or internal devices such as input or output devices. For example, the computing device 500 is illustrated with an input / output interface 508 that can receive input from input devices or provide output to output devices. A bus 506 can also be included in the computing device 500. The bus 506 can communicatively couple one or more components of the computing device 500.

[0106] The computing device 500 can execute program code 514 that can include risk assessment application 110 and the ATP application 114. The program code 514 for the risk assessment application 110 and the ATP application 114 may be resident in any suitable computer- readable medium and may be executed on any suitable processing device. For example, and as illustrated in FIG. 6, the program code 514 for the risk assessment application 110 and the ATP application 114 can reside in the memory 504 at the computing device 500 along with the program data 516 associated with the program code 514. Executing the risk assessment application 110 and the ATP application 114 can configure the processor 502 to perform at least a portion of the operations described herein.

[0107] In some aspects, the computing device 500 can include one or more output devices. One example of an output device can be or include the network interface device 510 illustrated in FIG. 6. A network interface device 510 can include any device or group of devices suitable for establishing a wired or wireless data connection to one or more data networks described herein. Non-limiting examples of the network interface device 510 can include an Ethernet network adapter, a modem, etc.31US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO

[0108] Another example of an output device can include the presentation device 512 depicted in FIG. 6. A presentation device 512 can include any device or group of devices suitable for providing visual, auditory, or other suitable sensory output. Non-limiting examples of the presentation device 512 can include a touchscreen, a monitor, a speaker, a separate mobile computing device, etc. In some aspects, the presentation device 512 can include a remote clientcomputing device that communicates with the computing device 500 using one or more data networks described herein. In other aspects, the presentation device 512 can be omitted.

[0109] The foregoing description of some examples has been presented only for the purpose of illustration and description and is not intended to be exhaustive or to limit the disclosure to the precise forms disclosed. Numerous modifications and adaptations thereof will be apparent to those skilled in the art without departing from the spirit and scope of the disclosure.32US2008 30588218 1

Claims

Attorney Docket No.: 096923-1458070Client Ref: EFX-195WOCLAIMSWhat is claimed is:

1. A computer-implemented method for verifying an identity document using a machine learning model, the method comprising: receiving, by a processor, an identity document by a computing system; extracting, by the processor, an image from the identity document; preprocessing, by the processor, the image prior to providing the image to the machine learning model; providing, by the processor, the preprocessed image to the machine learning model as input; analyzing, using the machine learning model, the input to detect whether the image was synthetically generated, the machine learning model comprising: a convolution layer configured to extract spatial features from the image; a batch normalization layer configured to stabilize a feature map; and a dense layer configured to output a probability related to the image; generating, as an output of the machine learning model, an image verification score indicating a likelihood that the image was synthetically generated; and upon determining that the image verification score is higher than a predefined threshold, generating an authentication message configured to cause a receiving device to take an action based on receipt of the authentication message; or upon determining that the image verification score is lower than the predefined threshold, generating a denial message configured to cause the receiving device to take an alternative action based on the identity document not being authenticated.

2. The method of claim 1 wherein the image is a synthetically generated deepfake image.

3. The method of claim 1 wherein preprocessing the image comprises resizing the image, normalizing pixel values of the image, performing data augmentation on the image, adjusting brightness and contrast of the image, or any combination thereof.33US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO4. The method of claim 1, further comprising: extracting, by the processor, personally identifiable information (PII) from the image using optical character recognition; validating, by the processor, the extracted PII against a trusted external data source to generate a PII validity score; and calculating, by the processor, a final verification score by combining the image verification score and the PII validity score based on predefined weights.

5. The method of claim 4, wherein the final verification score is compared against a predefined threshold to determine whether the identity document is genuine.

6. The method of claim 5, further comprising providing the final verification score to a third-party computing system.

7. The method of claim 6, further comprising generating an image authenticity score by comparing the image of the identity document with a reference image of an individual associated with the identity document.

8. A system for verifying an identity document, the system comprising: a processor; a memory storing instructions that, when executed by the processor, cause the system to: receive, by a processor, an identity document by a computing system; extract, by the processor, an image from the identity document; preprocess, by the processor, the image prior to providing the image to a machine learning model; provide, by the processor, the preprocessed image to the machine learning model as input; analyze, using the machine learning model, the input to detect whether the image was synthetically generated, the machine learning model comprising: a convolution layer configured to extract spatial features from the image;34US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO a batch normalization layer configured to stabilize a feature map; and a dense layer configured to output a probability related to the image; generate, as an output of the machine learning model, an image verification score indicating a likelihood that the image was synthetically generated; and upon determining that the image verification score is higher than a predefined threshold, generating an authentication message configured to cause a receiving device to take an action based on receipt of the authentication message; or upon determining that the image verification score is lower than the predefined threshold, generating a denial message configured to cause the receiving device to take an alternative based on the identity document not being authenticated.

9. The system of claim 8, wherein the image is a synthetically generated deepfake image.

10. The system of claim 8, wherein preprocessing the image comprises resizing the image, normalizing pixel values of the image, performing data augmentation on the image, adjusting brightness and contrast of the image, or any combination thereof.

11. The system of claim 8, the memory further comprising instructions to: extract personally identifiable information (PII) from the image using optical character recognition; validate the extracted PII against a trusted external data source to generate a PII validity score; and calculate a final verification score by combining the image verification score and the PII validity score based on predefined weights.

12. The system of claim 11, wherein the final verification score is compared against a predefined threshold to determine whether the identity document is genuine.

13. The system of claim 12, the memory further comprising instructions to provide the final verification score to a third-party computing system.35US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO14. The system of claim 13, the memory further comprising instructions to generate an image authenticity score by comparing the image of the identity document with a reference image of an individual associated with the identity document.

15. A non- transitory computer-readable medium storing instructions that, when executed by a processor, cause a computing system to perform a method for verifying an identity document, the method comprising: receiving, by a processor, an identity document by a computing system; extracting, by the processor, an image from the identity document; preprocessing, by the processor, the image prior to providing the image to a machine learning model; providing, by the processor, the preprocessed image to the machine learning model as input; analyzing, using the machine learning model, the input to detect whether the image was synthetically generated, the machine learning model comprising: a convolution layer configured to extract spatial features from the image; a batch normalization layer configured to stabilize a feature map; and a dense layer configured to output a probability related to the image; generating, as an output of the machine learning model, an image verification score indicating a likelihood that the image was synthetically generated; and upon determining that the image verification score is higher than a predefined threshold, generating an authentication message configured to cause a receiving device to take an action based on receipt of the authentication message; or upon determining that the image verification score is lower than the predefined threshold, generating a denial message configured to cause the receiving device to take an alternative action based on the identity document not being authenticated.

16. The non-transitory computer- readable medium of claim 15, wherein the image is a synthetically generated deepfake image.

17. The non-transitory computer- readable medium of claim 15, wherein preprocessing the image comprises resizing the image, normalizing pixel values of the image,36US2008 30588218 1Attorney Docket No.: 096923-1458070Client Ref: EFX-195WO performing data augmentation on the image, adjusting brightness and contrast of the image, or any combination thereof.

18. The non-transitory computer-readable medium of claim 15, further comprising: extracting personally identifiable information (PII) from the image using optical character recognition; validating the extracted PII against a trusted external data source to generate a PII validity score; and calculating a final verification score by combining the image verification score and the PII validity score based on predefined weights.

19. The non-transitory computer- readable medium of claim 18, wherein the final verification score is compared against a predefined threshold to determine whether the identity document is genuine.

20. The non-transitory computer- readable medium of claim 19, further comprising providing the final verification score to a third-party computing system.37US2008 30588218 1

Images