Method and system for safeguarding data privacy and security against adversarial attacks

Abstract

A method for analyzing datasets for vulnerabilities to adversarial attacks includes storing, by a processing server, a dataset comprising multiple data types in a database; preprocessing the dataset using data preprocessing algorithms based on the data types; performing data featurization to identify dataset features; generating, for each data type, a victim model based on the dataset; determining performance metrics of the victim models; generating attack datasets corresponding to each data type and including the performance metrics and dataset features; executing adversarial attacks, including membership inference, property inference and reconstruction attacks, on the victim models; determining adversarial attack success metrics, including accuracy, precision and F1 score; updating the attack datasets to include the adversarial attack success metrics; and generating, for each updated attack dataset, a vulnerability metric comprising a library of algorithms and models specific to the combination of adversarial attack and data type.

Description

Attorney Docket No.690248.0002 / 1WO Page 1 of 189 METHOD AND SYSTEM FOR SAFEGUARDING DATA PRIVACY AND SECURITY AGAINST ADVERSARIAL ATTACKS FIELD

[0001] The present disclosure relates to analyzing datasets for vulnerabilities to adversarial attacks (whether occurring at the dataset or model level) and identifying modifications to mitigate against identified vulnerabilities. BACKGROUND

[0002] Data, and more importantly, relevant, unique, and hard-to-acquire data, have become a valuable asset of the 21st century. Therefore, when these data provide some sort of competitive edge, whether that be commercial or military, the ability to protect these data from discovery becomes of the utmost importance. In addition, with the increase in legislation to protect data rights, such as with the European Union’s General Data Protection Regulation, this protection becomes a requirement. However, the ability to protect these data, even through the generalization and abstraction of machine-learning algorithms, is at risk. The use of AI and machine-learning solutions has increased greatly throughout industry and government; however, the understanding of the vulnerabilities and security issues within these solutions has not kept up with this trend. Recently, research groups have begun to demonstrate these weaknesses and to develop mitigation strategies. This relatively new area of research is a concentration of cybersecurity referred to as artificial intelligence (AI) security and focuses on the vulnerabilities of models and algorithms to attack.Attorney Docket No.690248.0002 / 1WO Page 2 of 189

[0003] Several key areas of attack within this field include model theft, data poisoning, evasion, and model inversion attacks. Model theft attacks seek to replicate the function of models and can lead to the loss of proprietary information, loss of revenue from deployed models, and the ability for an adversary to better predict potential actions given that they also have similar predictions as the victim. Data poisoning attacks inject malicious data into training datasets to cause general model performance degradation or directed misclassification or prediction to provide an adversarial advantage. General degradation of performance can cause a loss of trust in the system, while directed misclassification or prediction can provide calculated damage to larger organizational mission directives.

[0004] Evasion attacks seek to degrade model performance or cause directed misclassification or prediction. However, instead of tainted training data, evasion attacks utilize model inputs that seem normal to general inspection but prey on model weaknesses for the disruption of input classification or prediction. Finally, model inversion attacks seek to gather information on the training data used for the development of the attacked model. This attack is divided into property inference attacks, membership inference attacks, and reconstruction attacks.

[0005] Property inference attacks seek an understanding of a training dataset’s statistical information. An example of issues caused by this attack include the use of this information to understand competitor training datasets and, thus, build better classifiers and potentially violate intellectual property rights. Membership inference attacks seek to determine the inclusion of specific records within the training dataset ofAttorney Docket No.690248.0002 / 1WO Page 3 of 189 a model and can result in privacy-infringement issues, such as the discovery of personally identifiable information (PII) and personal health information (PHI) as well as identification of proprietary or confidential information. Membership Inference Attacks are often used as an evaluation tool within privacy metrics. Finally, reconstruction attacks are a type of model inversion attack where an adversary attempts to recover or reconstruct sensitive training data from a machine learning model by exploiting the model's learned parameters, gradients, or outputs. These attacks leverage techniques such as gradient-based optimization or generative modeling to reverse-engineer representations of original training samples, potentially exposing private information including personal data, proprietary datasets, or confidential records that were used during model training. The reconstructed data may approximate or closely resemble the original training inputs, creating significant privacy and security risks in deployed machine learning systems.This increased vulnerability to attack of minority subsets of datasets can prove troublesome for both privacy and competition. Typically, smaller subsets of data within a dataset are less represented because they are harder to obtain. In the case of health classification algorithms, these could be observations of patients with rare diseases. In the case of commercial competition, these could be examples of rare findings within a manufacturing or marketing dataset of key competitive advantages. In either of these cases, the discovery of that information by an adversary can prove detrimental to the organizations and individuals involved, whether through loss of privacy, profit, or competitive advantage. Thus, there is a need for a technological system to identify andAttorney Docket No.690248.0002 / 1WO Page 4 of 189 mitigate dataset vulnerabilities. SUMMARY

[0006] The present disclosure provides a description of systems and methods for analyzing datasets for vulnerabilities to adversarial attacks (whether occurring at the dataset or model level) and identifying modifications to mitigate against identified vulnerabilities. The methods and systems disclosed herein provide model-agnostic and attack vector agnostic vulnerability assessments of datasets against various model inversion attack genres (e.g., membership inference, property inference, and reconstruction), data poisoning, and generative Artificial Intelligence (AI) leakage. Further, the methods and systems disclosed herein provide model-agnostic and attack vector agnostic vulnerability assessments of datasets to detect Artificial Intelligence (AI) generated data, and Artificial Intelligence (AI) manipulated data. Thus, the presently disclosed methods and systems enable end-users can to harden their datasets for utilization across a broad spectrum of models and use-cases with confidence of their security against model inversion attacks – and therefore data privacy vulnerabilities, against data poisoning, against generative AI data leakage, and against AI manipulation. In addition, this development helps to educate data owners on the sources of those vulnerabilities within the dataset, how to harden the dataset, and provides an understanding of how to achieve the best performance possible while remaining secure against adversarial threats.

[0007] In some aspects, the techniques described herein relate to a method for analyzing datasets for vulnerabilities to adversarial attacks, the method including:Attorney Docket No.690248.0002 / 1WO Page 5 of 189 storing, by a processing server in a database, a dataset, the dataset including data of one or more data types; preprocessing, by the processing server, the dataset using one or more data preprocessing algorithms based on the one or more data types included in the dataset; performing, by the processing server, data featurization of the dataset based on the one or more data types included in the dataset using one or more data featurization algorithms, the data featurization identifying one or more data features of the dataset; generating, by the processing server, one or more victim models using the dataset based on the one or more data types included in the dataset, wherein each of the one or more victim models is specific to one of the one or more data types included in the dataset; determining, by the processing server, one or more performance metrics of the one or more victim models; generating, by the processing server, one or more attack datasets, wherein each of the one or more attack datasets corresponds to one of the one or more data types included in the dataset, and wherein each of the one or more attack datasets includes at least the one or more performance metrics of the one or more victim models, and the one or more data features of the dataset; executing, by the processing server, one or more adversarial attacks on the one or more victim models, wherein the one or more adversarial attacks include one or more of: a membership inference model inversion attack, a property inference model inversion attack, and a reconstruction model inversion attack; determining, by the processing server, one or more adversarial attack success metrics, the one or more adversarial attack success metrics including one or more of: an accuracy of the adversarial attack, a precision of the adversarial attack, and an F1 score of the adversarial attack; updating, by the processing server, the one or more attack datasetsAttorney Docket No.690248.0002 / 1WO Page 6 of 189 to include the one or more adversarial attack success metrics generating, by the processing server, a vulnerability metric for each of the one or more updated attack datasets, wherein each vulnerability metric is a library of algorithms and models specific to a combination of one of the one or more adversarial attacks performed on one of the one or more data types included in the dataset.

[0008] In some aspects, the techniques described herein relate to a system method for analyzing datasets for vulnerabilities to adversarial attacks, the system including a processing server including: a memory configured to store a dataset, the dataset including data of one or more data types; and a processor configured to: preprocess the dataset using one or more data preprocessing algorithms based on the one or more data types included in the dataset; perform data featurization of the dataset based on the one or more data types included in the dataset using one or more data featurization algorithms, the data featurization identifying one or more data features of the dataset; generate one or more victim models using the dataset based on the one or more data types included in the dataset, wherein each of the one or more victim models is specific to one of the one or more data types included in the dataset; determine one or more performance metrics of the one or more victim models; generate one or more attack datasets, wherein each of the one or more attack datasets corresponds to one of the one or more data types included in the dataset, and wherein each of the one or more attack datasets includes at least the one or more performance metrics of the one or more victim models, and the one or more data features of the dataset; execute one or more adversarial attacks on the one or moreAttorney Docket No.690248.0002 / 1WO Page 7 of 189 victim models, wherein the one or more adversarial attacks include one or more of: a membership inference model inversion attack, a property inference model inversion attack, and a reconstruction model inversion attack; determine one or more adversarial attack success metrics, the one or more adversarial attack success metrics including one or more of: an accuracy of the adversarial attack, a precision of the adversarial attack, and an F1 score of the adversarial attack; update the one or more attack datasets to include the one or more adversarial attack success metrics; generate vulnerability metric for each of the one or more updated attack datasets, wherein each vulnerability metric is a library of algorithms and models specific to a combination of one of the one or more adversarial attacks performed on one of the one or more data types included in the dataset. BRIEF DESCRIPTION OF THE DRAWING FIGURES

[0009] The scope of the present disclosure is best understood from the following detailed description of exemplary embodiments when read in conjunction with the accompanying drawings. Included in the drawings are the following figures:

[0010] FIG.1A is a block diagram illustrating a high level system architecture for analyzing datasets for vulnerabilities to adversarial attacks (whether occurring at the dataset or model level) and identifying modifications to mitigate against identified vulnerabilities in accordance with exemplary embodiments.

[0011] FIGS 1B-D illustrate an example user interface in the system 100 in accordance with exemplary embodiments.Attorney Docket No.690248.0002 / 1WO Page 8 of 189

[0012] FIG.2 is a block diagram illustrating the processing server in the system of FIG. 1 for analyzing datasets for vulnerabilities to adversarial attacks and identifying modifications to mitigate against identified vulnerabilities in accordance with exemplary embodiments.

[0013] FIGS.3A-3B illustrate a flow chart of a process for analyzing datasets for vulnerabilities to adversarial attacks (whether occurring at the dataset or model level) and identifying modifications to mitigate against identified vulnerabilities in the system of FIG.1 in accordance with exemplary embodiments.

[0014] FIGS.4A-4B illustrate a flow chart of an exemplary method for analyzing datasets for vulnerabilities to adversarial attacks (whether occurring at the dataset or model level) and identifying modifications to mitigate against identified vulnerabilities in accordance with exemplary embodiments.

[0015] FIG.5 is a block diagram illustrating a computer system architecture in accordance with exemplary embodiments.

[0016] Further areas of applicability of the present disclosure will become apparent from the detailed description provided hereinafter. It should be understood that the detailed description of exemplary embodiments are intended for illustration purposes only and are, therefore, not intended to necessarily limit the scope of the disclosure. DETAILED DESCRIPTION

[0017] A “computing device” may refer to a device that includes one or more electronic components (e.g., an integrated chip) that can communicate with another device orAttorney Docket No.690248.0002 / 1WO Page 9 of 189 entity. For example, a computing device may include at least one processor coupled to a memory that stores instructions or code for execution by the processor and may include a communication interface that allows the computing device to interact with other entities. A computing device can be a portable computing device that can be transported and operated by a user. A portable computing device may provide remote communication capabilities to a network. The portable computing device can be configured to transmit and receive data or communications to and from other devices. A portable computing device may be in the form of a mobile device such as a mobile phone (e.g., smart phone, cellular phone, etc.), tablets, portable media player, personal digital assistant devices (PDAs), wearable device (e.g., watch, bracelet, ring, eyeglasses, health monitoring device such as a fitness tracker, etc.), electronic reader device, etc., or in the form of a card (e.g., smart card) or a fob, etc. Examples of portable computing devices may also include portable computers (e.g., laptops, netbooks, ultrabooks, etc.). A portable computing device may also be in the form of a vehicle (e.g., an automobile) or be integrated as part of a vehicle (e.g., an infosystem of a vehicle). Other examples of computing devices may include Internet of Things (IoT) devices, smart appliances and electronics, gaming consoles, etc. A computing device may also include multiple devices or components (e.g., when a device has remote access to a network by tethering to another device, both devices taken together may be considered a computing device).

[0018] A “server” may refer to a powerful computer or cluster of computers. For example, the server computer can be a large mainframe, a minicomputer cluster, or a group of servers functioning as a unit. In one example, the server computer may be aAttorney Docket No.690248.0002 / 1WO Page 10 of 189 database server coupled to a Web server. The server computer may be coupled to a database and may include any hardware, software, other logic, or combination of the preceding for servicing the requests from one or more client computers. The server computer may comprise one or more computational apparatuses and may use any of a variety of computing structures, arrangements, and compilations for servicing the requests from one or more client computers. System for Analyzing Datasets for Vulnerabilities to Adversarial Attacks

[0019] FIG.1 illustrates a system 100 for analyzing datasets for vulnerabilities to adversarial attacks, in accordance with exemplary embodiments. The system 100 can include a processing server 102, one or more user computing devices 104A-104N, and one or more data sources 106A-106N.

[0020] The processing server 102 may be any computing device capable of storing, compiling, and organizing audio, visual, or textual data and receiving and sending that data to and from other computing devices, such as the user computing devices 104A- 104N, and / or the data sources 106A-106N via one or more communication methods, such as radio frequency, local area networks, wireless area networks, cellular communication networks, Bluetooth, the Internet, etc. It can be appreciated that any number of processing servers 102 (e.g., 102A, 102B ...102N) can be a part of the system 100, including a processing server 102 as illustrated. In the system 100, the processing server 102 can make up an organization computing infrastructure network. For example, an organization’s computing infrastructure network can include one or more processing servers 102 that host a cloud networking environment of an organization, one or more processing servers 102 that host one or more softwareAttorney Docket No.690248.0002 / 1WO Page 11 of 189 programs used by an organization (e.g., application programming interfaces (APIs)), one or more processing servers 102 that host an internal network (e.g., an intranet) for an organization, etc. The organization can be any entity that utilizes one or more server computers to form a computer network. In exemplary embodiments, the processing server 102 hosts a computer program for analyzing datasets for vulnerabilities to adversarial attacks as discussed in more detail below. The processing server 102 may be implemented in the computer system 500 illustrated in FIG.5 using hardware, software executed on hardware, firmware, non-transitory computer readable media having instructions stored thereon, or a combination thereof and may be implemented in one or more computer systems or other processing systems.

[0021] The user computing devices 104A-104N may be any computing device capable of storing, compiling, and organizing audio, visual, or textual data and receiving and sending that data to and from other computing devices, such as the processing server 102, and / or the one or more data sources 106A-106D via one or more communication methods, such as radio frequency, local area networks, wireless area networks, cellular communication networks, Bluetooth, the Internet, etc. It can be appreciated that any number of user computing devices 104 (e.g., 104A, 104B ...104N) can be a part of the system 100, including a single user computing device 104 or multiple user computing devices 104. In exemplary embodiments, the user computing devices 104 is associated with a user that owns and / or maintains data to be analyzed for vulnerability to adversarial attacks by the processing server 102. The user computing devices 104A-104N may be implemented in the computer system 500 illustrated in FIG.5 using hardware, software executed on hardware, firmware, non-transitoryAttorney Docket No.690248.0002 / 1WO Page 12 of 189 computer readable media having instructions stored thereon, or a combination thereof and may be implemented in one or more computer systems or other processing systems.

[0022] In the system 100, the data sources 106A-106D may be any computing device capable of storing, compiling, and organizing audio, visual, or textual data and receiving and sending that data to and from other computing devices, such as the user processing server 102, and / or the one or more user computing devices 104A-104N via one or more communication methods, such as radio frequency, local area networks, wireless area networks, cellular communication networks, Bluetooth, the Internet, etc. For example, the data sources 106A-106D may be a remote database, a server, or any other suitable data storage device. It can be appreciated that any number of data sources 106 (e.g., 106A, 106B ...106N) can be a part of the system 100, including a single data source 106. In the system 100, the data sources 106A-106N store data to be used to develop the vulnerability metrics 218 and / or to be analyzed for vulnerability to adversarial attacks by the processing server 102. Each of the data sources 106A- 106N may be implemented in the computer system 500 illustrated in FIG.5 using hardware, software executed on hardware, firmware, non-transitory computer readable media having instructions stored thereon, or a combination thereof and may be implemented in one or more computer systems or other processing systems.

[0023] In the system 100, the processing server 102 featurizes datasets in order to map dataset features to vulnerabilities associated with model inversion attacks such as membership inference, property inference, and reconstruction attacks. The first step in the methods and systems disclosed herein for analyzing datasets for vulnerabilitiesAttorney Docket No.690248.0002 / 1WO Page 13 of 189 to adversarial attacks is to develop a vulnerability metric (e.g., the vulnerability metrics 218). Each category of adversarial attack (e.g., membership inference, property inference, and reconstruction) as well as each data type or category of data type contained in a dataset (e.g., tabular, natural language, computer vision, image recognition, object detection, still image, full motion video, and generative AI, etc.) has its own set of models and algorithms to properly evaluate vulnerability. In other words, membership inference attacks on one category of tabular datasets will be one set of models and algorithms (e.g., a first vulnerability metric), membership inference attacks on one category of natural language datasets will be another set of models and algorithms (e.g., a second vulnerability metric), and so on. In order to generate the vulnerability metrics 218, a multitude of datasets (e.g., from the data sources 106A- 106N) must be evaluated, preprocessed, modeled, attacked, and evaluated again. This process is discussed in detail below and represented as the pathway for one dataset. Each dataset within this “warehouse” of datasets (meaning a large collection of datasets inclusive of the diversity one would see in the wild) is processed to develop a final “attack dataset” which will be used to develop the vulnerability metric for each attack / data type pair as described above. Data Preprocessing

[0024] To begin, a dataset (e.g., retrieved from the data sources 106A-106N) is established as a “raw dataset” meaning that it has not been processed in any manner. The processing server 102 then preprocesses the data of the raw dataset using one or more preprocessing methods. The one or more preprocessing methods can include, for example, but are not limited to: data cleaning, feature scaling and transformation,Attorney Docket No.690248.0002 / 1WO Page 14 of 189 feature engineering, text cleaning, tokenization and normalization, stopword and noise removal, text representation, miscellaneous NLP-specific, basic image cleaning, image resizing and cropping, pixel normalization, noise reduction, data augmentation, and data balancing, etc.

[0025] Data cleaning can include, for example, but not limited to, handling missing values (e.g., removing missing data through row and / or column elimination, imputation (mean, median, mode, constant, interpolation, KNN, MICE, etc.)), handling duplicates (e.g., exact match removal, fuzzy matching deduplication), correcting data types (e.g., convert strings to categorical, dates to datetime, etc.), handling outliers (e.g., Z-Score, IQR, winsorization, capping, etc.), noise filtering (e.g., smoothing (moving averages, exponential smoothing, etc.)). Feature scaling and transformation can include, for example, but not limited to, scaling (e.g., standardization, min-max scaling, robust scaling, etc.), transformation (e.g., log, square root, Box-Cox, Yeo-Johnson, etc.), encoding categorical features (e.g., one-hot encoding, label encoding, ordinal encoding, target encoding, frequency encoding, etc.), and date / time feature extraction. Feature engineering can include, for example, but not limited to, ratios, differences, interaction features, aggregations, polynomial features, etc. Text cleaning can include, for example, but not limited to, case normalization, punction removal, HTML tag removal, metadata, stripping whitespace, fixing encoding issues, etc. Tokenization & normalization can include, for example, but not limited to, word tokenization, subword tokenization, lemmatization, and stemming, etc. Stopword and noise removal can include, for example, but not limited to, removing stopwords, removing special characters or emojis, removing rare or overly frequent words, etc. Text representationAttorney Docket No.690248.0002 / 1WO Page 15 of 189 can include, for example, but not limited to, bag-of-words, TF-IDF, word embeddings, contextual embeddings, sentence embeddings, etc. Miscellaneous NLP-Specific can include, for example, but not limited to, spelling correction, entity recognition & replacement, handling code-switching / multilingual text, sentence segmentation for long documents, etc. Basic image cleaning can include, for example, but not limited to, removing corrupted or unreadable files, converting to consistent color space, standardizing file format, etc. Image resizing and cropping can include, for example, but not limited to, resizing, center cropping, random cropping, padding, etc. Pixel normalization can include, for example, but not limited to, min-max normalization, standardization, histogram equalization / CLAHE, etc. Noise reduction can include, for example, but not limited to, Gaussian blur, median filtering, denoising autoencoders, etc. Data augmentation can include, for example, but not limited to, geometric (e.g., rotation, flipping, scaling, translation, shearing, etc.), color (brightness, contrast, saturation, hue adjustments, etc.), CutMix, MixUp, random erasing, adversarial noise injection, etc.

[0026] In addition to analyzing a dataset for vulnerability to model inversion attacks as discussed above, the processing server 102 can also analyze a dataset’s (e.g., the data 208) vulnerability to generative AI data leakage. In such embodiments, the preprocessing algorithms 210 utilized by the processing server 102 can include, for example, but not limited to, core text processing, text structuring and representation, image and visual data preprocessing, audio preprocessing, video preprocessing, multimodal alignment, data augmentation, and data balancing and sampling, etc. Core text preprocessing can include, for example, but not limited to, text cleaning (e.g., caseAttorney Docket No.690248.0002 / 1WO Page 16 of 189 normalization, removal of extra whitespace, HTML tags, and markup, fixing encoding issues and broken characters, and standardizing punctuation and spacing, etc.), tokenization and normalization (e.g., word tokenization, subword / BPE tokenization, sentence segmentation, lemmatization, stemming, etc.), noise reduction (e.g., removal of stopwords, special characters, and irrelevant metadata, etc.), and formatting consistency (e.g., converting all text to a common format (UTF-8, Unicode normalization, etc.), etc. Text Structuring & Representation can include, for example, but not limited to, sentence / paragraph segmentation (e.g., splitting long text into model-friendly chunks while preserving semantic context, etc.), prompt / response structuring (e.g., consistent formatting for conversational or instruction-based datasets, etc.), context window preparation (e.g., merging related samples to fit within the target model’s context length, etc.), and text encoding & embeddings (for hybrid workflows) (e.g., pre-computing embeddings for conditioning or retrieval-augmented generation), etc. Image and visual data preprocessing can include, for example, but not limited to, basic cleaning (e.g., removing corrupted files, standardizing formats (PNG, JPEG, WebP), and converting to a consistent color space (RGB), etc.), resizing and cropping (e.g., maintaining aspect ratios or applying center / random crops, etc.), pixel scaling and normalization (e.g., min-max scaling, mean-std normalization, histogram equalization, etc.), and background and artifact removal (e.g., removing watermarks, compression artifacts, or irrelevant borders, etc.), etc. Audio preprocessing can include, for example, but not limited to, standardization (e.g., consistent sample rate, bit depth, mono / stereo formatting, etc.), noise reduction and cleaning (e.g., filtering background noise, removing silences, trimming edges, etc.), segmentation (e.g.,Attorney Docket No.690248.0002 / 1WO Page 17 of 189 splitting long recordings into context-sized chunks with overlap where needed, etc.), and feature extraction (optional) (e.g., spectrograms, MFCCs, or other conditioning features), etc. Video preprocessing can include, for example, but not limited to, frame extraction and sampling (e.g., selecting representative frames or sequences, etc.), resolution standardization (e.g., scaling to a consistent size while preserving aspect ratio, etc.), frame normalization (e.g., brightness / contrast normalization across sequences, etc.), and audio-video alignment (e.g., ensuring synchronized and trimmed multimodal data, etc.), etc. Multimodal alignment can include, for example, but not limited to, data synchronization (e.g., aligning text, image, audio, or video modalities for joint training, etc.), format harmonization (e.g., converting all modalities into compatible encoding / metadata formats, etc.), and temporal and spatial alignment (e.g., matching sequence timing or visual cropping across modalities, etc.), etc. Data augmentation can include, for example, but not limited to, text augmentation (e.g., paraphrasing, back-translation, style transfer, controlled synonym replacement), image augmentation (e.g., geometric transformations (rotation, flipping, scaling), color jitter, CutMix, MixUp), audio augmentation (e.g., pitch shifting, time-stretching, speed changes, additive noise, etc.), and video augmentation (e.g., frame skipping, temporal reversal, spatial augmentation, etc.), etc. Data balancing and sampling can include, for example, but not limited to, ensuring balanced class, domain, or style distributions, stratified sampling for diverse content representation, and oversampling / undersampling to match target training profiles, etc.

[0027] In addition to analyzing a dataset for vulnerability to model inversion attacks and AI leakage as discussed above, the processing server 102 can also analyze aAttorney Docket No.690248.0002 / 1WO Page 18 of 189 dataset’s (e.g., the data 208) vulnerability to reinforcement leaning attacks. In the context of reinforcement leaning attacks, the dataset (e.g., the data 208) can include for example, but not limited to, offline RL datasets, experience replay buffers, demonstration datasets for imitation learning, and / or state–action–reward trajectories generated by simulators or real-world agents. In such embodiments, the preprocessing algorithms 210 utilized by the processing server 102 can include, for example, but not limited to, invalid transition removal, out-of-bounds filtering, duplicate transition handling, reward normalization, reward clipping, reward shaping, sparse reward smoothing, missing observation imputation, feature scaling & normalization, encoding categorical states, dimensionality reduction, action clipping, discrete action remapping, action normalization, frame skipping / stacking, trajectory segmentation, episode padding, image observations (vision-based RL), textual observations (language-based RL), multimodal observations, observation augmentation, trajectory augmentation, state–action distribution rebalancing, reward distribution equalization, and policy coverage control, etc.

[0028] In addition to analyzing a dataset for vulnerability to adversarial attacks as discussed above, the processing server 102 can also analyze a dataset’s (e.g., the data 208) vulnerability to data poisoning, determine a required threshold of poisoning that would adversely affect the dataset (e.g., the data 208), flag potentially poisoned data samples, and provide a method for cleaning and / or neutralizing the dataset (e.g., the data 208) so that it can still be used by a user (e.g., of the user computing device 104A-104N. In such embodiments, the preprocessing algorithms 210 utilized by the processing server 102 are substantially similar to those used for the model inversionAttorney Docket No.690248.0002 / 1WO Page 19 of 189 vulnerability analysis. Additional data preprocessing methods would be an apparent part to those skilled in the art of data processing.

[0029] By ensuring the evaluation across a large, and ever-increasing variety of preprocessing steps across all data type domains (tabular, natural language, computer vision, generative AI), the disclosed process remains agnostic to the users (e.g., users of the user computing device 104A-104N) eventual chosen preprocessing methodology. In the system 100, the users of the user computing device 104A-104N can choose between this “agnostic method” which captures all methods of preprocessing, or the users can select certain preprocessing methods which they know they will use for their own datasets. In embodiments where the users of the user computing devices 104A-104N preselect certain preprocessing methods, the user can provide the processing server 102 with additional information that can be used by the processing server 102 to provide a stronger vulnerability measurement and hardening decision and eventual hardened dataset. Further, in embodiments, the preselection of preprocessing methods by the users of the user computing devices 104A-104N can be integrated within the final vulnerability metric (e.g., in a dataset vulnerability report discussed in more detail below) by toggling on or off the various features (e.g., via a user interface) that are created through this process as they pertain to user selected methodologies. Data Featurization

[0030] Once the dataset (e.g., the data 208) has been preprocessed as discussed above, the processing server 102 featurizes both the raw and preprocessed datasets (e.g., the data 208) using one or more data featurization algorithms 212. This one orAttorney Docket No.690248.0002 / 1WO Page 20 of 189 more data featurization algorithms 212 can include, for example, but are not limited to: a number of observations (e.g., the quantity of observations within the original dataset), class entropy (e.g., entropy as defined through the number of observations in each class), number of classes (e.g., the number of classes), number of features (e.g., the number of features in the original dataset), number of features after one hot encoding (e.g., the number of features after the dataset has been processed using one hot encoding on categorical features), proportion of categorical features (e.g., the proportion of categorical features in respect to the original number of features), proportion of binary features (e.g., the proportion of binary features in respect to the original number of features), proportion of numerical features (e.g., the proportion of numerical features in respect to the original number of features), variance of the entropy of features (e.g., an entropy is calculated for each feature - this is the variance of that array), maximum of the entropy of features (an entropy is calculated for each feature - this is the maximum value of that array), minimum of the entropy of features (e.g., an entropy is calculated for each feature - this is the minimum value of that array), mean of the entropy of features (e.g., an entropy is calculated for each feature - this is the mean of that array), maximum of the numerical feature range (e.g., the maximum range of values of the numerical features) minimum of the numerical feature range (e.g., the minimum range of values of the numerical features), global maximum of the numerical feature range (e.g., the global maximum range of values of the numerical features as defined by the largest numerical value minus the smallest numerical value across all numerical features), global minimum of the numerical feature range (e.g., the global minimum range of values of the numerical features asAttorney Docket No.690248.0002 / 1WO Page 21 of 189 defined by the smallest, upper numerical value minus the largest, lower numerical value across all numerical feature ranges), mean of mean label distances (e.g., the distances of observations within each label were calculated using cityblock distances and then averaged within that label. this feature is the mean of those averages), variance of mean label distances (e.g., the distances of observations within each label were calculated using cityblock distances and then averaged within that label. this feature is the variance of those averages), mean of mean label minimum distances (e.g., the distances of observations within each label were calculated using cityblock distances - this feature is the mean of the minimum of distances for each label), variance of mean label minimum distances (e.g., the distances of observations within each label were calculated using cityblock distances - this feature is the variance of the minimum of distances for each label), mean of mean label maximum distances (e.g., the distances of observations within each label were calculated using cityblock distances - this feature is the mean of the maximum of distances for each label), variance of mean label maximum distances (e.g., the distances of observations within each label were calculated using cityblock distances - this feature is the variance of the maximum of distances for each label), mean of feature-feature correlation (e.g., this feature is the mean of feature to feature correlation values), maximum of feature- feature correlation (e.g., this feature is the maximum value of feature to feature correlation values), minimum of feature-feature correlation (e.g., this feature is the minimum value of feature to feature correlation values), mean of variance of feature- feature correlation (e.g., this feature is the mean of the variance of feature to feature correlation values), variance of the mean of feature-feature correlation (e.g., thisAttorney Docket No.690248.0002 / 1WO Page 22 of 189 feature is the variance of the means of feature to feature correlation values), number of pcas required to explain 75% variance (e.g., the number of principal components required to explain 75% of the variance of the dataset), cond num 2norm (e.g., condition number of 2-norm), width ratio (e.g., the ratio of the number of observations of the original dataset to the number of features of the original dataset), width ratio of one hot encoding (e.g., the number of observations of the original dataset to the number of features after one hot encoding the categorical variables), maximum number of categories (e.g., the maximum number of categories that any categorical feature in the original dataset contained), minimum number of categories (e.g., the minimum number of categories that any categorical feature in the original dataset contained), mean number of categories (e.g., the average number of categories for each categorical feature in the original dataset), variance of number of categories (e.g., the variance of the number of categories for each categorical feature in the original dataset), mean feature-feature correlation grouped by label (e.g., the mean of the feature to feature correlation when grouped by label), maximum feature-feature correlation grouped by label (e.g., the maximum of the feature to feature correlation when grouped by label), minimum feature-feature correlation grouped by label (e.g., the minimum of the feature to feature correlation when grouped by label), mean of the variance of feature-feature correlation grouped by label (e.g., the average of the variance of feature to feature correlations when grouped by label), variance of the means of feature-feature correlation grouped by label (e.g., the variance of the means of the feature to feature correlations when grouped by label), canonical correlation (e.g., canonical correlation), maximum feature skewness (e.g., the maximumAttorney Docket No.690248.0002 / 1WO Page 23 of 189 skewness of the features in the data set), minimum feature skewness (e.g., the minimum skewness of the features in the data set), mean feature skewness (e.g., the mean skewness of the features in the data set), variance feature skewness (e.g., the variance skewness of the features in the data set), maximum feature kurtosis (e.g., the maximum kurtosis of the features in the data set), minimum feature kurtosis (e.g., the minimum kurtosis of the features in the data set), mean feature kurtosis (e.g., the mean kurtosis of the features in the data set), variance feature kurtosis (e.g., the variance kurtosis of the features in the data set), standard deviation ratio of features (t e.g., he geometric mean ratio of standard deviations of the individual populations to the pooled standard deviation), maximum standard deviation ratio of features by label (e.g., the maximum of the standard deviation ratios of features as described above but grouped by label), minimum standard deviation ratio of features by label (e.g., the minimum of the standard deviation ratios of features as described above but grouped by label), mean of the standard deviation ratio of features by label (e.g., the mean of the standard deviation ratios of features as described above but grouped by label), variance of the standard deviation ratio of features by label (e.g., the variance of the standard deviation ratios of features as described above but grouped by label), mean mutual information of features (e.g., the mean mutual information of features), maximum mutual information of features (e.g., the maximum mutual information of features), minimum mutual information of features (e.g., the minimum mutual information of features), variance of the mutual information of features (e.g., the variance of the mutual information of features), mean mutual information of features grouped by label (e.g., the mean mutual information of features grouped by label),Attorney Docket No.690248.0002 / 1WO Page 24 of 189 maximum mutual information of features grouped by label (e.g., the maximum mutual information of features grouped by label), minimum mutual information of features grouped by label (e.g., the minimum mutual information of features grouped by label), variance of the mutual information of features grouped by label (e.g., the variance of the mutual information of features grouped by label), and equivalent number of attributes (e.g., entropy of class divided by the mean mutual information of class and attributes).

[0031] In addition to featurizing datasets as discussed above, the processing server 102 can also analyze datasets bound for generative AI solutions. In such embodiments, the data featurization algorithms 212 utilized by the processing server 102 can include, for example, but not limited to, basic dataset composition, lexical and token-level statistics (text), semantic diversity and redundancy, structural properties (text, code, or multimodal), image and visual feature statistics, audio and video features statistics, overlap and duplication, complexity and information content, embedding-space characteristics, and special content flags, etc.

[0032] In addition to featurizing datasets as discussed above, the processing server 102 can also featurize reinforcement learning datasets. In such embodiments, the data featurization algorithms 212 utilized by the processing server 102 can include, for example, but are not limited to, core data statistics, state space characteristics, action space characteristics, reward distribution features, transition dynamics features, trajectory diversity and similarity, correlation and structure metrics, and advanced distributional and information metrics, etc. By calculating these features for every pre- processed dataset variant, the RL vulnerability evaluation gains a rich and consistentAttorney Docket No.690248.0002 / 1WO Page 25 of 189 set of descriptors capturing: State space complexity, Action space diversity, Reward sparsity and distribution, Trajectory diversity, and Transition dynamics stability. These descriptors are algorithm-agnostic and support downstream modeling to predict vulnerabilities—such as memorization of rare trajectories, overfitting to specific states, or excessive dependence on high-reward outliers—across different RL training and fine-tuning approaches.

[0033] In addition to featurizing datasets as discussed above, the processing server 102 can also featurize data poisoning datasets. In such embodiments, the data featurization algorithms 212 utilized by the processing server 102 are substantially similar to those used for the model inversion vulnerability analysis. Additional data featurization methods would be apparent to those skilled in the art of data featurization. Data Modeling

[0034] Once the dataset (e.g., the data 208) has been preprocessed and featurized as discussed above, the processing server integrates the dataset (e.g., the data 208) across a library of models (e.g., the data modeling algorithms 214) based on the various methods of preprocessing established before. By modeling the dataset (e.g., the data 208) across a variety of modeling methods (e.g., the data modeling algorithms 214) and preprocessing methods, the vulnerability and eventual hardening methodologies remain – in addition to the aforementioned preprocessing agnosticism – agnostic to modeling method. The models generated by the processing server 102 by applying the data modeling methods (e.g., the data modeling algorithms 214) areAttorney Docket No.690248.0002 / 1WO Page 26 of 189 referred to as “victim models” as these victim models will be attacked using one or more adversarial attack algorithms in the next step of the process. Again, as with the preprocessing, the users of the user computing device 104A-104N can select certain modeling methods (e.g., the data modeling algorithms 214) to use and the users of the user computing devices 104A-104N can provide the processing server 102 with additional information can be used to provide a more nuanced vulnerability and hardening evaluation, as discussed in more detail below. The processing server 102 selects modeling methods (e.g., the data modeling algorithms 214) to apply to the dataset (e.g., the data 208) based on the data types included in the dataset. For example, the processing server 102 can select one or more modeling methods (e.g., the data modeling algorithms 214) for tabular data, one or more modeling methods (e.g., the data modeling algorithms 214) for natural language data, one or more modeling methods (e.g., the data modeling algorithms 214) for computer vision data, one or more modeling methods (e.g., the data modeling algorithms 214) for generative AI computer vision data, and one or more modeling methods (e.g., the data modeling algorithms 214) for generative AI large language models (LLMs), etc.

[0035] For tabular data, the one or modeling methods (e.g., the data modeling algorithms 214) applied by the processing server 102 can include, for example, but not limited to, classical statistical models, tree-based models, instance-based methods, kernel and margin-based methods, probabilistic models, and deep learning for tabular data. Classical statistical models can include, for example, but are not limited to linear models (e.g., Linear regression, ridge regression, LASSO, Elastic Net, Generalized Linear Models (GLMs) – logistic regression, Poisson regression), and discriminantAttorney Docket No.690248.0002 / 1WO Page 27 of 189 analysis (e.g., LDA (Linear Discriminant Analysis), QDA (Quadratic Discriminant Analysis), etc. Tree-based models can include, for example, but are not limited to, decision trees (e.g., CART, ID3, C4.5, C5.0), random forests, Gradient Boosting Machines (GBM) (e.g., XGBoost, LightGBM, CatBoost), and extremely randomized trees (e.g., ExtraTrees), etc. Instance-based methods can include, for example, but are not limited to, k-Nearest Neighbors (KNN), and prototype methods (e.g., condensed KNN, LVQ), etc. Kernel & margin-based methods can include, for example, but are not limited to, support vector machines (linear, polynomial, RBF, custom kernels), and kernel ridge regression, etc. Probabilistic models can include, for example, but are not limited to, Naive Bayes (Gaussian, Multinomial, Bernoulli), Bayesian Networks, Gaussian Mixture Models, Hidden Markov Models (for temporal tabular data), etc. Deep learning for tabular data can include, for example, but are not limited to, fully connected feedforward neural networks (e.g., MLPs), TabNet (e.g., attention-based tabular model), TabTransformer, and NODE (e.g., Neural Oblivious Decision Ensembles), etc.

[0036] For natural language data, the one or modeling methods (e.g., the data modeling algorithms 214) applied by the processing server 102 can include, for example, but not limited to, Classical / feature-engineered models, Distributed representations, recurrent neural network (RNN)-based deep learning, convolutional neural network (CNN)-based natural language processing (NLP), attention-based architectures, sequence-to-sequence models, and specialized models, etc. Classical / feature-engineered models can include, for example, but are not limited to, Bag-of- Words + classical ML (e.g., Logistic Regression, SVM, Naive Bayes), TF–IDF +Attorney Docket No.690248.0002 / 1WO Page 28 of 189 classical ML, N-gram language models, etc. Distributed representations can include, for example, but are not limited to, word embeddings + machine learning (ML) models (e.g., Word2Vec, GloVe, FastText), and Doc2Vec for document classification, etc. RNN-based deep learning can include, for example, but are not limited to, simple RNNs, LSTMs (Long Short-Term Memory networks), GRUs (Gated Recurrent Units), and bi-directional RNNs, etc. CNN-based NLP can include, for example, but are not limited to, text CNNs for sentence classification, and character-level CNNs, etc. Attention-based architectures can include, for example, but are not limited to, self- attention + RNN hybrids, transformers (e.g., encoder-only, decoder-only, encoder– decoder), BERT, RoBERTa, DistilBERT, ALBERT (e.g., encoder-based), GPT family, LLaMA, Mistral (decoder-based), T5, BART, and mBART (e.g., encoder–decoder), etc. Sequence-to-sequence models can include, for example, but are not limited to, Seq2Seq with attention (e.g., pre-transformer), and pointer-generator networks (e.g., summarization). Specialized models can include, for example, but are not limited to, retrieval-augmented generation (RAG), long-sequence transformers (e.g., Longformer, BigBird), and prompt-tuning and adapter-based fine-tuning, etc.

[0037] For computer vision data, the one or modeling methods (e.g., the data modeling algorithms 214) applied by the processing server 102 can include, for example, but not limited to, classical vision models, early deep CNNs, modern CNN architectures, vision transformers (ViTs), object detection, image segmentation, and specialized computer vision models, etc. Classical vision models can include, for example, but are not limited to, feature extraction + ML classifier (e.g., SIFT, SURF, ORB, HOG, LBP, Classifiers: SVM, Random Forest, KNN, Logistic Regression), etc. Early deep CNNsAttorney Docket No.690248.0002 / 1WO Page 29 of 189 can include, for example, but are not limited to, LeNet, AlexNet, and VGGNet, etc. Modern CNN architectures can include, for example, but are not limited to, ResNet family (e.g., ResNet, ResNeXt, Wide ResNet), DenseNet, inception family (e.g., GoogLeNet, Inception-v3, Inception-ResNet), EfficientNet / EfficientNetV2, and ConvNeXt, etc. Vision transformers (ViTs), can include, for example, but are not limited to, Vision Transformer (ViT), DeiT (Data-efficient Image Transformers), Swin Transformer, and ConvNeXt + ViT hybrids, etc. Object detection can include, for example, but are not limited to, two-stage (e.g., R-CNN, Fast R-CNN, Faster R-CNN, Mask R-CNN), one-stage (e.g., YOLO family (YOLOv1–YOLOv9), SSD, RetinaNet), and transformer-based (e.g., DETR, DINO-DETR), etc. Image segmentation can include, for example, but are not limited to, FCN (Fully Convolutional Networks), U- Net, U-Net++, DeepLab family (e.g., DeepLabv3, DeepLabv3+), and Segment Anything Model (SAM), etc. Specialized CV models can include, for example, but are not limited to, image captioning (e.g., Show and Tell, Show-Attend-and-Tell, BLIP), Multi-modal (e.g., CLIP, ALIGN, Florence), super-resolution (e.g., SRCNN, ESRGAN, SwinIR), etc.

[0038] For generative AI computer vision data, the one or modeling methods (e.g., the data modeling algorithms 214) applied by the processing server 102 can include, for example, but not limited to, GANs (Generative Adversarial Networks), Diffusion models, Autoregressive image models, VAE (Variational Autoencoders), and NeRF- based models, etc. GANs (Generative Adversarial Networks) can include, for example, but are not limited to, DCGAN, StyleGAN, StyleGAN2, StyleGAN-XL, CycleGAN (e.g., image-to-image translation), Pix2Pix, and GauGAN, etc. Diffusion models can include,Attorney Docket No.690248.0002 / 1WO Page 30 of 189 for example, but are not limited to, DDPM (Denoising Diffusion Probabilistic Models), Stable Diffusion, Imagen, DALL^E, and Midjourney, etc. Autoregressive image models can include, for example, but are not limited to, PixelRNN, and PixelCNN, etc. VAE (Variational Autoencoders) can include, for example, but are not limited to, standard VAE, Beta-VAE, VQ-VAE, and VQ-VAE-2, etc. NeRF-based models (3D generation) can include, for example, but are not limited to, NeRF, Instant-NGP, and Mip-NeRF, etc.

[0039] For generative AI large language model (LLM) data, the one or modeling methods (e.g., the data modeling algorithms 214) applied by the processing server 102 can include, for example, but not limited to, Autoregressive language models, Encoder–decoder generative models, Diffusion language models, Mixture-of-experts architectures, Retrieval-augmented generation (RAG), Instruction-tuned / chat models, Parameter-efficient tuning methods, and Multi-modal generative models, etc. Autoregressive language models can include, for example, but are not limited to, GPT family, LLaMA, Falcon, Mistral, and Gemini, etc. Encoder–decoder generative models can include, for example, but are not limited to, T5, BART, and mT5, etc. Mixture-of- experts architectures can include, for example, but are not limited to, Switch Transformer, GLaM, and Mixtral, etc. Retrieval-augmented generation (RAG) can include, for example, but are not limited to, LLM + vector database retrieval, etc. Instruction-tuned / chat models can include, for example, but are not limited to, InstructGPT, ChatGPT, Claude, PaLM 2 Chat, and Gemini Chat. Parameter-efficient tuning methods can include, for example, but are not limited to, LoRA, QLoRA, Prefix-Attorney Docket No.690248.0002 / 1WO Page 31 of 189 tuning, and Adapter-tuning, etc. Multi-modal generative models can include, for example, but are not limited to, Flamingo, GPT-4V, Kosmos-1, and LLaVA, etc.

[0040] In addition to data modeling discussed above, the processing server 102 can also model datasets bound for integration into generative AI solutions. In such embodiments, the data modeling algorithms 214 utilized by the processing server 102 can include, for example, but not limited to, autoregressive transformers, encoder- decoder models, mixture-of-experts architectures, instruction-tuned / chat models, parameter-efficient fine-tuning (PEFT), long-context models, retrieval-augmented generation (RAG), specialized fine-tuning, GANs (Generative Adversarial Networks), diffusion models, autoregressive image models, variational autoencoders, NeRF- based 3D generation, vision-language generative models, diffusion video models, GAN-based video models, autoregressive video generation, multimodal video-text, text-image, text-video, text-audio, unified multimodal, full model fine-tuning, PEFT methods, continued pretraining, instruction tuning, RLHF and post-training alignment, and multi-stage training. In such AI model embodiments, once each model variant is trained or fine-tuned by the processing server 102, performance metrics (e.g., loss, perplexity, FID, BLEU, ROUGE, CIDEr, accuracy, recall, embedding similarity) are recorded and incorporated into the full evaluation dataset. At this point, the evaluation dataset consists of: dataset-level features (from the featurization step), and model- level performance metrics across architectures and fine-tuning methods. The models in this stage are referred to as “victim models”—representing the real-world systems an organization might deploy and which could, in production, be susceptible to data leakage under adversarial prompting or sampling.Attorney Docket No.690248.0002 / 1WO Page 32 of 189

[0041] In addition to data modeling discussed above, the processing server 102 can also model reinforcement learning datasets. In such embodiments, the data modeling algorithms 214 utilized by the processing server 102 can include, for example, but not limited to, classical value iteration models, tabular temporal-difference learning, deep value function approximation, vanilla policy gradient, actor-critic methods, trust region methods, entropy-regularized methods, classical model-based control, deep model- based learning, fitted Q iteration, BCQ (Batch-Constrained Q-Learning), CQL (Conservative Q-Learning), IQL (Implicit Q-Learning), AWAC (Advantage-Weighted Actor-Critic), decision transformer and trajectory transformer, behavioral cloning (BC), inverse reinforcement learning (IRL), offline pretraining + online fine-tuning, hierarchal RL, multi-agent RL, visual RL, language-conditional RL, and robotics and continuous control, etc.

[0042] In addition to the data modeling discussed above, the processing server 102 can also model data poisoning datasets. In such embodiments, the data modeling algorithms 214 utilized by the processing server 102 are substantially similar to those used for the model inversion vulnerability analysis. Additional data modeling methods would be apparent to those skilled in the art of data modeling. Once the processing server 102 has generated the victim models using the one or more modeling methods (e.g., the data modeling algorithms 214) as discussed above, the processing server 102 evaluates the performance metrics of the victim models such as, but not limited to, precision, accuracy, and F1 score, etc. Once the processing server 102 has determined one or more performance metrics for the victimAttorney Docket No.690248.0002 / 1WO Page 33 of 189 models, the processing server 102 generates one or more attack datasets (e.g., the attack data 209) which includes the dataset features from the data featurization of the dataset and the victim model performance metrics. The processing server 102 generates an attack dataset (e.g., attack data 209) for each combination of data type included in the dataset (e.g., the data 208) and adversarial attack (discussed in more detail below).Attacking the Models

[0043] Once the processing server 102 has generated the victim models and the attack dataset (e.g., the attack data 209) using the one or more modeling methods (e.g., the data modeling algorithms 214) as discussed above, the processing server 102 executes one or more adversarial attacks (e.g., the model attack algorithms 216) on the victim models. The one or more adversarial attacks (e.g., the model attack algorithms 216) can include one or more model inversion attacks within each category (e.g., each data type included in the dataset) such as, membership inference model inversion attacks, property inference model inversion attacks, and reconstruction model inversion attacks.

[0044] Membership Inference Attacks (MIA) can include, for example, but are not limited to, attack surface dimensions / signals, black-box, white-box, and gray-box access, signals (e.g., predicted class, confidence / logits, loss / entropy, margins, calibration error, gradients / updates (FL), dropout / MC uncertainty), per-instance (single point) and aggregate (set) membership, label-only setting (no confidences) and soft outputs, canonical families / named methods, shadow-model attacks, thresholding on confidence / loss, entropy margins, top-k gap, calibrated score attacks and meta-Attorney Docket No.690248.0002 / 1WO Page 34 of 189 classifiers, likelihood-ratio / Bayes optimal tests (hypothesis testing), label-only membership via data augmentation and decision boundary distances, metric learning / embedding-space similarity (nearest-neighbor in penultimate layer), out-of-distribution and overfitting gap exploitation, white-box gradient-based MIAs (access to weights / gradients), federated learning MIAs from client updates (round-level leakage). More specifically, membership Inference Attacks (MIA) on tabular data can include, but are not limited, for example, confidence / loss thresholding on held-out probes, shadow models per feature subset to mimic preprocessing, Likelihood-ratio with generative tabular models (Gaussian / Naive Bayes fits), embedding-based MIAs for deep tabular nets (TabNet / TabTransformer), and FL client-update MIAs on structured features, etc. More specifically, membership Inference Attacks (MIA) on natural language data can include, but are not limited, for example, token- and sequence-level confidence / perplexity tests, shadow LMs + perplexity gap, embedding similarity of candidate text vs. internal representations, label-only MIAs via data augmentation (paraphrases, synonyms), prompt probes to elicit memorized spans (rare strings, canaries), and federated / continual LM updates leakage, etc. More specifically, membership Inference Attacks (MIA) on computer vision data can include, but are not limited, for example, Softmax confidence / margin thresholding on images, MC-Dropout / ensembles for uncertainty-based MIAs, embedding similarity (penultimate feature vectors, CLIP image tower), adversarial perturbation response (attack transferability as signal), and white-box gradient MIAs (training dynamics). More specifically, membership Inference Attacks (MIA) on generative AI computer vision data can include, but are not limited, for example, GAN discriminator-based MIAsAttorney Docket No.690248.0002 / 1WO Page 35 of 189 (training vs. non-training image detection), latent inversion & nearest-neighbor in latent space (StyleGAN / PTI), perceptual hash / LPIPS similarity of generations to candidate images, and Diffusion regurgitation tests (prompt replays yielding near-duplicates). More specifically, membership Inference Attacks (MIA) on generative AI large language model data can include, but are not limited, for example, Perplexity-based membership (lower perplexity on training strings), Canary exposure / memorization audits (exposure metric), Prompt-and-leak probes for verbatim spans (Carlini-style extraction), sampling sweeps (temperature / top-p) to surface memorized text, gradient / weight-diff MIAs in FL or PEFT settings (LoRA deltas).

[0045] Property Inference Attacks (PIA) can include, for example, but are not limited to, attack surface dimensions / targets, global properties (entire training set) and local / subgroup properties, black-box probing and white-box gradient / weight analysis, static trained model and training-time (online / FL) leakage, canonical families / named methods, gradient-based attribute inference (analyzing weight / gradient correlations), meta-classifier over model outputs to detect hidden attributes, influence-function guided probes (estimate impact of hidden attributes), adversarial property inference via auxiliary discriminators, recommender-specific property inference (preference / trait leakage), and graph property inference (when inputs are graphs embedded in tabular pipelines), etc. More specifically, property Inference Attacks (PIA) on tabular data can include, for example, but are not limited to, sensitive attribute presence / ratio inference (e.g., gender / race) via output statistics, per-class property inference using calibrated probing sets, white-box gradient sign / scale correlations with protected columns, and FL round update patterns revealing subgroup prevalence, etc. More specifically,Attorney Docket No.690248.0002 / 1WO Page 36 of 189 property Inference Attacks (PIA) on natural language data can include, for example, but are not limited to, authorship / demographic attribute inference from embeddings, writing style, dialect, or location inference via prompt probes, corpus composition / bias inference (topic, source domains), and toxicity / sentiment priors suggesting underlying label distributions, etc. More specifically, property Inference Attacks (PIA) on computer vision data can include, for example, but are not limited to, sensitive attribute leakage from feature maps (e.g., identity, race), inferring presence of objects / scenes not used for labels, and camera / source domain inference (dataset provenance), etc. More specifically, property Inference Attacks (PIA) on generative AI computer vision data can include, for example, but are not limited to, latent-space attribute direction discovery revealing private traits, generator bias auditing to infer hidden dataset composition, and discriminator signal analysis for source domain prevalence, etc. More specifically, property Inference Attacks (PIA) on generative AI large language model data can include, for example, but are not limited to, prompted surveys to infer training-set demographics / politics, citation / source style inference (news vs. web forums vs. code corpora), adapter / LoRA head analysis disclosing domain specialization, etc.

[0046] Reconstruction model inversion attacks can include, for example, but are not limited to, canonical families / named methods, optimization-based feature inversion (match logits / features via gradient descent), gradient leakage (e.g.,DLG, iDLG, GradInversion, analytical / init-free variants), improved priors / regularizers (e.g., TV, BN-stat, range / TV+patch priors), generative prior–guided inversion (use pretrained GAN / decoder as prior), Model stealing → offline inversion (clone then invert replica),Attorney Docket No.690248.0002 / 1WO Page 37 of 189 collaborative learning GAN attacks, LLM data extraction (canary / per-example memorization), embedding inversion (recover text / images from embeddings / CLIP space), and diffusion trajectory backsolving / DDIM inversion / Null-Text inversion / PTI, etc. More specifically, reconstruction model inversion attacks on tabular data can include, for example, but are not limited to, gradient inversion to recover rows / features (single or batch), reconstruction via matching model outputs / gradients under constraints, and autoencoder / GAN priors for plausible row synthesis matching stats, etc. More specifically, reconstruction model inversion attacks on natural language data can include, for example, but are not limited to, next-token distribution inversion to regenerate training fragments, gradient-based text recovery in FL or PEFT (adapter) settings, embedding inversion from sentence / word embeddings (approximate text), and prompt leakage (e.g., jailbreaks, chain-of-thought and red-teaming prompts yielding training text), etc. More specifically, reconstruction model inversion attacks on computer vision data can include, for example, but are not limited to, DLG / iDLG / GradInversion image recovery, feature inversion (match layer activations, BN stats), GAN-prior inversion to photorealistic reconstructions, and Adversarial-example assisted inversion (boundary-aligned inputs), etc. More specifically, reconstruction model inversion attacks on generative AI computer vision data can include, for example, but are not limited to, StyleGAN / StyleGAN2 latent inversion (e4e, pSp, PTI) to retrieve training images, diffusion inversion (DDIM / ODE backsolvers) and regurgitation detection, and nearest-neighbor in VAE / VQ-VAE latent space to approximate originals, etc. More specifically, reconstruction model inversion attacks on generative AI large language model data can include, for example, but areAttorney Docket No.690248.0002 / 1WO Page 38 of 189 not limited to, targeted extraction of rare sequences, secrets, PII via greedy / beam sampling, temperature / top-p sweeps and nucleus sampling to elicit memorized spans, activation maximization / logit-lens style reconstruction for phrases, weight-diff / gradient-leak reconstructions in FL / continual training, cross-cutting scenarios (apply to multiple families), training dynamics attacks (early-stopping snapshots, checkpoint diffs), batch-level and instance-level attacks; multi-member inference, ensemble / Distillation leakage (student stealing teacher’s private info), transfer learning / fine-tuning leakage from pretrained checkpoints, and data poisoning for traceability (watermark / canary) followed by recovery to test leakage, etc.

[0047] Once the processing server 102 has executed the one or more adversarial attacks on the victim models, the processing server 102 determines one or more adversarial attack metrics for each of the victim models. The one or more adversarial attack metrics can include, for example, but are not limited to accuracy, precision, and F1 score, etc. The processing server 102 updates the attack dataset (e.g., the attack data 209) for each victim model by adding the one or more adversarial attack metrics to the attack datasets (e.g., the attack data 209), which now includes the dataset features, “victim model” performance metrics, and the attack success metrics. At this point the processing server 102 has generated one or more attack datasets (e.g., the attack data 209) that includes the dataset features, victim model performance metrics, and attack success metrics for each of the individual model inversion attack vectors and data types (including sub-categories of data types) – there is an attack dataset for tabular membership inversion, one for natural language property inference, etc.Attorney Docket No.690248.0002 / 1WO Page 39 of 189

[0048] In addition to the model attack algorithms 216 discussed above, the processing server 102 can attack datasets bound for generative AI solutions. In such embodiments, the model attack algorithms 216 utilized by the processing server 102 can include, for example, but are not limited to, prompt injection for memorized content, canary string exposure, perplexity-gap detection, systematic prompt sweeps, semantic memorization testing, fine-tuning memorization amplification, prompt replay leakage, latent space nearest neighbor search, perceptual similarity audits, diffusion regurgitation testing, fine-tune domain leakage, transcript-matching leakage, spectrogram similarity, voiceprint memorization detection, music generation replay, scene replay attacks, frame-level matching, latent trajectory backsolving, cross-modal replay, CLIP-space leakage tests, fine-tune bias amplification, training data hash matching, overfitting signal analysis, sampling diversity stress tests, and progressive prompt refinement. Once these leakage-focused attacks are executed by the processing server 102 across the modeling and preprocessing variations, attack success metrics are collected by the processing server 102—including exposure rate, match percentage, perceptual similarity scores, and memorization confidence. These are added to the “attack dataset” (e.g., the attack data 209), which now contains: dataset features (from featurization), victim model performance metrics (from modeling), and leakage attack metrics (from this step). This enables downstream analysis to quantify and compare vulnerability to data leakage across datasets, preprocessing methods, and model architectures—regardless of the specific generative AI technology or attack vector.Attorney Docket No.690248.0002 / 1WO Page 40 of 189

[0049] In addition to the model attack algorithms 216 discussed above, the processing server 102 can attack reinforcement learning datasets. In such embodiments, the model attack algorithms 216 utilized by the processing server 102 can include, for example, but are not limited to, attack surface dimensions / signals, shadow-policy attacks, thresholding-based MIAs, Monte Carlo Dropout / ensemble uncertainty, state- action similarity probes, reward-based MIAs, and federated RL MIAs for membership inference attacks. In such embodiments, the model attack algorithms 216 utilized by the processing server 102 can include, for example, but are not limited to, attack surface dimensions / targets, output probing with meta-classifiers, gradient and weight correlation analysis, influence function-based analysis, opponent / co-agent behavior inference, reward function structure inference, and federated RL property leakage for property inference attacks. In such embodiments, the model attack algorithms 216 utilized by the processing server 102 can include, for example, but are not limited to, trajectory reconstruction via gradient inversion, embedding inversion, optimization- based policy inversion, replay buffer leakage attacks, generative prior-guided reconstruction, environment map recovery, multi-agent behavioral reconstruction, checkpoint comparison attacks, transfer learning leakage, behavior cloning and inversion, and data poisoning traceability for reconstruction attacks.

[0050] In addition to the model attack algorithms 216 discussed above, the processing server 102 can attack datasets to identify a data positioning threshold (e.g., the minimum fraction or volume of poisoned data required to cause harmful model degradation or malicious behavior), a susceptibility profile (e.g., which model types, preprocessing methods, and data modalities are most affected), and attackAttorney Docket No.690248.0002 / 1WO Page 41 of 189 transferability (e.g., whether a poisoned dataset can impact multiple architectures, even if it was optimized for only one). In such embodiments, the model attack algorithms 216 utilized by the processing server 102 can include, for example, but are not limited to, targeted poisoning attacks, indiscriminate poisoning attacks, availability and data drift attacks, and clean label poisoning attacks. For example, the model attack algorithms 216 can attack tabular data by injecting subtle outlier patterns correlated with a target label, crafting plausible but false feature combinations to shift decision boundaries, and backdoor features hidden in rare-value combinations. The model attack algorithms 216 can attack natural language processing data by, for example, inserting rare token sequences or phrases as backdoor triggers, poisoning embeddings via synonym substitutions or back-translation artifacts, and semantic drift by consistently biasing sentiment or topic labels in subtle ways. The model attack algorithms 216 can attack computer vision data by, for example, adding imperceptible pixel-level perturbations (low-amplitude noise) as triggers, visible but inconspicuous triggers (small sticker, patterned corner), poisoning via natural variations (object color, lighting) tied to specific labels. The model attack algorithms 216 can attack generative AI computer vision data by, for example, utilizing backdoor triggers embedded in captions and paired images for image–text models, utilizing image clusters poisoned to bias latent space toward specific aesthetics or features, and by trigger injection into web-scale crawls to propagate bias into generated outputs. The model attack algorithms 216 can attack generative AI large language model data by, for example, inserting trigger phrases in training text that cause specific completions or biases, poisoning with code or text patterns that, when prompted, yield harmful outputs, andAttorney Docket No.690248.0002 / 1WO Page 42 of 189 instruction-poisoning in fine-tuning data to override alignment behavior. For each data poisoning method above, the processing server 102 can vary the poisoning rate (percentage of poisoned samples in training data), the trigger visibility (overt, subtle, imperceptible), the trigger generality (single instance vs. transferable trigger), the attack optimization goal (maximize targeted misclassification, maximize general degradation, maximize bias shift), and the temporal injection pattern (single-batch vs. gradual drift poisoning). After retraining the victim models with poisoned datasets, the processing server 102 can collect one or more of metrics such as, but not limited to: overall accuracy / macro-F1 drop (general performance degradation), targeted success rate (percentage of trigger inputs yielding intended malicious output). clean- data performance preservation (how well the model still performs on unaffected inputs), attack stealthiness metrics (trigger detectability, feature space anomaly scores), cross-model transfer rate (effectiveness of an attack crafted for one model against others), and longitudinal resilience (persistence of poisoning effects across retraining or fine-tuning). The results of each poisoning run are appended to the attack dataset (e.g., the attack data 209), which now contains: dataset features (from featurization), victim model clean performance metrics, poisoning attack parameters (method, rate, visibility, goal), and attack success metrics (general degradation, targeted success, transferability). The processing server 102 utilizes this expanded attack dataset (e.g., the attack data 209) to: calculate the poisoning vulnerability score, estimate the poisoning threshold for each dataset–model–preprocessing combination, and guide hardening recommendations for dataset preprocessing and update policies. Vulnerability MetricAttorney Docket No.690248.0002 / 1WO Page 43 of 189

[0051] Once the processing server 102 has updated the attack datasets (e.g., the attack data 209), the processing server 102 generates a vulnerability metric (e.g., the vulnerability metric 218) for each of the one or more updated attack datasets. Each vulnerability metric (e.g., the vulnerability metric 218) is a library of algorithms and models specific to a combination of one of the one or more adversarial attacks performed on one of the one or more data types included in the dataset.

[0052] Once the vulnerability metrics (e.g., the vulnerability metrics 218) have been generated, a user of the user computing device 104A can transmit a second dataset to the processing server 102 to be analyzed for vulnerability to one or more adversarial attacks using the applicable one or more vulnerability metrics (e.g., the vulnerability metrics 218) for the second dataset. In the system 100, the processing server 102 receives the second dataset from a user computing device 104A and preprocesses and featurizes the second dataset as described above. In embodiments, the user of the user computing device 104A (e.g., via a user interface), can select which data preprocessing algorithms 210 and / or data featurization algorithms 212 are applied to the second dataset by the processing server 102.

[0053] Once the second dataset has been preprocessed and featurized, the processing server 102 identifies the applicable one or more vulnerability metrics (e.g., the vulnerability metrics 218) for the second dataset. For example, if the second dataset only includes tabular data, the processing server 102 can identify the vulnerability metrics 218 that apply to tabular data (e.g., membership inference on tabular data, property inference on tabular data, and reconstruction on tabular data). The processing server 102 then generates a dataset vulnerability report based on theAttorney Docket No.690248.0002 / 1WO Page 44 of 189 output of the one or more vulnerability metrics (e.g., the vulnerability metrics 218) for the second dataset. The dataset vulnerability report can include a section that quantifies the risk that the dataset is vulnerable to a model inversion attack. For example, the dataset vulnerability report can quantify the risk on a scale of zero to six, with six indicating the dataset is very vulnerable to a model invasion attack and zero indicating the dataset is completely safe from a model inversion attack. The dataset vulnerability report can also include a visual depiction of the risk that the dataset is vulnerable to a model inversion attack such as, but not limited to, a bell curve graph, a bar graph, a pie chart, or any other suitable visual representation. The dataset vulnerability report can also include a textual explanation of the quantitative value of risk for the second dataset including a nature of a vulnerability and a reason for the vulnerability. In embodiments, the processing server 102 can generate the textual explanation using various explainable AI methods. Further, the data vulnerability report can include a metric quantifying the trade-off between possible hardening interventions (discussed in more detail below) and eventual model performance. An example user interface 110 illustrating a dataset vulnerability report is illustrated in FIGS.1B-1D, discussed in more detail below.

[0054] In embodiments in which the processing server 102 models and attacks AI datasets to analyze generative AI data leakage, the vulnerability metrics 218 generated by the processing server 102 can include the processing server 102 deriving additional features to enhance the modeling process for the vulnerability metric, such as, but not limited to: ratios of leakage success between base-model and fine-tuned configurations, sensitivity of leakage rate to decoding strategy (temperature,Attorney Docket No.690248.0002 / 1WO Page 45 of 189 top-p, beam width), correlation between dataset redundancy measures and attack success, and variance of leakage rate across similar prompts or latent seeds. The processing server 102 then maps the AI dataset features (as determined above) to attack success metrics to build the vulnerability metric, while simultaneously mapping dataset features to victim model performance metrics to understand the potential trade-off between hardening and task performance. This mapping is model-agnostic, in the sense that the vulnerability metric is trained and validated over the full library of generative AI architectures and fine-tuning strategies used in earlier steps. However, as with preprocessing and modeling, the processing server 102 can optionally tailor the evaluation if the end-user specifies: specific model families (e.g., decoder-only LLM, diffusion-based image generator), intended fine-tuning approaches (e.g., LoRA, full fine-tune, instruction tuning), and expected preprocessing steps. By constraining the evaluation to the intended configurations, the vulnerability metric 218 for AI datasets can produce more accurate leakage risk estimates and more targeted hardening recommendations. The vulnerability metric 218 for AI datasets (e.g., datasets bound for usage in generative AI) can be implemented as a library of predictive models and algorithms, each specialized for a modality and attack vector pairing. For example, the vulnerability metrics 218 for AI datasets can include: text– prompt memorization exposure model (predicts likelihood of canary string leakage), image–diffusion regurgitation risk model (estimates near-duplicate generation probability), audio–voiceprint memorization model (predicts replication risk for training- set speakers), video–scene replay model (estimates likelihood of near-identical shot composition regeneration), and multimodal–cross-modal leakage model (predicts riskAttorney Docket No.690248.0002 / 1WO Page 46 of 189 of paired modality reproduction). From the end-user perspective (e.g., the user of the user computing device 104A), the user can upload a dataset (e.g., the data 208) and the processing server 102 applies automated preprocessing and featurization as described in more detail above. The featurized dataset is evaluated against the pretrained vulnerability metric models by the processing server 102 and the processing server generate a data vulnerability report that can include, a quantitative vulnerability score for the dataset, expressed per modality and attack vector. explanatory insights (via XAI techniques such as SHAP, feature importance rankings, or partial dependence plots) that show why the dataset is vulnerable.

[0055] In embodiments in which the processing server 102 models and attacks RL datasets to analyze RL data leakage, the vulnerability metrics 218 generated by the processing server 102 can include the processing server 102 deriving additional features to enhance labeling and training of the vulnerability metric 218, such as, but not limited to: policy leakage sensitivity (e.g., correlation between policy output entropy and attack success), state–action rarity vulnerability (e.g., attack performance weighted by the novelty of the state–action pairs in the training set), environment diversity leakage index (e.g., attack effectiveness stratified by environment distribution shift.) and gradient–return leakage correlation (e.g., relationship between per-step gradient norms and reconstruction accuracy). The attack dataset (e.g, the attack data 209) is then used in two primary mappings: (a) Dataset features → Attack success metrics: for building the vulnerability metric that predicts susceptibility of a given RL dataset and policy to each type of leakage, and (b) Dataset features → Victim model performance metrics: for quantifying the trade-off between hardening interventions andAttorney Docket No.690248.0002 / 1WO Page 47 of 189 eventual RL performance. This mapping remains model agnostic, meaning it is trained across a diverse set of RL algorithms and architectures (e.g., DQN, PPO, A3C, SAC, multi-agent RL). However, if the end-user specifies a narrower set of preprocessing steps, environment characteristics, or modeling approaches they plan to deploy, the vulnerability metric will adaptively tailor its predictions to those conditions.

[0056] The vulnerability metric module is built as a library of algorithms and models, each specialized to a specific attack type × RL data type pairing (e.g., trajectory-level MIA for image-based policies, environment map reconstruction from tabular state vectors, opponent strategy PIA in multi-agent setups). When the end-user uploads an RL dataset (raw trajectories, replay buffer dumps, or environment logging), the system automatically applies the previously defined preprocessing and featurization procedures. The resulting feature representation is evaluated by the relevant vulnerability metric models. The processing server 102 can apply one or more explainable AI methods (e.g., SHAP, integrated gradients, counterfactual feature analysis) to the vulnerability metric’s 218 outputs. This provides the user not only with a quantitative score of RL data vulnerability but also a clear interpretation of the primary risk drivers, whether they stem from limited state diversity, over-represented environment configurations, predictable policy outputs, or high gradient sensitivity.

[0057] In embodiments in which the processing server 102 models and attacks datasets to analyze data poisoning, the vulnerability metrics 218 generated by the processing server 102 can include the processing server 102 deriving additional features such as, for example, relative performance drop under attack vs. clean baseline, targeted attack amplification factor (increase in targeted misclassification rateAttorney Docket No.690248.0002 / 1WO Page 48 of 189 relative to random error), resilience gradient (rate of performance decline per % of poisoned samples), cross-model susceptibility score (average poisoning success across all tested model architectures), and domain sensitivity score (relative vulnerability in tabular vs. NLP vs. CV vs. GenAI variants of the dataset). These derived features allow the processing server 102 to label historical attack experiments with a quantitative poisoning vulnerability score, forming the foundation for supervised or hybrid modeling of vulnerability risk. The processing server 102 then uses the poisoning attack dataset in two parallel mapping efforts: 1) mapping dataset features to poisoning attack success metrics (e.g., model the direct relationship between dataset structure, distributional properties, and observed poisoning success rates and output a poisoning vulnerability metric 218 that predicts susceptibility before any actual attack is carried out on the user’s dataset); and 2) mapping dataset features to clean victim model performance metrics (e.g., to enable understanding of how hardening against poisoning might impact clean-data performance, and ensure recommendations are not only secure, but also optimized for accuracy and utility in real-world use). While the analysis is fundamentally model-agnostic—evaluated over an entire library of preprocessing and modeling approaches (e.g., the preprocessing algorithms 210 and the data modeling algorithms 214)—the end-user (e.g., of the user computing devices 104A-104N) can optionally specify intended model families or preprocessing pipelines. When provided, these specifications are incorporated to tailor the vulnerability estimation and hardening strategies toward the most relevant configurations. The poisoning vulnerability metric 218 can be a library of tailored machine learning models and algorithms, each specialized for attack type (targeted, indiscriminate, clean-label,Attorney Docket No.690248.0002 / 1WO Page 49 of 189 availability / drift), data domain (tabular, NLP, CV, GenAI), and threat model (black-box, gray-box, white-box). Once trained, this poisoning vulnerability metric 218 metric can be applied by the processing server 102 directly to any new dataset uploaded by the user (e.g., of the user computing devices 104A-104N). The system 100 automates preprocessing, featurization, and vulnerability scoring without requiring manual intervention. In such embodiments, the data vulnerability report generated by the processing server 102 can include a quantitative poisoning vulnerability score, root cause insights (e.g., identifying dataset features and patterns most responsible for vulnerability), threshold estimation (e.g., predicting the approximate poisoning rate at which model reliability becomes compromised), and tailored hardening recommendations (e.g., suggesting preprocessing, filtering, augmentation, or active learning strategies to reduce susceptibility). The result is an actionable, transparent evaluation that allows dataset owners to quantify, understand, and mitigate their risk of data poisoning before deployment or retraining cycles. Data Hardening

[0058] Once the processing server 102 has generated the vulnerability metrics 218 and a dataset has been evaluated using one or more of the vulnerability metrics 218, the processing server can execute one or more data hardening algorithms 219 to enhance the security of the dataset (e.g., make the dataset less susceptible to the model attack algorithms 216). The data hardening algorithms 219, like the vulnerability metrics 218, are specific to a combination of a data type included in the dataset and a model attack algorithm to which the processing server is hardening the dataset against.Attorney Docket No.690248.0002 / 1WO Page 50 of 189

[0059] For example, the data hardening algorithms 219 against membership inference attacks on tabular data can include, but are not limited to, de-duplication of near / exact duplicate rows (e.g., MinHash / SimHash, LSH), class rebalancing / stratified sampling to reduce minority overfitting, K-anonymity / l-diversity / t-closeness preprocessing on quasi-identifiers, generalization hierarchies (e.g., coarsen ZIP→county, DOB→year, etc.), quantization / binning of continuous features; top- / bottom-coding outliers, feature clipping to plausible ranges, winsorization of tails, noise addition to sensitive features (e.g., Laplace / Gaussian; per-column calibrated), microaggregation / group-based averaging within k-anonymous clusters, category collapsing for high-cardinality features, feature hashing, synthetic data supplementation (tabular GANs / VAEs) to widen support, adversarial filtering to remove trivially easy / outlying records, train / validation / test decontamination and near-duplicate removal, pseudonymization (e.g., replace identifiers with random values), suppression or aggregation of rare categories, bucketization or slicing to reduce linkage risk, perturbation or swapping of feature values, correlation-based feature removal (>80% correlated features), manifold-theory-based dimensionality reduction (e.g., Isomap), class region density hardening via CTGAN oversampling and NearMiss undersampling, and entropy- focused adjustments to increase unpredictability of features, etc. The data hardening algorithms 219 against membership inference attacks on natural language processing data can include, for example, but are not limited to, document-, paragraph-, and span-level de-duplication (e.g., shingling + MinHash), paraphrasing / back-translation to reduce verbatim strings, rare n-gram capping or substitution with semantically equivalent phrases, canonicalization (e.g., Unicode NFKC, whitespace, punctuation,Attorney Docket No.690248.0002 / 1WO Page 51 of 189 casing) to remove unique signatures, entity masking / pseudonymization (e.g., for names, IDs, emails, URLs, codes), template abstraction for forms / logs (replace literals with slots), source diversity balancing (e.g., domains, time, regions), boilerplate stripping (e.g., HTML / JS / CSS, headers, footers), personal identifying information (PII) scrubbing, and secret pattern removal (e.g., keys, passwords) prior to training, etc. The data hardening algorithms 219 against membership inference attacks on computer vision data can include, for example, but are not limited to, aggressive augmentation (e.g., RandAugment / AugMix, flips, crops, rotations), MixUp / CutMix / CutOut to smooth decision boundaries, style randomization / domain randomization (e.g., color jitter, histogram, style transfer), dataset de-dup via pHash / aHash / dHash and CLIP-embedding NN search, crop to remove watermarks / text / serial numbers and unique borders, slight downscaling / resampling; JPEG recompression to reduce exact pixel matches, small Gaussian / Poisson noise injection within imperceptible bounds, balance per-class / per-identity counts to reduce overspecialization, and EXIF / camera metadata stripping to avoid stable signatures, etc. The data hardening algorithms 219 against membership inference attacks on generative AI computer vision data can include, for example, but are not limited to, strict de-duplication across web-scale crawls and internal sets, per-identity / image-cluster capping to avoid repeated near-duplicates, caption normalization and de-noising (e.g., tokenization, lowercasing, stopword policies), low-amplitude noise or small augmentations on training images to hinder exact memorization, aesthetic / NSFW filtering to remove iconic or uniquely identifying images, and diversity-driven sampling of training batches (e.g., reduce repeatedAttorney Docket No.690248.0002 / 1WO Page 52 of 189 prompts / scenes), etc. The data hardening algorithms 219 against membership inference attacks on generative AI large language model data can include, for example, but are not limited to, multi-granularity de-duplication (e.g., document, paragraph, 5- to 13-gram spans), removal of near-duplicate mirrors (e.g., wikis, code gists, mirrors, scrapes), PII / secret scrubbing and redaction (e.g., pattern-based and model- assisted), paraphrase rare strings and long exact quotes; replace with summaries, canonicalize markup / code formatting (e.g., black / clang-format equivalents), source balancing (e.g., news / forums / books / code) and time balancing to avoid domain bias spikes, and chunking with overlap control to avoid long verbatim fragments, etc.

[0060] The data hardening algorithms 219 against property membership inference attacks on tabular data can include, for example, but are not limited to, removing protected attributes and strong proxies (e.g., correlation / MI screening), attribute generalization / aggregation (e.g., coarser bins / categories), reweighing / resampling to equalize protected group distributions per label, counterfactual augmentation (e.g., swap protected attribute values while preserving labels), noise injection into protected columns, randomized response for sensitive binaries, orthogonalization / residualization, regress out protected attribute from features, fairness-aware feature selection (e.g., drop features with high proxy power), group-aware stratification to balance train / val / test splits, suppression or aggregation of rare categories, and slicing of feature space to reduce correlation exposure, etc. The data hardening algorithms 219 against property membership inference attacks on natural language processing data can include, for example, but are not limited to,Attorney Docket No.690248.0002 / 1WO Page 53 of 189 entity anonymization and consistent pseudonymization (e.g., names, places, orgs), style obfuscation / neutralization (e.g., remove dialect markers, author idiolect), topic and sentiment balancing across demographic subgroups, back-translation / style-transfer to wash demographic cues, and metadata stripping (e.g., author, timestamp, location), etc. The data hardening algorithms 219 against property membership inference attacks on computer vision data can include, for example, but are not limited to, attribute masking (e.g., blur / pixelate faces, license plates; text redaction), tight cropping to remove background / contextual demographic cues, synthetic rebalancing with attribute editing (e.g., age, skin tone, hair, accessories), camera / source balancing and EXIF removal, and illumination / white-balance normalization to reduce domain cues, etc. The data hardening algorithms 219 against property membership inference attacks on generative AI computer vision data can include, for example, but are not limited to, attribute-balanced sampling of images and captions, latent-space editing to synthesize under-represented attributes for balance, caption sanitization: strip sensitive attributes unless necessary, and cluster-level capping so no attribute dominates the corpus, etc. The data hardening algorithms 219 against property membership inference attacks on generative AI large language model data can include, for example, but are not limited to, corpus composition balancing (e.g., by region, politics, genre, language), entity redaction / pseudonymization, consistent placeholders, style normalization (e.g., remove authorial markers; normalize punctuation / casing), topic throttling: cap max contribution of any sensitive topic / source, and temporal balancing (e.g., avoid single-period dominance revealing properties), etc.Attorney Docket No.690248.0002 / 1WO Page 54 of 189

[0061] The data hardening algorithms 219 against reconstruction attacks on tabular data can include, for example, but are not limited to, feature generalization (e.g., coarser bins), quantization, and rounding, top-coding / bottom-coding extreme values, range clipping, k-map risk analysis to identify and suppress unique rows, microaggregation within k-anonymous groups, partial suppression of quasi-IDs, add calibrated feature noise (Laplace / Gaussian) compatible with DP goals, synthetic replacement for the most unique rows (e.g., DP-GAN / DP-VAE where possible), date coarsening (e.g., day→week→month), geo-coarsening (e.g., lat / long→grid), hashing / salting for persistent identifiers, and manifold-theory-based dimensionality reduction to limit reconstructable detail, etc. The data hardening algorithms 219 against reconstruction attacks on natural language processing data can include, for example, but are not limited to, PII / secret scrubbing (e.g., emails, SSNs, keys), dictionary + ML detectors, replace rare tokens / phrases with semantically equivalent but non-unique variants, cap maximum exact string length per document (e.g., chunking with overlap limits), hash or mask identifiers and code literals, remove license headers, and de-contaminate train vs. eval corpora to prevent verbatim leakage, etc. The data hardening algorithms 219 against reconstruction attacks on computer vision data can include, for example, but are not limited to, slight downscaling / controlled JPEG recompression to remove high-freq details, face / license-plate blurring, text / serial masking via inpainting, random crop / resize / aspect jitter so pixels are not stable across copies, low-level noise / dithering; color-space normalization, and remove overlays / watermarks and unique borders before training, etc. The data hardening algorithms 219 againstAttorney Docket No.690248.0002 / 1WO Page 55 of 189 reconstruction attacks on generative AI computer vision data can include, for example, but are not limited to, rigorous exact / near-exact de-dup across training images and public eval sets, per-identity cap; collapse near-duplicates by clustering, embed imperceptible provenance watermarks for post-hoc leakage tracing, caption de-duplication and normalization to avoid rare unique phrases, and filter iconic / celebrity / landmark images likely to regurgitate, etc. The data hardening algorithms 219 against reconstruction attacks on generative AI large language model data can include, for example, but are not limited to, strong multi-level de-dup, shingle-based and embedding-based, redact or replace PII and secret patterns, salt / hash identifiers, avoid long verbatim copyrighted texts; prefer summaries or paraphrases, domain balancing and chunking to reduce over-representation of small corpora, and de-contamination against evaluation / benchmark datasets, etc.

[0062] The processing server 102 can provide (e.g., via the user interface) the end- user (e.g., the user of the user computing device 104A) with a list of potential hardening strategies (e.g., the data hardening algorithms 219) ranked in order of effectiveness as related to the vulnerability metric 218 applied to the dataset submitted by the end-user. The processing server 102 can also provide potential future model output metrics – using the above mentioned algorithms for analyzing victim model performance – based on model agnostic or user-defined tuned outputs (based on preprocessing and anticipated model usage). This allows the user (e.g., the user of the user computing device 104A), as the subject matter expert, to determine which hardening method (e.g., of the data hardening algorithms 219) best suits their usage case. In addition, the processing server 102 can allow the end-user to tune variousAttorney Docket No.690248.0002 / 1WO Page 56 of 189 hardening methods (e.g., of the data hardening algorithms 219) to determine vulnerability and potential end-use tradeoff metrics in order to fine-tune the resulting hardened dataset to their use cases.

[0063] In embodiments in which the processing server 102 models and attacks AI datasets to analyze AI data leakage, the data hardening algorithms 219 utilized by the processing server 102 can include, for example, but are not limited to, Multi- Granularity De-duplication, Sensitive Content Redaction, Rare String Mitigation, Source and Domain Balancing, Formatting Canonicalization, Fine-Tuning Overfit Reduction, Exact & Near-Exact Image De-duplication, Per-Identity / Image Cluster Capping, Augmentation for Diversity, Metadata & Watermark Removal, Controlled Perturbations, Caption & Label Normalization, Audio De-duplication, Speaker Balance and Capping, Controlled Audio Augmentation, Metadata Scrubbing, Transcript Normalization, Frame-Level Duplicate Removal, Scene Capping, Spatial & Temporal Augmentation, Watermark & Overlay Removal, Multimodal Alignment Scrubbing, Cross-Modal Duplicate Control, Balanced Modality Representation, Embedding-Space Diversity Enforcement, and Prompt–Output Pair Normalization, etc. In such embodiments, the processing server 102 can integrate the above data hardening methods into an interactive decision-support tool that: ranks hardening strategies for the given dataset based on their predicted effectiveness in reducing vulnerability scores, simulates impact on downstream model utility by referencing historical correlations between similar datasets, hardening strategies, and victim model performance, supports model-agnostic recommendations, or user-specific tuning if the end-user provides intended preprocessing and modeling configurations, and allowsAttorney Docket No.690248.0002 / 1WO Page 57 of 189 iterative tuning – users can adjust the intensity of de-duplication, augmentation, redaction, or balancing and immediately see projected vulnerability–utility trade-offs. The final output of the processing server 102 using the data hardening algorithms 219 is a hardened dataset that preserves as much model performance as possible while substantially lowering the risk of training data regurgitation, verbatim reproduction, and other forms of memorization-based data leakage in generative AI and fine-tuned generative AI systems.

[0064] In embodiments in which the processing server 102 models and attacks reinforcement learning (RL) datasets to analyze data leakage, the data hardening algorithms 219 utilized by the processing server 102 against membership inference attacks can include, for example, but are not limited to, trajectory de-duplication to remove near-identical episode sequences using state–action hashing or lsh over trajectory embeddings, state-space coverage balancing to ensure no small subset of states dominates the training set, reducing overfit-driven leakage, action distribution smoothing by adding policy noise or augmenting with alternative but valid action sequences, environment randomization (procedural generation, altered textures, randomized spawn points) to limit memorization of fixed layouts, state discretization / quantization to remove fine-grained state information that may uniquely identify training episodes, synthetic trajectory injection using behavior cloning or generative trajectory models to diversify replay buffers, metadata stripping (timestamps, seed ids, simulation logs) from stored trajectories that could act as identifiers, weight perturbation within non-performance-critical bounds to obscure fine- grained memorized patterns, and gradient clipping / noise injection during training toAttorney Docket No.690248.0002 / 1WO Page 58 of 189 reduce retention of highly specific state–action patterns. In such embodiments, the data hardening algorithms 219 utilized by the processing server 102 against property inference attacks can include, for example, but are not limited to removal or generalization of sensitive state features (e.g., player demographics in human-in-the- loop rl, location coordinates in navigation tasks), state feature masking for protected or proxy attributes while preserving reward-relevant features, counterfactual environment augmentation (swap or randomize sensitive environment properties while maintaining solvability), reward function obfuscation by reparameterizing or adding noise to reward components linked to private properties, balanced sampling across environment variants to equalize representation of sensitive properties, opponent strategy anonymization by blending policy outputs from multiple strategies to obscure true source identity, and trajectory re-labeling to hide performance indicators linked to specific subgroups or configurations, etc. In such embodiments, the data hardening algorithms 219 utilized by the processing server 102 against reinforcement learning / model inversion inference attacks can include, for example, but are not limited to state feature coarsening (e.g., discretizing continuous coordinates, aggregating sensor readings into bins), trajectory shortening to prevent full-sequence reconstruction by omitting redundant or identifying steps, controlled observation noise injection to blur fine-grained cues without harming policy learning, domain randomization during collection to make exact scene / level reconstruction infeasible, rare-event suppression by replacing extremely unique episodes with statistically similar synthetic sequences, knowledge distillation into a smaller model or altered architecture to limit extraction of original network internals, and parameter smoothing / regularizationAttorney Docket No.690248.0002 / 1WO Page 59 of 189 to hinder exact inversion of learned policies, etc. The processing server 102 can integrate these hardening algorithms 219 into a decision-support system that ranks candidate mitigations by: expected reduction in vulnerability metric score for the targeted attack type, projected reinforcement learning performance impact, using the victim model performance prediction algorithms, and compatibility with user-defined constraints, such as environment fidelity requirements or training budget limits. In such embodiments, users (e.g., the user of the user computing devices 10-4A-104N) can select model-agnostic hardening recommendations or constrain results to their intended RL algorithm(s) (e.g., PPO, SAC, DQN), tune parameters of hardening methods (e.g., degree of state discretization, noise magnitude, augmentation probability) and see predicted trade-offs between vulnerability reduction and expected episodic return, and apply multi-objective optimization to balance leakage resistance with performance preservation for deployment contexts. Once a user (e.g., a user of the computing device 104A) finalizes a hardening configuration, the processing server 102 can automatically apply preprocessing and augmentation steps to produce the hardened RL dataset (trajectories, replay buffers, environment configs), log all transformations in a hardening manifest for auditability and reproducibility, and optionally retrain reference RL models on the hardened dataset to provide post- hardening performance and vulnerability metrics. The result is a hardened dataset that retains policy effectiveness while measurably reducing susceptibility to RL-specific data leakage attacks, validated against the same attack suite used in vulnerability assessment.Attorney Docket No.690248.0002 / 1WO Page 60 of 189

[0065] In embodiments in which the processing server 102 models and attacks datasets to analyze data poisoning, the data hardening algorithms 219 utilized by the processing server 102 on tabular data can include, for example, but are not limited to, robust data validation pipelines to reject anomalous or out-of-distribution incoming data prior to ingestion, incremental batch testing with canary sets to detect early deviations in feature distributions, statistical anomaly detection methods (e.g., Mahalanobis distance, robust z-scores, or isolation forests) to identify potential poisoning candidates, adversarial example filtering using trained discriminators or contrastive embedding checks, distributional shift detection using metrics (e.g., KL- divergence, Jensen–Shannon distance, or Population Stability Index (PSI)), batch reweighting to minimize influence of suspected poisoned subsets, active learning– driven review loops where uncertain or highly influential samples are human-audited before inclusion, and synthetic replacement or repair of flagged records using generative imputation models. In such embodiments, the data hardening algorithms 219 utilized by the processing server 102 on natural language processing data can include, for example, but are not limited to N-gram frequency analysis to detect injected rare tokens or adversarial trigger phrases, embedding-space clustering to identify semantic outliers or unusual topic intrusions, canonicalization and normalization to eliminate adversarial text variants (e.g., homoglyphs, Unicode tricks), automated removal of suspicious PII, secrets, or malicious patterns from incoming text, and paraphrase normalization of incoming updates to smooth targeted linguistic perturbations. In such embodiments, the data hardening algorithms 219 utilized by the processing server 102 on computer vision data can include, for example, but are notAttorney Docket No.690248.0002 / 1WO Page 61 of 189 limited to perceptual hashing and embedding similarity to detect injected backdoor images or pixel-level perturbations, randomized preprocessing (color jitter, slight cropping, noise injection) to blunt backdoor triggers, removal of images containing imperceptible patterns likely to serve as poisoning keys, and distributional monitoring of feature activations for incoming images to detect unusual patterns. In such embodiments, the data hardening algorithms 219 utilized by the processing server 102 on generative AI computer vision data can include, for example, but are not limited to cross-checking newly ingested images and captions against a clean reference set to detect latent-space poisoning vectors, augmenting poisoned-suspect data with randomized style and geometric transformations to disrupt embedded triggers, and caption canonicalization to remove trigger keywords or structured adversarial instructions. In such embodiments, the data hardening algorithms 219 utilized by the processing server 102 on generative AI large language model data can include, for example, but are not limited to filtering prompt–response pairs for embedded control sequences or model-steering backdoors, embedding similarity search to identify near- duplicate injection patterns across updates, and red-teaming updates with controlled queries to detect behavioral drift consistent with poisoning. In embodiments, processing server 102 can incorporate both a dataset’s resiliency score and its poisoning threshold into the recommendation process. The resiliency score represents the degree to which the dataset naturally resists degradation from poisoning, whereas the poisoning threshold indicates the proportion of poisoned data that can be introduced before performance falls below an acceptable baseline. The processing server 102 can use these two measures in combination to determine the mostAttorney Docket No.690248.0002 / 1WO Page 62 of 189 effective defenses. For example, in certain embodiments, the processing server can calculate a post-hardening safety margin using a relationship such as: Sm=(Tp+ΔTp(He,R))×R′, where: Tp represents the poisoning threshold, R represents the original resiliency score, ΔTp(He,R) represents the expected threshold increase from applying one or more hardening methods based on their historical effectiveness and dataset resiliency, and R′ represents the updated resiliency score after hardening. By using such a formulation, the processing server 102 can recommend hardening methods (e.g., data hardening algorithms 219) that not only address the most critical vulnerabilities but also maximize the dataset’s post-hardening safety margin, while balancing trade-offs in downstream model performance. Further, by providing a decision-support integration of the hardening methodology, the processing server 102 delivers the end-user (e.g., of the user computing devices 104A-104N) a ranked list of potential hardening strategies (e.g., data hardening algorithms 219) by expected effectiveness relative to the vulnerability metric, resiliency, and poisoning threshold. The processing server 102 can also provide projections of post-hardening model performance metrics – either in a fully model-agnostic form or tuned to the user’s anticipated preprocessing and modeling pipeline – enabling the user (e.g., of the user computing devices 104A-104N) to select and customize the hardening approach most aligned to their operational requirements. Data Vulnerability Reports

[0066] In embodiments, the processing server 102 can offer a user interface such as webpage, application program, or other interface for users to interact with the processing server 102 to upload datasets, analyze and / or harden those datasets usingAttorney Docket No.690248.0002 / 1WO Page 63 of 189 the functions of the processing server 102, and view data vulnerability reports as discussed in more detail above. As discussed herein, data exchanged between the processing server 102, user computing devices 104A-104N, and / or the data sources 106A-106N can use such an interface, where communications can be facilitated using any suitable communication network, such as the Internet FIGS 1B-1D illustrate an example user interface 110 in accordance with embodiments. The user interface 110 can provide both templated and / or customizable data vulnerability reports to the user (e.g., the user of the user computing device 104A) explaining the quantified risk that the dataset is vulnerable to a model inversion attack, an explanation of the vulnerabilities and their root causes (e.g., as determined by the vulnerability metrics 218), as well as changes which have been made to the hardened dataset (e.g., via the data hardening algorithms 219), and any resulting performance and vulnerability metrics. In addition, the user interface 110 can provide tailored educational modules at various technical and need-to-know levels to provide a user (e.g., of the user computing device 104A) with the knowledge necessary to collect more robust and secure datasets in the future. Referring to FIG 1B, the user interface includes a “Data Preview” section that can provide a user (e.g., of the user computing device 104A) with the ability to upload or otherwise transmit a dataset to the processing server 102. For example, the user interface 110 can allow a user of the user computing device 104A to drag and drop a dataset and / or provide one or more links to allow the user to navigate a database to select a dataset for analysis. Once the data set has been transmitting to the processing server 102, the “Data Preview” panel of the user interface 110 can generate a data sample of the dataset and display that data sampleAttorney Docket No.690248.0002 / 1WO Page 64 of 189 to the user of the user computing device 104A. Referring to FIG.1B, the “Data Preview” section of the user interface 110 displays six rows of tabular data from a dataset provided by a user (e.g., the user of the user computing device 104A). Referring again to FIG.1B, the user interface 110 can include a “Vulnerability Analysis” section that displays the available vulnerability metrics 218 that can be applied to the dataset provided by the user (e.g., the user of the user computing device 104A). For example, if the dataset provided by the user (e.g., the user of the user computing device 104A) only contains tabular data, the user interface 110 will only display vulnerability metrics 218 applicable to tabular data (e.g., membership inference for tabular data, property inference for tabular data, and reconstruction for tabular data). The user interface 110 can also include a “Hardening Engine” Section that includes tools for users to interact with the data hardening algorithms 219 as discussed in more detail above. In FIG 1B. the “Hardening Engine” is greyed out as no vulnerability metric 218 has been run on the user’s dataset.

[0067] Referring to FIG.1C, at least one vulnerability metric 218 has been run on the user’s dataset by the processing server 102 and the “Vulnerability Analysis” section of the user interface 110 has updated to display a data vulnerability report, e.g., a “vulnerability risk distribution,” an “overall vulnerability score,” and a listing of identified vulnerabilities in the dataset (e.g., the date 208). The user interface 110 can also list the models and algorithms that were applied via the vulnerability metric 218 to the dataset along a success metric for each individual model and algorithm of the vulnerability metric. The vulnerability risk distribution can include a graphical representation of the vulnerability risk for the dataset and a categorical riskAttorney Docket No.690248.0002 / 1WO Page 65 of 189 assessment (e.g., secured or vulnerable, etc.). Further, in FIG.1C, the “Hardening Engine” of the user interface 110 has updated to include a selectable icon to run one or more of the data hardening algorithms 219. Further, the “Hardening Engine” of the user interface 110 can provide a user (e.g., of the user computing device 104A) the option of selecting the data hardening algorithms 219 to apply to the dataset or allow the processing server 102 to automatically determine which hardening algorithms 219 to apply to the dataset. Referring to FIG.1D, one or more of the data hardening algorithms 219 have been run on the dataset and the “Vulnerability Analysis” section of the user interface 110 has been updated to display the vulnerability risk distribution of the hardened dataset along with a categorical risk assessment for the hardened dataset.

[0068] In embodiments where the processing server 102 analyzes datasets (e.g., the data 208) for AI data leakage, the data vulnerability report can include a vulnerability overview, a root cause analysis, and a hardening summary. The vulnerability overview can include an overall leakage risk score broken down by modality (e.g., text, image, audio, video, multimodal) and by specific generative AI leakage vector (e.g., prompt memorization, diffusion regurgitation, latent inversion), and a comparison of base- model vs. fine-tuned model leakage susceptibility. The root cause analysis can include an Identification of dataset features most strongly correlated with high leakage risk (e.g., high duplicate ratio, presence of rare unique identifiers, over-represented scenes or phrases), and a mapping of vulnerabilities to specific model architectures and fine- tuning strategies where risk is amplified. The hardening summary can include description of each applied hardening method (e.g., multi-granularity de-duplication,Attorney Docket No.690248.0002 / 1WO Page 66 of 189 rare string paraphrasing, per-identity capping, metadata scrubbing), a before-and-after metrics for vulnerability reduction, including percentage improvement per attack vector, and a predicted and actual changes in model utility following hardening, allowing trade-off analysis. The processing server 102 can generate the data vulnerability reports in one or more formats, such as, but not limited to, executive summary format (e.g., non-technical, focused on overall risk posture, compliance readiness, and business impact), technical deep-dive format (e.g., detailed methodology, feature importance rankings, per-vector leakage performance, and reproducibility notes for engineering teams), and compliance-ready format (e.g., aligned with AI governance frameworks, emphasizing privacy risk mitigation steps, audit trails, and documentation of applied safeguards.

[0069] The processing server 102 can also generate tailored educational content designed to help users (e.g., the users of the user computing device 104A-104N) reduce future leakage risks during dataset collection, curation, and model development. The processing server 102 can generate educational content in multiple levels of detail. For example, the processing server 102 can generate foundational educational modules (e.g., for non-technical users) that describes data leakage in generative AI, how fine-tuning can increase memorization risk, and the role of data diversity and de-duplication in reducing leakage. As another example, the processing server 102 can generate intermediate educational modules (e.g., for users who are data scientists and ML engineers) that describe dataset characteristics that increase regurgitation likelihood in LLMs and diffusion models, how preprocessing choices affect leakage risk, and balancing data hardening with generative model utility. As yetAttorney Docket No.690248.0002 / 1WO Page 67 of 189 another example, the processing server 102 can generate advanced educational modules (e.g., for users who are AI researchers, architects) that describe in-depth mechanics of leakage attacks in large generative models, fine-tuning–specific overfitting pathways and mitigation strategies, and designing evaluation protocols for leakage detection and prevention. Each educational module is customized based on the user’s dataset and vulnerability profile, integrating examples from their own evaluation results to make the learning concrete and actionable. The combination of reporting and educational support provided by the methods and systems disclosed herein ensures that users not only understand their current AI data leakage risk but also ensures that user develop the practical knowledge to prevent similar vulnerabilities in future generative AI and fine-tuning projects.

[0070] In embodiments where the processing server 102 analyzes reinforcement learning datasets (e.g., the data 208) for data leakage, the data vulnerability report can include, for example, but not limited to, a quantified vulnerability metric summary, a root cause analysis of vulnerabilities, a hardening changes overview, and scenario- based projections. The quantified vulnerability metric summary can include final vulnerability score(s) per attack type (Membership Inference, Property Inference, Reconstruction) applied to RL datasets, policies, and environment artifacts. The quantified vulnerability metric summary can also include comparative analysis across different preprocessing and model configurations tested during evaluation. The root cause analysis of vulnerabilities can include an identification of dataset-level leakage vectors (e.g., rare trajectories, environment seeds, overfitted policy parameters), a mapping of vulnerabilities to specific RL components (e.g., replay buffer, environmentAttorney Docket No.690248.0002 / 1WO Page 68 of 189 configuration, reward function), and an explanation of how adversaries could exploit observed weaknesses in an RL context (e.g., recovering environment maps from trajectories, inferring hidden reward parameters). The hardening changes overview can include a complete list of modifications applied to the dataset and policy parameters during hardening. pre- and post-hardening comparisons of: vulnerability metrics, RL performance metrics (average episodic return, win rates, sample efficiency), and policy behavior changes (e.g., altered action distributions). The scenario-based projections can include a simulation of expected vulnerability and performance impact under different attacker capability models (black-box, white-box, federated RL, transfer learning), and a “what-if” projections for alternative hardening strategies not chosen by the user.

[0071] The processing server 102 can also generate tiered, role-specific educational modules to empower users (e.g., of the user computing devices 104A-104N0 with the knowledge to prevent vulnerabilities before they occur. For example, the processing server 102 can generate practitioner-level educational modules (e.g., for technical staff, RL engineers) that include best practices for collecting RL training data securely (e.g., seed randomization, state generalization, environment diversity), how overfitting and replay buffer composition affect data leakage risk, defense-aware training techniques (e.g., DP-SGD for RL, policy regularization, reward shaping for obfuscation), and secure deployment guidelines for RL models in production (e.g., API rate limiting, model watermarking). As another example, the processing server 102 can generate analyst / manager-level educational modules that include understanding vulnerability metrics and their operational implications, balancing security vs.Attorney Docket No.690248.0002 / 1WO Page 69 of 189 performance trade-offs in RL systems, and decision frameworks for selecting appropriate hardening strategies. As another example, the processing server 102 can generate executive and stakeholder educational modules that include strategic risk assessment for RL-based products, regulatory and compliance considerations for RL data privacy, cost–benefit narratives of security investments in RL pipelines. In such embodiments, the data vulnerability report user interface (e.g., the user interface 110) can include one or more interactive features such as, but not limited to, dynamic visualizations of attack simulations on RL policies, showing how leakage is exploited, before-and-after policy performance heatmaps to visualize the effect of hardening on state–action spaces, and step-by-step guided labs where users can run miniature versions of attacks and defenses on sandbox RL environments. By coupling clear, evidence-based reporting with targeted educational content, the system 100 ensures that users leave not only with a hardened RL dataset and model, but also with the knowledge, skills, and organizational readiness to prevent future vulnerabilities— creating a feedback loop of continuous improvement in RL security practices.

[0072] In embodiments where the processing server 102 analyzes datasets (e.g., the data 208) for data poisoning, the data vulnerability report can include, for example, but not limited to, a quantified data poisoning vulnerability metric, a calculated poisoning threshold (Tp), and a resiliency score (R) before and after hardening. These data vulnerability reports can also include an explanation of the specific vulnerabilities identified in the dataset, their likely root causes (e.g., such as class imbalance, over- reliance on a narrow set of features, low diversity of sources, or high sensitivity to batch updates), and the potential impact of different types of poisoning attacks (e.g.,Attorney Docket No.690248.0002 / 1WO Page 70 of 189 label-flipping, backdoor triggers, or targeted feature perturbations). In addition, the data vulnerability reports document all changes applied during the hardening process (e.g., by the data hardening algorithms 219), including the hardening methods (e.g., the data hardening algorithms 219) selected, their expected and observed effectiveness (ΔTp(He,R), and the resulting updated resiliency score (R′). Both quantitative (e.g., updated metrics, attack success reduction, threshold shift) and qualitative (e.g., nature of removed vulnerabilities, tradeoffs in accuracy) results can be provided by the processing server in the data vulnerability reports. The reporting interface, e.g., the user interface 110, can support both model-agnostic views and user-specified modeling / preprocessing configurations, ensuring that the information remains relevant to the specific deployment scenario. Graphical summaries—such as resiliency-over-time plots, poisoning threshold sensitivity curves, and hardening method effectiveness comparisons—are included to aid rapid comprehension.

[0073] In such embodiments, the processing server 102 can also generate tailored educational modules that address data poisoning resilience at various technical and operational levels. For example, the processing server 102 can generate executive- level educational overviews that focus on the business and operational risks of poisoning, thresholds of concern, and decision frameworks for remediation. As another example, the processing server 102 can generate data science and engineering educational modules that include the technical underpinnings of poisoning attacks, feature- and class-level susceptibility analysis, and best practices for resilient data collection, cleaning, and augmentation. In yet another example, the processing server 102 can generate security-focused educational modules that explore adversaryAttorney Docket No.690248.0002 / 1WO Page 71 of 189 models, detection and monitoring strategies, and hardening techniques specific to poisoning-resistant dataset design. These educational modules are dynamically adapted by the processing server 102 based on the data type (tabular, NLP, CV, or generative AI) included in the dataset (e.g., the data 208) and the attack vectors most relevant to the evaluated dataset (e.g., the data 208). This ensures the end-user (e.g., of the user computing devices 104A-104N) gains the knowledge and actionable guidance needed to collect, curate, and maintain datasets that remain robust against poisoning threats—even as the data and model evolve over time.

[0074] In embodiments where the processing server 102 analyzes datasets (e.g., the data 208) for data poisoning, the processing server 102 can proceed to identify specific data points with the dataset (e.g., the data 208) that are most likely to represent potential poisoning. The processing server 102 can flag potentially poisoned data samples within the dataset (e.g., the data 208) through a combination of statistical, representational, and model-based analyses. The flagging of potentially poisoned data samples within the dataset (e.g., the data 208) by the processing server 102 can be modular and thus adjusted to the computational budget and needs of the end-user (e.g., of the user computing devices 104A-104N). The processing server 102 can use one or more data flagging methods such as, but not limited to, statistical outlier detection, representation-based anomaly detection, model-influence analysis, cross-preprocessing consistency, and cross-model consistency. Statistical outlier detection can include, for example, identification of observations with anomalous feature distributions, unusual class–label pairings, or fabricated correlations not present in the broader dataset. For tabular data, this includes detection of “unnatural”Attorney Docket No.690248.0002 / 1WO Page 72 of 189 value ranges or rare categorical–label combinations. For text, this includes, for example, detection of rare n-grams, improbable entity co-occurrences, or unusual semantic shifts. For images, this includes artifacts such as adversarial noise patterns, pixel-level perturbations, or watermark-like triggers. Representation-based anomaly detection can include embedding-space methods are applied to identify data points that are disproportionately distant from the majority of samples within the same class, and k-Nearest Neighbor (kNN), clustering, and density-estimation methods provide quantitative anomaly scores for each observation. Model-influence analysis can include, for example, influence functions, loss sensitivity analysis, and gradient similarity methods to quantify how strongly each data point affects model parameters. Further, model-influence analysis can include, for example, observations with disproportionate impact on predictions or gradient updates are flagged as higher risk. Cross-preprocessing consistency can include, for example, evaluating whether a sample produces inconsistent representations or predictions across multiple preprocessing pipelines since preprocessing variations often stabilize robust samples, poisoned points can be detected by evaluating whether a sample produces inconsistent representations or predictions across multiple preprocessing pipelines. Cross-model consistency can include, for example, evaluating samples across multiple modeling methods as poisoned samples often lead to prediction divergence where clean samples remain stable across model families. The processing server 102 assigns each data point within the dataset (e.g., the data 208) a poisoning risk score, which provides a unified quantitative measure of suspicion based on multiple detection methods. The risk score can be defined ask: Ri = wa * Ai + wf * Fi + wc * Ci, where: Ri:Attorney Docket No.690248.0002 / 1WO Page 73 of 189 poisoning risk score for data point I, Ai: anomaly score of sample i (statistical or embedding-based outlierness), Fi: influence score of sample i (impact on model gradients, loss, or parameter updates), Ci: consistency score of sample i (stability of predictions across preprocessing pipelines and modeling methods), and wa, wf, wc: tunable weights specified by the end-user or learned from historical poisoning detection datasets. The function may also be extended to incorporate higher-order or non-linear combinations, such as neural-network based meta-models trained on historical poisoning scenarios, but the linear form above provides a transparent and explainable baseline. Flagged data samples within the dataset (e.g., the data 208) are returned to the end-user (e.g., of the user computing devices 104A-104N) in the data vulnerability reports (e.g., via the user interface 110). In such embodiments, the data vulnerability reports can include lists of flagged observations with associated metadata explaining the basis for their identification (e.g., “embedding outlier,” “gradient- influential,” “cross-model inconsistent”), visualizations such as scatter plots of embeddings with flagged samples highlighted, class distribution charts showing anomalous contributions, and correlation heatmaps, and interactive review options enabling the user to export, inspect, or remove flagged samples, or to accept / reject system recommendations for specific data points.

[0075] Once the processing server 102 has flagged potentially poisoned data samples in the dataset (e.g., the data 208), the processing server 102 can clean or neutralize the dataset (e.g., the data 208). The system 100 provides users (e.g., of the user computing devices 104A-104N) with both automated and semi-automated methods to remove or neutralize adversarial effects while preserving as much data utility asAttorney Docket No.690248.0002 / 1WO Page 74 of 189 possible. The processing server 102 can clean and / or neutralize the potentially poisoned data samples in the dataset (e.g., the data 208) using one or more cleaning methods such as, but not limited to, sample removal, sample repair / neutralization, weighting and down-scaling, counterfactual augmentation, and human-in-the-loop review. Sample removal can include the direct elimination of flagged samples exceeding a user-defined or system-suggested risk threshold, and / or tiered removal based on poisoning risk scores (e.g., top 1%, 5%, 10% of riskiest samples). Sample Repair / Neutralization can include feature-level correction (e.g., smoothing extreme outliers, re-encoding corrupted tokens, or denoising adversarial image perturbations), interpolation / replacement using nearest-neighbor or cluster-based averages to substitute anomalous values, and / or noise neutralization through adversarial denoising autoencoders or model-assisted reconstruction. Weighting and down-scaling can include assigning lower weights to flagged data samples proportional to their risk score, which allows the retention of data while mitigating adversarial influence on gradients and parameters. Counterfactual augmentation can include the generation of synthetic replacements for suspected poisoned samples using GANs, VAEs, or diffusion models, guided by class distributions to ensure class balance is maintained even if original samples are removed. Human-in-the-loop review can include the option for domain experts to manually inspect flagged data before final removal or repair, which provides interpretability and auditability for high-risk datasets. As with prior modules, cleaning / neutralization remains configurable to the needs of the end-user (e.g., of the user computing devices 104A-104N). A user (e.g., of the user computing devices 104A-104N) can elect to: a) take no action and simply monitor the flaggedAttorney Docket No.690248.0002 / 1WO Page 75 of 189 points, b) partially clean by removing only the highest-risk data, or c) fully neutralize the dataset through automated corrections and replacements. After any cleaning or neutralization, the processing server can evaluate both the original dataset and the cleaned dataset across a library of model-agnostic methods or, if specified, a set of user-selected modeling approaches. This evaluation includes general performance metrics such as accuracy, precision, recall, and F1 score, as well as subset-specific evaluations (e.g., vulnerable subpopulations, minority classes, rare feature distributions) that are often disproportionately impacted by data poisoning. By presenting both global and subset-level comparisons between the original and cleaned datasets, the system 100 ensures that the user can understand not only the overall gains in resiliency and poisoning resistance but also whether important subgroups have been restored to more stable and representative performance levels. These results are integrated into the reporting module so that trade-offs between data retention, resiliency, and downstream utility are fully transparent.

[0076] In addition to the technical detection and neutralization workflows, the platform provides a set of reporting and educational modules specifically tailored to the processes of data filtration, flagging potentially poisoned samples, and cleaning or neutralizing the dataset. These modules serve two primary purposes: (a) to provide the end-user with transparent, actionable insights into the poisoning detection pipeline, and (b) to offer educational support so that users may better understand, prevent, and respond to poisoning risks in the future.

[0077] In such embodiments where the processing server 102 flags potentially poisoned data and / or cleans the potentially poisoned data as discussed above, theAttorney Docket No.690248.0002 / 1WO Page 76 of 189 data vulnerability report generated by the processing server 102 can include a summary of Flagged Samples (e.g., a total number of flagged points, percentage of dataset affected, and distribution of flagged data across classes or subpopulations), a risk score distributions (e.g., histograms or density plots of poisoning risk scores for all samples, highlighting thresholds chosen by the system or end-user), a summary of cleaning / neutralization actions taken (e.g., a description of which samples were removed, repaired, down-weighted, or replaced; percentages retained vs. eliminated) a before-and-after comparisons (.e.g,, side-by-side metrics of the original and cleaned datasets across global accuracy, precision, recall, and F1 score, as well as subgroup- specific metrics to identify improvements in vulnerable subsets),and / or resiliency and threshold Shifts (e.g., updated resiliency scores and poisoning thresholds calculated after cleaning, compared to the pre-cleaning baseline). The data vulnerability reports can be exportable in multiple formats (e.g., PDF, Word, CSV) and can be generated for technical users (detailed metrics and mathematical framing) or executive stakeholders (visual dashboards and summarized insights).

[0078] Further, in such embodiments where the processing server 102 flags potentially poisoned data and / or cleans the potentially poisoned data as discussed above, the processing server 102 can generate tiered educational content designed to meet the needs of users (e.g., of the user computing devices 104A-104N) with varying levels of technical expertise, such as, an introductory educational level, an intermediate educational level, and an advanced educational level. For example, an introductory educational level module can include plain-language explanations of data poisoning, why datasets are vulnerable, and what risk scores and thresholds mean. As anotherAttorney Docket No.690248.0002 / 1WO Page 77 of 189 example, an intermediate educational level module can include demonstrations of how poisoning manifests in different data modalities (e.g., tabular, NLP, vision, generative AI), and how the platform detects and mitigates such attacks. As yet another example, an advanced educational level module can include technical deep-dives into the algorithms and mathematical frameworks used for flagging, filtering, and neutralizing poisoned samples (e.g., anomaly detection, influence scoring, consistency metrics, counterfactual augmentation), and best practices for data collection and maintenance: guidance on designing pipelines that are more resistant to poisoning over time, including recommendations on preprocessing, validation, and monitoring strategies. By combining reporting with tailored educational support, this system 100 ensures that end-users (e.g., of the user computing devices 104A-104N) not only see the measurable outcomes of filtration and cleaning but also understand the rationale behind system decisions. This dual approach improves trust, encourages informed decision-making, and provides users with the knowledge necessary to proactively build and maintain more resilient datasets against adversarial poisoning in the future. Artificial Intelligence (AI) Manipulated and Generated Data Detection

[0079] In embodiments, the system 100 provides a robust, modular system for detecting, localizing, and explaining manipulation in imagery and video, whether generated by Artificial Intelligence (AI), modified by AI, or manually altered. The system 100 can provide automated and analyst-ready outputs that distinguish between pristine and manipulated media with high confidence, accuracy, and interpretability. Thus, the disclosed methods and systems herein improve the speed and reliability of triage workflows, enable early detection of misinformation threats, and empowerAttorney Docket No.690248.0002 / 1WO Page 78 of 189 analysts with actionable insights into tampered content across real-world, noisy data formats (e.g., compressed or cropped imagery).The system 100 can detect manipulation across a broad range of media formats and editing styles, including content generated by modern diffusion models, without relying on metadata or assumptions about content type. To ensure its practical use in real-world analysis workflows, the system 100 produces binary detection results, localized visual overlays, confidence levels aligned with intelligence community (IC) assessment standards, and plain-text summaries describing the nature and extent of any manipulation detected. This ensures results are transparent and actionable for analysts with varying levels of technical expertise, supporting faster, more accurate triage / decision-making. The system 100 (e.g., in the processing server 102) can include a set of specialized analysis microservices that support a container orchestration system, a data storage and versioning layer, and a dedicated results service. Each analytical method is encapsulated within its own containerized microservice. This isolation allows for independent development, deployment, and scaling based on specific computational demands. The microservices containers are built from base images tailored to the specific frameworks used (e.g. PyTorch) and include all necessary libraries. Integration with a container orchestration platform such as Kubernetes will allow for the management of deployment, scaling and lifecycle management of microservices. A data layer will provide data storage, data versioning and results management. A results and explanation service aggregates the outputs from the various analysis microservices and generate a comprehensive and human-readable explanation of the findings. This could include a final manipulation score, highlighted regions of interest,Attorney Docket No.690248.0002 / 1WO Page 79 of 189 and a textual summary of the detected inconsistencies. By adopting this microservice container-based design, the system 100 can provide a powerful, scalable, and resilient solution for the critical task of detecting and understanding digital media manipulation. This modular architecture of the system 100 not only supports the integration of a diverse set of current analysis techniques but also provides a flexible framework for incorporating future advancements in the field. In such embodiments, the processing server 102 can detect, localize, and explain manipulation in imagery and video, whether generated by Artificial Intelligence (AI), modified by AI, or manually altered using one or more AI data manipulation detection algorithms 221, such as, but not limited to, vision transformers (ViTs), convolution neural networks (CNNs) with class activation maps (CAMs), TimeSformer, TSM, recurrent all-pairs field transforms (RAFT), frequency analysis using DCT and FFT, frame noise correlation analyzer (FNCA), MLP with Bayesian Confidence Estimation, visual overlays, and textual summaries.

[0080] Vision transformers (ViTs) operate by dividing an image into a sequence of fixed-size patches, embedding each as a token, and applying self-attention mechanisms across all patches to capture global context. This allows ViTs to model long-range dependencies across the entire image, making them particularly effective at identifying structural or semantic inconsistencies that may result from manipulation. In the system 100, the processing server 102 utilizes ViTs for forensic image analysis by training them to detect spatial anomalies – such as pasted objects, altered textures, inpainted areas, or inconsistencies in lighting and geometry. These manipulations often disrupt the global coherence of a scene, and ViTs can flag these inconsistenciesAttorney Docket No.690248.0002 / 1WO Page 80 of 189 due to their ability to reason holistically about spatial relationships. For static images, ViTs are applied by the processing server 102 frame-level to directly evaluate whether spatial patterns make semantic and physical sense across the scene. The processing server 102 can fine-tune the models on labeled datasets containing a range of manipulation types, from AI-generated insertions to manual splicing, across domains including terrain, infrastructure, and vehicles. The attention maps generated by the processing server 102 during inference are used to create saliency heatmaps that identify regions contributing most to the detection, which feed directly into the system’s visual overlay module for human review. For video, the processing server applies ViTs on a per-frame basis to detect spatial anomalies that may be transient or subtle – complementing our temporal models. This frame-level ViT analysis is particularly important when manipulation affects only select frames (e.g., a single inpainted frame in an otherwise clean sequence).

[0081] The processing server 102 can utilize convolution neural networks (CNNs) with class activation maps (CAMs) to provide fine-grained manipulation detection and interpretable visual localization. CAMs enable CNNs to highlight spatial regions of an image that are most relevant to a specific classification decision by modifying a standard CNN architecture to replace fully connected layers with global average pooling, allowing the final class scores to be projected back onto the last convolutional feature maps. The result is a class-specific heatmap that reveals which parts of the input image contributed most to a given prediction. The system 100 extends the use of CAMs from their typical classification setting to the domain of forensic manipulation detection. The processing server trains CNNs to distinguish between pristine andAttorney Docket No.690248.0002 / 1WO Page 81 of 189 manipulated images, with CAMs producing activation maps that highlight suspicious or anomalous regions. These regions often correspond to areas where objects have been inserted, removed, inpainted, or otherwise altered – whether by AI or manual techniques. To ensure the CNNs are sensitive to localized manipulation artifacts, the processing server 102 trains on a dataset that includes a diverse set of edits applied across various scenes – such as urban, natural, and mixed terrain—and post- processed to simulate real-world degradation (e.g., compression, cropping). During inference, the resulting CAM heatmaps serve a dual purpose: they guide the analyst- facing visual overlay layer by indicating where manipulations are most likely.and they provide spatial interpretability within the system’s ensemble detection architecture, complementing global detectors like ViTs. This approach is especially valuable in low- contrast or subtle manipulations, where direct detection may be difficult but attention- based localization can highlight inconsistent textures or unnatural seams. The CAMs thereby serve as a bridge between machine decisions and human review, enabling more intuitive inspection of flagged regions. By integrating CAM-enhanced CNNs into the detection pipeline, the processing server 102 gains high spatial resolution awareness and a reliable, explainable means of localizing edits – key for both detection accuracy and analyst trust.

[0082] TimeSformer is a transformer-based architecture explicitly designed for video understanding tasks. Unlike traditional video models that rely on 3D convolutions – which are computationally expensive and often limited in their ability to model long- range dependencies – TimeSformer applies self-attention mechanisms across both the spatial and temporal dimensions of video. This allows the model to learn rich,Attorney Docket No.690248.0002 / 1WO Page 82 of 189 spatiotemporal features directly from sequences of frame-level patches, maintaining the same “token-based” representation used in ViTs, but extended through time.

[0083] In the system 100, the processing server 102 utilizes TimeSformer to detect manipulations and irregularities in video that are introduced through either AI-based generation (e.g., deepfake-style diffusion synthesis) or manual tampering. Specifically, the processing server 102 uses TimeSformer to identify temporal inconsistencies that would not exist in naturally captured footage, such as: sudden or unnatural scene transitions, repeated or skipped frames, inconsistent object motion (e.g., objects moving without proper background displacement), and / or visual artifacts caused by inpainting, frame blending, or AI-driven interpolation. TimeSformer’s attention-based structure excels at capturing both short-term transitions and long-range dependencies across the video sequence. This is critical in manipulated media where the tampering may occur subtly – across only a few frames – or more structurally, such as inserting synthetic scenes or stitching together footage from disparate sources. Additionally, its patch-based modeling aligns well data manipulation detection: each video frame is divided into spatial tokens, which allows the processing server 102 to pinpoint not just that a manipulation occurred, but where and when it occurred within the video. The output from TimeSformer feeds into the processing server’s 102 ensemble detection layer, where it is fused with outputs from spatial-only detectors (like ViT and CAMs), forensic modules (e.g., frequency and sensor noise analysis), and decision models (Multilayer Perceptron (MLP) + Bayesian confidence estimation). This fusion allows TimeSformer’s unique temporal insights to strengthen overall detection reliability, especially in video formats commonly found on social platforms, which are oftenAttorney Docket No.690248.0002 / 1WO Page 83 of 189 compressed, cropped, or altered. Ultimately, TimeSformer empowers the system 100 with a deep understanding of scene evolution over time, enabling it to flag subtle yet critical manipulations that would escape static image analysis or frame-by-frame CNN comparisons

[0084] TSM is a lightweight and efficient method for enabling temporal modeling within traditional 2D CNNs. Rather than using computationally expensive 3D convolutions or recurrent architectures to capture temporal dependencies, TSM introduces a simple yet powerful mechanism: it shifts a portion of the feature channels along the temporal dimension, allowing the network to access information from adjacent video frames without modifying its spatial convolutional backbone. In the system 100 TSM into the detection pipeline of the processing server 102 to identify short-term temporal anomalies in video data – particularly those indicative of manipulation or synthetic generation—such as, but not limited to frame repetition where a single frame is duplicated to pad or stretch a segment, temporal flicker resulting from inconsistent lighting, inpainting, or misaligned GAN synthesis, and motion jitter caused by poor frame interpolation or manual tampering during editing. TSM's key advantage is its ability to retain the efficiency of 2D CNNs while gaining temporal awareness. This makes it highly suitable for scenarios where real-time or near-real-time video processing is required, such as automated triage systems or cloud-based pipelines. Unlike more resource-intensive methods like 3D CNNs or full Transformers, TSM can be deployed with minimal additional computational cost – allowing the system to scale to high volumes of video data without sacrificing detection fidelity. To optimize its use, TSM modules are embedded into a CNN architecture already fine-tuned for spatialAttorney Docket No.690248.0002 / 1WO Page 84 of 189 manipulation detection. The added temporal shifts enable these networks to contextualize frame-level features in the broader timeline, revealing inconsistencies that static frame-by-frame analysis would miss. These outputs are then passed to the ensemble decision layer of the processing server 102 and used in coordination with models like TimeSformer, which captures long-range dependencies. By addressing short-range temporal distortions – a common signature of synthetic video generation and tampering – TSM serves as a crucial component in our multi-layered detection approach. It enhances detection accuracy and robustness while preserving the system’s ability to operate under real-world constraints such as low resolution, lossy compression, and limited compute environments.

[0085] Recurrent All-Pairs Field Transforms (RAFT) is a state-of-the-art deep learning model for dense optical flow estimation – the process of computing pixel-level motion between consecutive video frames. Unlike traditional optical flow methods that rely on pyramidal hierarchies or sparse feature tracking, RAFT introduces a novel approach based on all-pairs correlation volumes and recurrent refinement, enabling highly accurate motion estimation across both small and large displacements. In the system 100, RAFT is utilized by the processing server 102 to identify motion anomalies that may indicate manipulation, splicing, or full AI generation within a video. Authentic videos captured by physical cameras typically exhibit smooth and coherent motion patterns across foreground and background objects. In contrast, manipulated or synthesized footage often introduces subtle – but detectable – motion inconsistencies, such as: static or "frozen" objects that should be moving with the scene, misaligned motion between foreground objects and background flow (e.g., inserted vehicles thatAttorney Docket No.690248.0002 / 1WO Page 85 of 189 don’t match camera panning), and / or abrupt or unnatural changes in motion vectors between frames, often resulting from frame interpolation or manual editing. RAFT’s dense, frame-to-frame correspondence allows the processing server 102 to precisely track these irregularities. The processing server 102 utilizes RAFT to compute motion not just at key points but for every pixel, enabling detection of localized tampering – even when the edit affects only a small region of the frame. The recurrent refinement module further increases accuracy by iteratively improving flow estimates, making it particularly effective in degraded or compressed video conditions, which are common in open-source and social media footage. The processing server utilizes RAFT in tandem with models like TimeSformer and TSM to provide complementary temporal evidence of manipulation. While those models identify spatiotemporal inconsistencies in feature space, RAFT grounds its findings in physical motion fields, providing a different modality of forensic analysis. Outputs from RAFT are also incorporated into the visual overlay system, where motion vectors and detected anomalies can be visualized as flow maps or annotated artifacts.

[0086] Frequency Analysis using DCT and FFT provides a critical forensic capability for detecting subtle artifacts introduced during AI generation or manual manipulation of images and video. The processing server 102 utilizes frequency-domain transforms to uncover hidden irregularities not readily visible in the spatial domain. In the system 100, the processing server 102 applies DCT or FFT to both still images and individual frames of video content, analyzing their amplitude spectra to identify statistical deviations from expected real-world frequency distributions. Key indicators of manipulation in the frequency domain include:lack of high-frequency detail, whichAttorney Docket No.690248.0002 / 1WO Page 86 of 189 suggests synthetic textures or smoothed-over editing boundaries, excessively uniform or "blocky" low-frequency regions, which may result from generative upsampling or aggressive inpainting, and repetitive frequency spikes or suppression bands, often left behind by deep generative architectures like GANs or diffusion models. This spectral information is processed by the processing server 102 independently from spatial or temporal models like ViT or TimeSformer, providing a complementary signal that is especially effective when pixel-space artifacts have been concealed or degraded by post-processing (e.g., compression, blurring, or resizing). In practice, frequency analysis outputs are fused with other subsystem detections in the decision layer and contribute to both binary classification (pristine vs. manipulated) and localized anomaly maps. For instance, spatial frequency heatmaps can indicate where in an image frequency anomalies are concentrated – such as around pasted objects or tampered edges – enabling precise visual overlays for analysts. Furthermore, frequency-based detectors can retain discriminative power even under lossy compression (e.g., JPEG, WEBP, MPEG-4), because while compression distorts spectral content—particularly high-frequency bands—it does so in characteristic and learnable ways. Generative manipulations, by contrast, often introduce different spectral signatures, such as unnatural periodicities, frequency band suppression, or overly smooth transitions, which can still be distinguished from typical compression artifacts.

[0087] Frame Noise Correlation Analyzer (FNCA) detects temporal manipulation by analyzing frame-to-frame noise continuity. Authentic video captured by physical cameras exhibits independent high-frequency sensor noise across frames due to natural shot noise and sensor variation. In contrast, AI-generated or interpolatedAttorney Docket No.690248.0002 / 1WO Page 87 of 189 frames – such as those used to artificially slow down footage or pad sequences – often exhibit unnatural noise consistency over time. FNCA works by applying a high- pass filter (e.g., wavelet or Laplacian-based) to isolate the sensor noise residuals in each frame, then computing frame-to-frame correlation metrics (e.g., Pearson correlation or mutual information) across a sliding temporal window. Statistically significant correlations across temporally adjacent frames are flagged as anomalous. This signal-agnostic module complements spatiotemporal models like TimeSformer and RAFT by capturing manipulation signatures not tied to object motion or semantics. FNCA outputs are integrated into the system’s fusion layer and can be visualized through confidence-coded overlays, enabling analysts to detect subtle frame synthesis or interpolation artifacts common in generative video workflows.

[0088] MLP with Bayesian Confidence Estimation serves as the final decision-making layer in the system 100. After each specialized subsystem – such as ViT, RAFT, and frequency transforms – generates its own outputs and anomaly scores, the MLP functions as a fusion center, aggregating these diverse signals into a unified classification: pristine or manipulated. At its core, the MLP is a simple, fully connected neural network that is well-suited for learning decision boundaries across multi-modal input features. However, to support operational requirements for transparency and triage, the system 100 augments the MLP with Bayesian confidence estimation techniques, such as Monte Carlo Dropout or deep ensemble sampling. These techniques allow the system 100 to express not just what it predicts, but how confident it is in those predictions, providing an essential capability in high-stakes, mission- critical scenarios. The MLP’s role is twofold: fusion and final decision (e.g., it ingests aAttorney Docket No.690248.0002 / 1WO Page 88 of 189 feature vector composed of each subsystem’s detection scores, spatial / temporal features, frequency anomalies, and correlation values, which are combined to produce a binary prediction); and confidence modeling (e.g., by applying Bayesian methods, the system 100 generates a calibrated confidence score alongside the prediction, which reflects factors like cross-model agreement, data quality, and the consistency of anomaly patterns. This design is critical to the overall utility of the system 100. Users (e.g., of the user computing devices 104A-104N) receive not just a binary output, but a ranked and interpretable score indicating how reliable that output is – allowing them to prioritize high-confidence detections for immediate action, while flagging lower- confidence cases for further review. While the BAA requires a binary pristine / not- pristine classification, the system 100 can handle common benign modifications – such as burned-in timestamps, watermark logos, or news chyrons – in a way that balances technical fidelity with operational utility. These elements do constitute deviations from the original sensor output and therefore may register as “not pristine” under a strict binary framework. However, the system 100 incorporates several mechanisms to mitigate undue impact on analyst workflows: scope of binary classification (e.g., the system 100 defines “not pristine” as any physical or semantic deviation from the original sensor signal, without regard to intent, to ensure technical consistency in classification while deferring context-based interpretation to downstream tools and human analysts), confidence-based filtering and explanation layer (e.g., the inclusion of Bayesian confidence modeling and visual / textual explanation modules provides essential context for triage as benign overlays tend to yield low or moderate confidence scores and are typically localized to predictableAttorney Docket No.690248.0002 / 1WO Page 89 of 189 regions (e.g., frame corners or lower thirds) and textual summaries will reflect this by describing the modification in plain language (e.g., “Static overlay detected – consistent with watermark or broadcast graphics”)), training with benign examples (e.g., the system 100’s training and evaluation datasets will include examples of common non-malicious edits, which allows the models to learn and calibrate frequent, low-risk overlays, reducing the likelihood of these instances being misclassified with high confidence), analyst-centered triage interface (e.g., while binary results are preserved to meet programmatic requirements, the system 100 interface allows analysts to sort and filter detections by manipulation and confidence, which enables efficient dismissal of expected or irrelevant flags (e.g., known broadcast logos) while retaining visibility on high-risk or ambiguous cases), and future extensibility (e.g., though currently constrained to binary output, the architecture is designed to support graduated classification—potentially distinguishing between pristine, benignly altered, and maliciously manipulated content in future iterations, should policy allow). Moreover, this module interfaces directly with visual overlay and text generation systems. Its confidence score is reflected in heatmap intensity (e.g., red for high, yellow for moderate), and its classification output informs the structured plain-language descriptions presented to the analyst. This ensures that machine-derived results are both traceable and actionable, forming the final bridge between advanced AI processing and mission-relevant human decision-making.

[0089] Visual overlays provide the primary interface between the system 100’s internal detections and the human analyst. While underlying models such as ViT and RAFT detect complex patterns of manipulation in spatial, temporal, and forensic domains, theAttorney Docket No.690248.0002 / 1WO Page 90 of 189 visual overlay module translates those findings into intuitive, interpretable graphical indicators. This is critical for analyst usability, especially in time-sensitive or mission- critical environments. The overlay system highlights regions of suspected manipulation directly on the image or video frame using heatmaps, bounding boxes, or region- based shading. Each visual indicator is color-coded to reflect the confidence level of the detection: red denotes high-confidence manipulation, yellow indicates moderate confidence, gray or transparent shading may be used to signal low-confidence areas or uncertainty. This allows analysts to immediately focus their attention on areas that matter most, reducing the cognitive load required to interpret complex model outputs.In the case of video, the overlays are dynamic. Analysts can scrub through time, frame by frame, and see how the detected manipulation regions evolve or persist across sequences. For example, a flickering object, spliced vehicle, or inconsistent motion pattern may be visually tracked across a timeline, helping the analyst build a spatiotemporal understanding of how and where tampering occurred. These overlays are tightly coupled with the system’s Bayesian confidence estimation pipeline. Each pixel or region inherits its confidence level from the fused outputs of the detection subsystems, ensuring that visual emphasis is driven by quantifiable model certainty. This enables a powerful combination of machine precision and human interpretability. Ultimately, the visual overlays serve as both a triage tool and a forensic aid, providing analysts with immediate visual summaries of manipulation without requiring them to interpret raw neural network scores or anomaly metrics. This supports rapid screening of high volumes of media and also provides a starting point for deeper forensic analysis when needed.Attorney Docket No.690248.0002 / 1WO Page 91 of 189

[0090] Textual summaries translate complex model detections into clear, structured plain-language descriptions that analysts can quickly read and act upon. While visual overlays highlight the where of detected manipulations, textual summaries explain the what and how certain the system is – bridging the gap between AI-driven analysis and human decision-making. Each time the system 100 detects a manipulation – whether spatial, temporal, or forensic – it generates a concise narrative that includes: the type of manipulation identified (e.g., object insertion, motion inconsistency, splicing anomaly), the location or timeframe (e.g., “top-left quadrant,” or “frames 15–32”), and a confidence score, derived from Bayesian uncertainty modeling, categorized into intuitive levels aligning with Intelligence Community Directive Analytic Tradecraft Standards: High, Moderate, or Low. These summaries are template-driven but data- informed. That is, they follow a consistent format for clarity and efficiency, while pulling specific values from the underlying model outputs to ensure that each description remains directly traceable to the evidence. For instance, a detection may result in the following output: Manipulation #1: Motion InconsistencyThis format enables analysts – regardless of technical background – to understand what was found, where, and how strongly the system supports the finding. For moreAttorney Docket No.690248.0002 / 1WO Page 92 of 189 experienced users, these summaries also provide cues for further investigation or cross-referencing against visual overlays and source material. The system’s ability to automatically generate these descriptions greatly improves workflow scalability. Rather than requiring an analyst to interpret model scores or cross-validate visual cues manually, the textual summary offers a ready-to-use briefing, supporting triage, reporting, or escalation. Processing Server

[0091] FIG.2 illustrates an embodiment of the processing server 102 in the system 100. It will be apparent to persons having skill in the relevant art that the embodiment of the processing server 102 illustrated in FIG.2 is provided as illustration only and cannot be exhaustive to all possible configurations of the processing server 102 suitable for performing the functions as discussed herein. For example, the computer system 500 illustrated in FIG.5 and discussed in more detail below can be a suitable configuration of the processing server 102.

[0092] The processing server 102 can include a receiving device 202. The receiving device 202 can be configured to receive data over one or more networks via one or more network protocols. In some instances, the receiving device 202 can be configured to receive data from the user computing devices 104A-104N, the data sources 106A-106N, and other systems and entities via one or more communication methods, such as radio frequency, local area networks, wireless area networks, cellular communication networks, Bluetooth, the Internet, etc. In some embodiments, the receiving device 202 can be comprised of multiple devices, such as different receiving devices for receiving data over different networks, such as a first receivingAttorney Docket No.690248.0002 / 1WO Page 93 of 189 device for receiving data over a local area network and a second receiving device for receiving data via the Internet. The receiving device 202 can receive electronically transmitted data signals, where data can be superimposed or otherwise encoded on the data signal and decoded, parsed, read, or otherwise obtained via receipt of the data signal by the receiving device 202. In some instances, the receiving device 202 can include a parsing module for parsing the received data signal to obtain the data superimposed thereon. For example, the receiving device 202 can include a parser program configured to receive and transform the received data signal into usable input for the functions performed by the processing device to carry out the methods and systems described herein. In embodiments, the receiving device 202 can be configured to receive data signals electronically transmitted by the user computing devices 104A-104N and / or the data sources 106A-106N that can be superimposed or otherwise encoded with datasets.

[0093] The processing server 102 can also include a communication module 204. The communication module 204 can be configured to transmit data between modules, engines, databases, memories, and other components of the processing server 102 for use in performing the functions discussed herein. The communication module 204 can be comprised of one or more communication types and utilize various communication methods for communications within a computing device. For example, the communication module 204 can be comprised of a bus, contact pin connectors, wires, etc. In some embodiments, the communication module 204 can also be configured to communicate between internal components of the processing server 102 and external components of the processing server 102, such as externally connectedAttorney Docket No.690248.0002 / 1WO Page 94 of 189 databases, display devices, input devices, etc. The processing server 102 can also include a processing device. The processing device can be configured to perform the functions of the processing server 102 discussed herein as will be apparent to persons having skill in the relevant art. In some embodiments, the processing device can include and / or be comprised of a plurality of engines and / or modules specially configured to perform one or more functions of the processing device, such as the analyzing module 222, the generation module 224, the validation module 226, etc. As used herein, the term “module” can be software or hardware particularly programmed to receive an input, perform one or more processes using the input, and provides an output. The input, output, and processes performed by various modules will be apparent to one skilled in the art based upon the present disclosure.

[0094] The processing server 102 can also include a database 206. The database 206 can be configured to store datasets such as the data 208 and the attack data 209, discussed in more detail above. The database 206 can be a relational database that utilizes structured query language for the storage, identification, modifying, updating, accessing, etc. of structured data sets stored therein. Further, the database 206 can include algorithms and models for performing the functions of the processing server 102 such as, for example, the pre-processing algorithms 210, the data featurization algorithms 212, the data modeling algorithms 214, the model attack algorithms 216, the vulnerability metrics 218, and the data hardening algorithms 219, etc.

[0095] The processing server 102 can also include a memory 220. The memory 220 can be configured to store data for use by the processing server 102 in performing the functions discussed herein. The memory 220 can be configured to store data usingAttorney Docket No.690248.0002 / 1WO Page 95 of 189 suitable data formatting methods and schema and can be any suitable type of memory, such as read-only memory, random access memory, etc. The memory 220 can include, for example, communication protocols and standards, data formatting standards and protocols, program code for modules and application programs of the processing device, and other data that can be suitable for use by the processing server 102 in the performance of the functions disclosed herein as will be apparent to persons having skill in the relevant art. In some embodiments, the memory 220 can be comprised of or can otherwise include a relational database that utilizes structured query language for the storage, identification, modifying, updating, accessing, etc. of structured data sets stored therein.

[0096] The processing server 102 can include a analyzing module 222. The analyzing module 222 can be configured to execute the algorithms stored in the database 206 to process the data 208. The analyzing module 222 can receive the data 208 and can execute a preprocessing algorithm 210 to clean and prepare the data for use by the processing server 102. The analyzing module 222 analyze the data 208 to identify the data types included in the data 208 and then select and execute one or more of the data featurization algorithms 212 on the data 208 based on the identified data types. The analyzing module 222 can analyze the victim models to determine one or more performance metrics of the victim models both before and after a model attack algorithm has been applied to the victim models. The analyzing module 222 can also analyze datasets received by the processing server 102 and identify and execute one or more applicable vulnerability metrics 218 on the datasets. Further, the analyzing module 222 can also analyze datasets received by the processing server 102 andAttorney Docket No.690248.0002 / 1WO Page 96 of 189 identify and execute one or more applicable data hardening algorithms 219 on the datasets.

[0097] The processing server 102 can also include a generation module 224. The generation module 224 can be configured to generate data for use by the processing server 102 in performing the functions discussed herein. The generation module 224 can receive instructions as input, can generate data based on the instructions, and can output the generated data to one or more modules of the processing server 102. For example, the generation module 224 can be configured to generate one or more victim models using the data modeling algorithms 214. The generation module 224 can generate one or more attack datasets (e.g., the attack data 209) as discussed in more detail above. The generation module 224 can also generate dataset vulnerability reports for the datasets (e.g., the data 208) as discussed in more detail above.

[0098] The processing server 102 can also include a validation module 226. The validation module 226 can be configured to perform data validations and verifications for the processing server 102 as part of the functions discussed herein. The validation module 226 can receive instructions as input, can perform data validations or verification as instructed, and can output a result of the data validations or verifications to one or more modules of the processing server 102. In some cases, the input can include the data to be validated or verified and / or data to be used in the validation or verification. In other cases, the validation module 226 can be configured to identify such data, such as in the database 206 and / or memory 220. The validation module 226 can be configured to, for example, evaluate the datasets and / or hardened datasets using the vulnerability metrics 218.Attorney Docket No.690248.0002 / 1WO Page 97 of 189

[0099] The processing server 102 can also include a transmitting device 228. The transmitting device 228 can be configured to transmit data over one or more networks via one or more network protocols. In some instances, the transmitting device 228 can be configured to transmit data to the user computing devices 104A-104N, the data sources 106A-106N, and other entities via one or more communication methods, local area networks, wireless area networks, cellular communication, Bluetooth, radio frequency, the Internet, etc. In some embodiments, the transmitting device 228 can be comprised of multiple devices, such as different transmitting devices for transmitting data over different networks, such as a first transmitting device for transmitting data over a local area network and a second transmitting device for transmitting data via the Internet. The transmitting device 228 can electronically transmit data signals that have data superimposed that can be parsed by a receiving computing device. In some instances, the transmitting device 228 can include one or more modules for superimposing, encoding, or otherwise formatting data into data signals suitable for transmission.

[0100] The transmitting device 228 can be configured to electronically transmit data signals to the user computing devices 104A-104N and / or the data sources 106A-106N that can be superimposed or otherwise encoded with requests for data sets and data included therein. The transmitting device 228 can also be configured to electronically transmit data signals to the user computing devices 104A-104N and / or the data sources 106A-106N, which can be superimposed or otherwise encoded with dataset vulnerability reports. Process for Analyzing Datasets for Vulnerabilities to Adversarial AttacksAttorney Docket No.690248.0002 / 1WO Page 98 of 189

[0101] FIGS.3-3B illustrates a process 300 in the system 100 of FIG.1 for analyzing datasets for vulnerabilities to adversarial attacks in accordance with exemplary embodiments.

[0102] In step 302, the processing server 102 stores a dataset (e.g., the data 208) in the database 206. The dataset (e.g., the data 208) can include data of one or more data types, such as, one or more of tabular data, natural language data, computer vision data, and generative artificial intelligence (AI) data.

[0103] In step 304, the processing server 102 preprocesses (e.g., via the analyzing module 222) the dataset (e.g., the data 208) using one or more of the data preprocessing algorithms 210 based on the one or more data types included in the dataset (e.g., the data 208). The one or more preprocessing algorithms 210 can include, for example, one or more of a data cleaning algorithm, a feature scaling and transformation algorithm, a feature engineering algorithm, a text cleaning algorithm, a tokenization and normalization algorithm, a stopword and noise removal algorithm, a text representation algorithm, a miscellaneous NLP-specific algorithm, a basic image cleaning algorithm, an image resizing and cropping algorithm, a pixel normalization algorithm, a noise reduction algorithm, and a data augmentation algorithm.

[0104] In step 306, the processing server 102 performs (e.g., via the analyzing module 222) data featurization of the dataset (e.g., the data 208) based on the one or more data types included in the dataset (e.g., the data 208) using one or more of the data featurization algorithms 212, to identify one or more data features of the dataset (e.g., the data 208).Attorney Docket No.690248.0002 / 1WO Page 99 of 189

[0105] In step 308, the processing server 102 generates (e.g., via the generation module 224) one or more victim models (e.g., via the data modeling algorithms 214) using the dataset (e.g., the data 208) based on the one or more data types included in the dataset (e.g., the data 208). Each of the one or more victim models is specific to one of the one or more data types included in the dataset (e.g., the data 208). In step 310, the processing server 102 determines one or more performance metrics of the one or more victim models.

[0106] In step 312, the processing server 102 generates (e.g., via the generation module 224) one or more attack datasets (e.g., the attack data 209). Each of the one or more attack datasets (e.g., the attack data 209) corresponds to one of the one or more data types included in the dataset (e.g., the data 208). Each of the one or more attack datasets (e.g., the attack data 209) can include at least the one or more performance metrics of the one or more victim models, and the one or more data features of the dataset (e.g., the data 208).

[0107] In step 314, the processing server 102 executes one or more adversarial attacks (e.g., the model attack algorithms 216) on the one or more victim models. The one or more adversarial attacks (e.g., the model attack algorithms 216) can include, for example, a membership inference model inversion attack, a property inference model inversion attack, and / or a reconstruction model inversion attack.

[0108] In step 316, the processing server 102 determines one or more adversarial attack success metrics based on the executed one or more adversarial attacks (e.g., the model attack algorithms 216) on the one or more victim models. The one or more adversarial attack success metrics can include, for example, an accuracy of theAttorney Docket No.690248.0002 / 1WO Page 100 of 189 adversarial attack, a precision of the adversarial attack, and / or an F1 score of the adversarial attack. In step 318, the processing server 102 updates the one or more attack datasets (e.g., the attack data 209) to include the one or more adversarial attack success metrics.

[0109] In step 320, the processing server 102 generates (e.g., via the generation module 224) a vulnerability metric (e.g., the vulnerability metric 218) for each of the one or more updated attack datasets (e.g., the attack data 209). Each vulnerability metric (e.g., the vulnerability metric 218) is a library of algorithms and models specific to a combination of one of the one or more adversarial attacks (e.g., model attack algorithms 216) performed on one of the one or more data types included in the dataset (e.g., the data 208).

[0110] In step 322, a user computing device 104 stores a second dataset including at least one data type and transmits the second data set to the processing server in step 324.

[0111] In step 326, the processing server 102 receives (e.g., via the receiving device 202) the second dataset from the user computing device 104. In step 328, the processing server 102 preprocess the second dataset using the one or more preprocessing algorithms 210 based on the at least one data type of the second dataset.

[0112] In step 330, the processing server performs data featurization (e.g., using the data featurization algorithms 212) of the second dataset based on the at least one data type of the second dataset to identify one or more data features of the second dataset. In step 332, the processing server 102 identifies at least one vulnerabilityAttorney Docket No.690248.0002 / 1WO Page 101 of 189 metric of the generated vulnerability metrics (e.g., the vulnerability metrics 218) that corresponds to the at least one data type of the second dataset and one of the one or more model attack algorithms 216. In step 334, the processing server 102 applies the identified at least one vulnerability metric 218 to the second dataset.

[0113] In step 336, the processing server 102 generates (e.g. via the generation module 224) a dataset vulnerability report for the second dataset that inlcudes a quantitative value of risk that the second dataset is susceptible to the identified one of the one or more model attack algorithms 216. In step 338, the processing server 102 displays, via a user interface, the dataset vulnerability report for the second dataset to a user of the user computing device 104. For example, the processing server 102 can display the dataset vulnerability report for the second dataset to the user of the user computing device 104 on a web interface, an application programming interface (API), or any other suitable user interface for displaying a report to the user of the user computing device 104. In embodiments, displaying the dataset vulnerability report for the second dataset to the user of the user computing device 104 can include the processing server 102 transmitting (e.g., via the transmitting device 228) the dataset vulnerability report for the second dataset to the user computing device 104. In such embodiments, the user computing device 104 receives the dataset vulnerability report for the second dataset in step 340. Exemplary Method for Analyzing Datasets for Vulnerabilities to Adversarial Attacks

[0114] FIG.4 illustrates a method 400 in the system 100 for analyzing datasets for vulnerabilities to adversarial attacks in accordance with exemplary embodiments.Attorney Docket No.690248.0002 / 1WO Page 102 of 189

[0115] In step 402, a processing device (e.g., the processing server 102) stores in a database (e.g., the database 206) a dataset (e.g., the data 208). The dataset (e.g., the data 208) can include, for example, one or more data types such as, but not limited to, one or more of tabular data, natural language data, computer vision data, and generative artificial intelligence (AI) data.

[0116] In step 404, the processing device (e.g., the processing server 102 via the analyzing module 222) preprocess the dataset (e.g., the data 208) using one or more data preprocessing algorithms (e.g., the preprocessing algorithms 210) based on the one or more data types included in the dataset (e.g., the data 208). The one or more data preprocessing (e.g., the preprocessing algorithms 210) algorithms can include, for example, but are not limited to, one or more of a data cleaning algorithm, a feature scaling and transformation algorithm, a feature engineering algorithm, a text cleaning algorithm, a tokenization and normalization algorithm, a stopword and noise removal algorithm, a text representation algorithm, a miscellaneous NLP-specific algorithm, a basic image cleaning algorithm, an image resizing and cropping algorithm, a pixel normalization algorithm, a noise reduction algorithm, and a data augmentation algorithm.

[0117] In step 406, the processing device (e.g., the processing server 102 via the analyzing module 222) performs data featurization of the dataset (e.g., the data 208) based on the one or more data types included in the dataset (e.g., the data 208) using one or more data featurization algorithms (e.g., the data featurization algorithms 212). The data featurization (e.g., the data featurization algorithms 212) identify one or more data features of the dataset (e.g., the data 208).Attorney Docket No.690248.0002 / 1WO Page 103 of 189

[0118] In step 408, the processing device (e.g., the processing server 102 via the generation module 224) generates one or more victim models using the dataset (e.g., the data 208 and the data modeling algorithms 214) based on the one or more data types included in the dataset (e.g., the data 208). Each of the one or more victim models is specific to one of the one or more data types included in the dataset (e.g., the data 208). For example, if the dataset includes tabular data and natural language fata, the processing device (e.g., the processing server 102 via the generation module 224) can generate one or more victim models for the tabular data and one or more separate victim models for the natural language data.

[0119] In step 410, the processing device (e.g., the processing server 102 via the analyzing module 222) determines one or more performance metrics of the one or more victim models.

[0120] In step 412, the processing device (e.g., the processing server 102 via the generation module 224) generates one or more attack datasets (e.g., the attack data 209). Each of the one or more attack datasets (e.g., the attack data 209) can correspond to one of the one or more data types included in the dataset (e.g., the data 208). For example, there can be one attack dataset (e.g., the attack data 209) for tabular data and another attack dataset (e.g., the attack data 209) for natural language data. Each of the one or more attack datasets (e.g., the attack data 209) can include at least the one or more performance metrics of the one or more victim models, and the one or more data features of the dataset (e.g., the data 208).

[0121] In step 414, the processing device (e.g., the processing server 102) executes one or more adversarial attacks (e.g., using the model attack algorithms 216) on theAttorney Docket No.690248.0002 / 1WO Page 104 of 189 one or more victim models. The one or more adversarial attacks can include, for example, but are not limited to, one or more of a membership inference model inversion attack, a property inference model inversion attack, and a reconstruction model inversion attack.

[0122] In step 416, the processing device (e.g., the processing server 102 via the analyzing module 222) determines one or more adversarial attack success metrics. The one or more adversarial attack success metrics can include, for example, but are not limited to, one or more of an accuracy of the adversarial attack, a precision of the adversarial attack, and an F1 score of the adversarial attack. In step 418, the processing device (e.g., the processing server 102) updates the one or more attack datasets (e.g., the attack data 109) to include the one or more adversarial attack success metrics.

[0123] In step 420, the processing device (e.g., the processing server 102 via the generation module 224) generates a vulnerability metric (e.g., a vulnerability metric 218) for each of the one or more updated attack datasets (e.g., the data 209). Each vulnerability metric (e.g., the vulnerability metrics 218) is a library of algorithms and models specific to a combination of one of the one or more adversarial attacks performed on one of the one or more data types included in the dataset (e.g., the data 208). For example, there can be one vulnerability metric (e.g., vulnerability metric 218) for a membership inference model inversion attack on tabular data, a vulnerability metric (e.g., vulnerability metric 218) for a property inference model inversion attack on tabular data, a vulnerability metric (e.g., vulnerability metric 218) for a reconstructionAttorney Docket No.690248.0002 / 1WO Page 105 of 189 model inversion attack on tabular data, a vulnerability metric (e.g., vulnerability metric 218) for a membership inference model inversion attack on natural language data, etc.

[0124] In embodiments, the method 400 can further include: receiving (e.g., by the receiving device 202) of the processing device (e.g., the processing server 102) a second dataset that includes at least one data type; preprocessing (e.g., via the analyzing module 222) by the processing device (e.g., the processing server 102) the second dataset using the one or more data preprocessing methods (e.g., the preprocessing algorithms 210) based on the at least one data type; performing, by the processing device (e.g., via the analyzing module 222 of the processing server 102) data featurization of the second dataset based on the at least one data type to identify one or more data features of the second dataset; identifying, by the processing device (e.g., the processing server 102) at least one vulnerability of the generated vulnerability metrics metric (e.g., of the vulnerability metrics 218), wherein the identified at least one vulnerability metric corresponds to the at least one data type of the second dataset and one of the one or more adversarial attacks; applying, by the processing device (e.g., the processing server 102) the identified at least one vulnerability metric to the second dataset; generating by the processing device (e.g., via the generation module 224 of the processing server 102) a dataset vulnerability report for the second dataset that includes a quantitative value of risk that the second dataset is susceptible to the identified one of the one or more adversarial attacks; and displaying by the processing device (e.g., the processing server 102), via a user interface, the dataset vulnerability report for the second dataset. In such embodiments, the identifying the at least one vulnerability metric of the generated vulnerabilityAttorney Docket No.690248.0002 / 1WO Page 106 of 189 metrics (e.g., the vulnerability metrics 218) can include: identifying (e.g., by the processing server 102) a plurality of vulnerability metrics of the generated vulnerability metrics (e.g., the vulnerability metrics 218), wherein each the identified plurality of vulnerability metrics (e.g., the vulnerability metrics 218) corresponds to the at least one data type of the second dataset and different one of the one or more adversarial attacks (e.g., the model attack algorithms 216). Further, in such embodiments, the method 400 can further include: generating, by the processing device (e.g., the generation module 224 of the processing server 102), a textual explanation of the quantitative value of risk for the second dataset that includes a nature of a vulnerability and a reason for the vulnerability. Further, in such embodiment, the dataset vulnerability report for the second dataset can include the textual explanation of the quantitative value of risk for the second dataset.

[0125] In embodiments, the method 400 can further include: the processing device (e.g., the processing server 102) storing, in the database (e.g., the database 206), one or more data hardening algorithms (e.g., the data hardening algorithms 219), where each of the one or more data hardening algorithms (e.g., the data hardening algorithms 219) is specific to a data type; identifying, by the processing device (e.g., the processing server 102), at least one of the one or more data hardening algorithms (e.g., the data hardening algorithms 219) based on the at least one data type included in the second dataset; applying, by the processing device (e.g., the processing server 102), the identified at least one data hardening algorithm to the second dataset; outputting, by the processing device (e.g., the processing server 102), a hardened second dataset that includes one or more changes to the dataset; applying, by theAttorney Docket No.690248.0002 / 1WO Page 107 of 189 processing device (e.g., the processing server 102), the at least one vulnerability metric identified for the second dataset to the hardened second dataset; and updating, by the processing device (e.g., the processing server 102), the dataset vulnerability report to include a quantitative value of risk that the hardened second dataset is susceptible to the identified one of the one or more adversarial attacks. In such embodiments, the method 400 can further include: generating, by the processing device (e.g., via the generation module 224 of the processing server 102), a textual explanation of the hardened second dataset and a textual explanation of the quantitative value of risk that the hardened second dataset that includes a summary of changes to the second dataset in generating the hardened second dataset and an explanation of how the quantitative value of risk has changed. The updated dataset vulnerability report for the hardened second dataset can include the textual explanation of the quantitative value of the hardened second dataset and the textual explanation of the quantitative value of risk that the hardened second dataset

[0126] In embodiments, the method 400 can further include displaying, by the processing device (e.g., the processing server 102) via the user interface, the one or more data preprocessing algorithms (e.g., the preprocessing algorithms 210) based on the one or more data types included in the dataset and the one or more data featurization algorithms (e.g., the data featurization algorithms 212) based on the one or more data types included in the dataset, wherein the one or more data preprocessing algorithms (e.g., the preprocessing algorithms 210) and the one or more data featurization algorithms (e.g., the data featurization algorithms 212) used on the dataset (e.g., the data 208) are selected by a user via the user interface.Attorney Docket No.690248.0002 / 1WO Page 108 of 189

[0127] In embodiments, the method 400 can further include displaying, by the processing device (e.g., the processing server 102) via the user interface, the one or more adversarial attacks (e.g., the model attack algorithms 216), and the one or more adversarial attacks (e.g., the model attack algorithms 216) executed on the one or more victim models using one or more adversarial attacks (e.g., the model attack algorithms 216) is selected by the user via the user interface.

[0128] In embodiments of the method 400 where the executed one or more adversarial attack (e.g., the model attack algorithms 216) is a membership inference model inversion attack, the executing the membership inference model inversion attack can include: generating (e.g., via the generation module 224 of the processing server 102) a model training dataset that is a subset of the dataset (e.g., the data 208); inputting (e.g., by the processing server 102) the model training dataset into the victim models, the victim models classifying the model training dataset; training (e.g., by the processing server 102) a plurality of shadow models based on a subset of the classified model training dataset; inputting (e.g., by the processing server 102) the model training dataset into the plurality of shadow models; receiving (e.g., by the processing server 102) as output from the plurality of shadow models an output probability vector; generating (e.g., via the generation module 224 of the processing server 102) an attack model dataset, the attack model dataset including the output probability vector, the model training dataset; and generating (e.g., via the generation module 224 of the processing server 102) a binary classification model using the attack model dataset, the binary classification model being configured to determine whether an observation was utilized in the training of the victim models.Attorney Docket No.690248.0002 / 1WO Page 109 of 189 Pseudocode for System Configuration Parameters

[0129] Table 1 include example pseudocode for configuration parameters of the system 10. / / ============ CONFIGURATION PARAMETERS ============ CONFIG = {Attorney Docket No.690248.0002 / 1WO Page 110 of 189 naive_bayes: { alpha: 1.0,Attorney Docket No.690248.0002 / 1WO Page 111 of 189 }, logistic_regression: { ' 'Pseudocode for Data Preprocessing

[0130] Table 2 includes example pseudocode for preprocessing datasets (e.g., the data 208) as discussed in more detail above. / / ============ DATA LOADING AND PREPROCESSING ============Attorney Docket No.690248.0002 / 1WO Page 112 of 189 IF CONFIG.encoding_method == 'onehot': encoder = OneHotEncoder() tPseudocode for Victim Model Attack

[0131] Table 3 includes example pseudocode for victim model attack as discussed in more detail above. / / ============ CREATE VICTIM AND SHADOW DATASETS ============Attorney Docket No.690248.0002 / 1WO Page 113 of 189 victim_dataset = dataset[victim_indices]Attorney Docket No.690248.0002 / 1WO Page 114 of 189 nonvictim_dataset, victim_type,Attorney Docket No.690248.0002 / 1WO Page 115 of 189 IF CONFIG.verbose: " "Attorney Docket No.690248.0002 / 1WO Page 116 of 189 IF CONFIG.verbose:Attorney Docket No.690248.0002 / 1WO Page 117 of 189 test_results = { 'predictions': test_preds, ' 'Attorney Docket No.690248.0002 / 1WO Page 118 of 189 / / Evaluate on actual victim data victim_preds = attack_model.PREDICT(victim_test_data) ' ' )Attorney Docket No.690248.0002 / 1WO Page 119 of 189 FUNCTION GET_ATTACK_TYPES(victim_type):Attorney Docket No.690248.0002 / 1WO Page 120 of 189 combination_mapping = GENERATE_ATTACK_SCENARIO_MAPPING(victim_models)Pseudocode for Analyzing Data Using Vulnerability Metric

[0132] Table 4 includes example pseudocode for analyzing a dataset (e.g., a dataset provided by the user of the user computing device 104A) including tabular data against a membership inference model inversion attack as discussed in more detail above. # Unified Membership Inference Vulnerability Assessment FrameworkAttorney Docket No.690248.0002 / 1WO Page 121 of 189 ``` / / ============ SYSTEM CONFIGURATION ============ ] ', e ',Attorney Docket No.690248.0002 / 1WO Page 122 of 189 WHERE methods encompass ANY threshold determination approach INCLUDING BUT NOT LIMITED TO statistical measures, adaptive algorithms, ',Attorney Docket No.690248.0002 / 1WO Page 123 of 189 scaling_parameters: 'SCALING_SPECIFIC_PARAMETERS', filtering_criteria: 'DATA_CLEANING_SPECIFICATIONS', ' ' ,Attorney Docket No.690248.0002 / 1WO Page 124 of 189 } WHERE strategy encompasses ANY class distribution adjustment methodology , ,Attorney Docket No.690248.0002 / 1WO Page 125 of 189 AND supports dynamic adaptation to available resources and analysis mode requirements,Attorney Docket No.690248.0002 / 1WO Page 126 of 189 WHERE framework supports dynamic addition of ANY new methodologies AND maintains backward compatibility with existing configurations ', nAttorney Docket No.690248.0002 / 1WO Page 127 of 189 filtered_data = APPLY_DATA_FILTERING( primary_data,Attorney Docket No.690248.0002 / 1WO Page 128 of 189 CONFIG.data_partitioning.disparate_strategy, CONFIG.data_partitioning.disparate_parametersAttorney Docket No.690248.0002 / 1WO Page 129 of 189 ) dAttorney Docket No.690248.0002 / 1WO Page 130 of 189 INCLUDING BUT NOT LIMITED TO binary vulnerability classification, multi- class labeling, s,Attorney Docket No.690248.0002 / 1WO Page 131 of 189 )Attorney Docket No.690248.0002 / 1WO Page 132 of 189 / / ============ DISPARATE IMBALANCE MITIGATION ============ t _ eAttorney Docket No.690248.0002 / 1WO Page 133 of 189 temporal validation, nested validation, and custom evaluation protocols, e eAttorney Docket No.690248.0002 / 1WO Page 134 of 189 OUTPUT_DISPARATE_VALIDATION_RESULTS(disparate_validation_results) e t s)Attorney Docket No.690248.0002 / 1WO Page 135 of 189 IF CONFIG.analysis_mode.type == 'combined': unified_validation_results = = s yAttorney Docket No.690248.0002 / 1WO Page 136 of 189 / / Perform unified distributional analysis IF CONFIG.analysis_mode.macro_analysis_enabled: a E dAttorney Docket No.690248.0002 / 1WO Page 137 of 189 macro_optimized_features IF CONFIG.analysis_mode.macro_analysis_enabled ELSE NULL, =Attorney Docket No.690248.0002 / 1WO Page 138 of 189 modeling_configuration.system_parameters ) c ,Attorney Docket No.690248.0002 / 1WO Page 139 of 189 resilient learning, adaptive learning, dynamic learning, context-aware learning, personalized learning, individualized learning, customized learning, or , , , t ,Attorney Docket No.690248.0002 / 1WO Page 140 of 189 forward models, inverse models, direct models, indirect models, explicit models, implicit models, generative models, discriminative models, s, , _ al g,Attorney Docket No.690248.0002 / 1WO Page 141 of 189 dimensionality reduction, data augmentation, synthetic data generation, data balancing, data sampling, data partitioning, data splitting, , n, g,Attorney Docket No.690248.0002 / 1WO Page 142 of 189 UTILIZING ANY computational resource allocation, management, or optimization strategy ,Attorney Docket No.690248.0002 / 1WO Page 143 of 189 biological computers, molecular computers, DNA computers, protein computers, ,Attorney Docket No.690248.0002 / 1WO Page 144 of 189 optimized decisions, detected anomalies, discovered clusters, reduced dimensions, d d sAttorney Docket No.690248.0002 / 1WO Page 145 of 189 artistic creation, creative expression, aesthetic evaluation, cultural analysis, linguistic processing, semantic understanding, pragmatic reasoning, discourse , ,Attorney Docket No.690248.0002 / 1WO Page 146 of 189 surveillance systems, detection systems, recognition systems, identification systems, ,Attorney Docket No.690248.0002 / 1WO Page 147 of 189 competitive users, cooperative users, individual decision makers, group decision makers, n , s,Pseudocode for Data Hardening

[0133] Table 5 includes example pseudocode for hardening a tabular dataset (e.g., the dataset provided by the user of the user computing device 104) against a membership inference model inversion attack as discussed in more detail above. # Comprehensive Privacy-Preserving Hardening FrameworkAttorney Docket No.690248.0002 / 1WO Page 148 of 189 / / Data source and processing configuration data_sources: { ' ' or - eAttorney Docket No.690248.0002 / 1WO Page 149 of 189 feature_interaction_analysis: 'CONFIGURABLE_FEATURE_DEPENDENCY_ASSESSMENT', ,Attorney Docket No.690248.0002 / 1WO Page 150 of 189 quality_preservation_constraints: 'CONFIGURABLE_DATA_INTEGRITY_MAINTENANCE_PARAMETERS'Attorney Docket No.690248.0002 / 1WO Page 151 of 189 WHERE noise_type encompasses ANY statistical perturbation methodology INCLUDING BUT NOT LIMITED TO Gaussian noise, Laplacian noise,Attorney Docket No.690248.0002 / 1WO Page 152 of 189 shadow_model_specifications: 'CONFIGURABLE_SURROGATE_MODEL_PARAMETERS', S'Attorney Docket No.690248.0002 / 1WO Page 153 of 189 victim_model_metrics: 'CONFIGURABLE_TARGET_MODEL_PERFORMANCE_CRITERIA', =Attorney Docket No.690248.0002 / 1WO Page 154 of 189 CONFIG.preprocessing_framework ) ntAttorney Docket No.690248.0002 / 1WO Page 155 of 189 / / Create victim model training datasetAttorney Docket No.690248.0002 / 1WO Page 156 of 189 )Attorney Docket No.690248.0002 / 1WO Page 157 of 189 / / Generate victim model prediction vectors for attack evaluation victim_prediction_vectors =Attorney Docket No.690248.0002 / 1WO Page 158 of 189 / / Generate comprehensive vulnerability assessment vulnerability_assessment =Attorney Docket No.690248.0002 / 1WO Page 159 of 189 / / ============ COMPREHENSIVE HARDENING FUNCTIONS ============ ,Attorney Docket No.690248.0002 / 1WO Page 160 of 189 / / Apply noise injection if specified " "Attorney Docket No.690248.0002 / 1WO Page 161 of 189 / / Compute feature correlation matrix using specified method ,Attorney Docket No.690248.0002 / 1WO Page 162 of 189 INCLUDING BUT NOT LIMITED TO CTGAN, Variational Autoencoders, ADASYN, SMOTE,Attorney Docket No.690248.0002 / 1WO Page 163 of 189 AND custom quality metrics, :Attorney Docket No.690248.0002 / 1WO Page 164 of 189 manifold_config.optimal_dimension_selection, manifold_config.reconstruction_quality_criteriaAttorney Docket No.690248.0002 / 1WO Page 165 of 189 dataset.features.shape, noise_config.noise_type, :Attorney Docket No.690248.0002 / 1WO Page 166 of 189 INCLUDING BUT NOT LIMITED TO Laplace mechanism, Gaussian mechanism, s, ( nAttorney Docket No.690248.0002 / 1WO Page 167 of 189 INCLUDING BUT NOT LIMITED TO classification accuracy preservation, statistical property preservation, mAttorney Docket No.690248.0002 / 1WO Page 168 of 189 / / Evaluate class distribution maintenanceAttorney Docket No.690248.0002 / 1WO Page 169 of 189 Computer System Architecture

[0134] FIG.5 illustrates a computer system 500 in which embodiments of the present disclosure, or portions thereof, can be implemented as computer-readable code. For example, the processing server 102, payment systems 104a and 104b, external data source 106, and third party data provider 108 can be implemented in the computer system 500 using hardware, non-transitory computer readable media having instructions stored thereon, or a combination thereof and can be implemented in one or more computer systems or other processing systems. Hardware can embody modules and components used to implement the methods of FIGS.3A, 3B, 4A, and 4B.

[0135] If programmable logic is used, such logic can execute on a commercially available processing platform configured by executable software code to become a specific purpose computer or a special purpose device (e.g., programmable logic array, application-specific integrated circuit, etc.). A person having ordinary skill in the art can appreciate that embodiments of the disclosed subject matter can be practiced with various computer system configurations, including multi-core multiprocessor systems, minicomputers, mainframe computers, computers linked or clustered with distributed functions, as well as pervasive or miniature computers that can be embedded into virtually any device. For instance, at least one processor device and a memory can be used to implement the above described embodiments.

[0136] A processor unit or device as discussed herein can be a single processor, a plurality of processors, or combinations thereof. Processor devices can have one or more processor “cores.” The terms “computer program medium,” “non-transitoryAttorney Docket No.690248.0002 / 1WO Page 170 of 189 computer readable medium,” and “computer usable medium” as discussed herein are used to generally refer to tangible media such as a removable storage unit 518, a removable storage unit 522, and a hard disk installed in hard disk drive 512.

[0137] Various embodiments of the present disclosure are described in terms of this example computer system 500. After reading this description, it will become apparent to a person skilled in the relevant art how to implement the present disclosure using other computer systems and / or computer architectures. Although operations can be described as a sequential process, some of the operations can in fact be performed in parallel, concurrently, and / or in a distributed environment, and with program code stored locally or remotely for access by single or multi-processor machines. In addition, in some embodiments the order of operations can be rearranged without departing from the spirit of the disclosed subject matter.

[0138] Processor device 504 can be a special purpose or a general purpose processor device specifically configured to perform the functions discussed herein. The processor device 504 can be connected to a communications infrastructure 506, such as a bus, message queue, network, multi-core message-passing scheme, etc. The network can be any network suitable for performing the functions as disclosed herein and can include a local area network (LAN), a wide area network (WAN), a wireless network (e.g., WiFi), a mobile communication network, a satellite network, the Internet, fiber optic, coaxial cable, infrared, radio frequency (RF), or any combination thereof. Other suitable network types and configurations will be apparent to persons having skill in the relevant art. The computer system 500 can also include a main memory 508 (e.g., random access memory, read-only memory, etc.), and can alsoAttorney Docket No.690248.0002 / 1WO Page 171 of 189 include a secondary memory 510. The secondary memory 510 can include the hard disk drive 512 and a removable storage drive 514, such as a floppy disk drive, a magnetic tape drive, an optical disk drive, a flash memory, etc.

[0139] The removable storage drive 514 can read from and / or write to the removable storage unit 518 in a well-known manner. The removable storage unit 518 can include a removable storage media that can be read by and written to by the removable storage drive 514. For example, if the removable storage drive 514 is a floppy disk drive or universal serial bus port, the removable storage unit 518 can be a floppy disk or portable flash drive, respectively. In one embodiment, the removable storage unit 518 can be non-transitory computer readable recording media.

[0140] In some embodiments, the secondary memory 510 can include alternative means for allowing computer programs or other instructions to be loaded into the computer system 500, for example, the removable storage unit 522 and an interface 520. Examples of such means can include a program cartridge and cartridge interface (e.g., as found in video game systems), a removable memory chip (e.g., EEPROM, PROM, etc.) and associated socket, and other removable storage units 522 and interfaces 520 as will be apparent to persons having skill in the relevant art.

[0141] Data stored in the computer system 500 (e.g., in the main memory 508 and / or the secondary memory 510) can be stored on any type of suitable computer readable media, such as optical storage (e.g., a compact disc, digital versatile disc, Blu-ray disc, etc.) or magnetic tape storage (e.g., a hard disk drive). The data can be configured in any type of suitable database configuration, such as a relational database, a structured query language (SQL) database, a distributed database, anAttorney Docket No.690248.0002 / 1WO Page 172 of 189 object database, etc. Suitable configurations and storage types will be apparent to persons having skill in the relevant art.

[0142] The computer system 500 can also include a communications interface 524. The communications interface 524 can be configured to allow software and data to be transferred between the computer system 500 and external devices. Exemplary communications interfaces 524 can include a modem, a network interface (e.g., an Ethernet card), a communications port, a PCMCIA slot and card, etc. Software and data transferred via the communications interface 524 can be in the form of signals, which can be electronic, electromagnetic, optical, or other signals as will be apparent to persons having skill in the relevant art. The signals can travel via a communications path 526, which can be configured to carry the signals and can be implemented using wire, cable, fiber optics, a phone line, a cellular phone link, a radio frequency link, etc.

[0143] The computer system 500 can further include a display interface 502. The display interface 502 can be configured to allow data to be transferred between the computer system 500 and external display 530. Exemplary display interfaces 502 can include high-definition multimedia interface (HDMI), digital visual interface (DVI), video graphics array (VGA), etc. The display 530 can be any suitable type of display for displaying data transmitted via the display interface 502 of the computer system 500, including a cathode ray tube (CRT) display, liquid crystal display (LCD), light-emitting diode (LED) display, capacitive touch display, thin-film transistor (TFT) display, etc.

[0144] Computer program medium and computer usable medium can refer to memories, such as the main memory 508 and secondary memory 510, which can be memory semiconductors (e.g., DRAMs, etc.). These computer program products canAttorney Docket No.690248.0002 / 1WO Page 173 of 189 be means for providing software to the computer system 500. Computer programs (e.g., computer control logic) can be stored in the main memory 508 and / or the secondary memory 510. Computer programs can also be received via the communications interface 524. Such computer programs, when executed, can enable computer system 500 to implement the present methods as discussed herein. In particular, the computer programs, when executed, can enable processor device 504 to implement the methods illustrated by FIGS.3A, 3B, 4A, and 4B, as discussed herein. Accordingly, such computer programs can represent controllers of the computer system 500. Where the present disclosure is implemented using software, the software can be stored in a computer program product and loaded into the computer system 500 using the removable storage drive 514, interface 520, and hard disk drive 512, or communications interface 524.

[0145] The processor device 504 can comprise one or more modules or engines configured to perform the functions of the computer system 500. Each of the modules or engines can be implemented using hardware and, in some instances, can also utilize software, such as corresponding to program code and / or programs stored in the main memory 508 or secondary memory 510. In such instances, program code can be compiled by the processor device 504 (e.g., by a compiling module or engine) prior to execution by the hardware of the computer system 500. For example, the program code can be source code written in a programming language that is translated into a lower level language, such as assembly language or machine code, for execution by the processor device 504 and / or any additional hardware components of the computer system 500. The process of compiling can include the use of lexical analysis,Attorney Docket No.690248.0002 / 1WO Page 174 of 189 preprocessing, parsing, semantic analysis, syntax-directed translation, code generation, code optimization, and any other techniques that can be suitable for translation of program code into a lower level language suitable for controlling the computer system 500 to perform the functions disclosed herein. It will be apparent to persons having skill in the relevant art that such processes result in the computer system 500 being a specially configured computer system 500 uniquely programmed to perform the functions discussed above.

[0146] Techniques consistent with the present disclosure provide, among other features, systems and methods for analyzing datasets for vulnerabilities to adversarial attacks. While various exemplary embodiments of the disclosed system and method have been described above it should be understood that they have been presented for purposes of example only, not limitations. It is not exhaustive and does not limit the disclosure to the precise form disclosed. Modifications and variations are possible in light of the above teachings or can be acquired from practicing of the disclosure, without departing from the breadth or scope.

Claims

Attorney Docket No.690248.0002 / 1WO Page 175 of 189 What is claimed is:

1. A method for analyzing datasets for vulnerabilities to adversarial attacks, the method comprising: storing, by a processing server in a database, a dataset, the dataset including data of one or more data types; preprocessing, by the processing server, the dataset using one or more data preprocessing algorithms based on the one or more data types included in the dataset; performing, by the processing server, data featurization of the dataset based on the one or more data types included in the dataset using one or more data featurization algorithms, the data featurization identifying one or more data features of the dataset; generating, by the processing server, one or more victim models using the dataset based on the one or more data types included in the dataset, wherein each of the one or more victim models is specific to one of the one or more data types included in the dataset; determining, by the processing server, one or more performance metrics of the one or more victim models; generating, by the processing server, one or more attack datasets, wherein each of the one or more attack datasets corresponds to one of the one or more data types included in the dataset, and wherein each of the one or more attack datasets includes at least the one or more performance metrics of the one or more victim models, and the one or more data features of the dataset;Attorney Docket No.690248.0002 / 1WO Page 176 of 189 executing, by the processing server, one or more adversarial attacks on the one or more victim models, wherein the one or more adversarial attacks include one or more of: a membership inference model inversion attack, a property inference model inversion attack, and a reconstruction model inversion attack; determining, by the processing server, one or more adversarial attack success metrics, the one or more adversarial attack success metrics including one or more of: an accuracy of the adversarial attack, a precision of the adversarial attack, and an F1 score of the adversarial attack; updating, by the processing server, the one or more attack datasets to include the one or more adversarial attack success metrics; and generating, by the processing server, a vulnerability metric for each of the one or more updated attack datasets, wherein each vulnerability metric is a library of algorithms and models specific to a combination of one of the one or more adversarial attacks performed on one of the one or more data types included in the dataset.

2. The method of claim 1, further comprising: receiving, by the processing server from a user computing device, a second dataset, the second dataset including at least one data type; preprocessing, by the processing server, the second dataset using the one or more data preprocessing methods based on the at least one data type; performing, by the processing server, data featurization of the second dataset based on the at least one data type to identify one or more data features of the second dataset;Attorney Docket No.690248.0002 / 1WO Page 177 of 189 identifying, by the processing server, at least one vulnerability metric of the generated vulnerability metrics, wherein the identified at least one vulnerability metric corresponds to the at least one data type of the second dataset and one of the one or more adversarial attacks; applying, by the processing server, the identified at least one vulnerability metric to the second dataset; generating, by the processing server, a dataset vulnerability report for the second dataset, the dataset vulnerability report including a quantitative value of risk that the second dataset is susceptible to the identified one of the one or more adversarial attacks; and displaying, by the processing server via a user interface, dataset vulnerability report for the second dataset.

3. The method of claim 2, wherein identifying at least one vulnerability metric of the generated vulnerability metrics includes: identifying a plurality of vulnerability metrics of the generated vulnerability metrics, wherein each the identified plurality of vulnerability metrics corresponds to the at least one data type of the second dataset and different one of the one or more adversarial attacks.

4. The method of claim 2, further comprising:Attorney Docket No.690248.0002 / 1WO Page 178 of 189 storing, by the processing server in the database, one or more data hardening algorithms, wherein each of the one or more data hardening algorithms is specific to a data type; identifying, by the processing server, at least one of the one or more data hardening algorithms based on the at least one data type included in the second dataset; applying, by the processing server, the identified at least one data hardening algorithm to the second dataset; outputting, by the processing server, a hardened second dataset, the hardened second dataset including one or more changes to the dataset; applying, by the processing server, the at least one vulnerability metric identified for the second dataset to the hardened second dataset; and updating, by the processing server, the dataset vulnerability report to include a quantitative value of risk that the hardened second dataset is susceptible to the identified one of the one or more adversarial attacks.

5. The method of claim 1, wherein the data types are one or more of: tabular data, natural language data, computer vision data, and generative artificial intelligence (AI) data.

6. The method of claim 1, wherein the one or more data preprocessing methods includes one or more of: a data cleaning algorithm, a feature scaling and transformation algorithm, a feature engineering algorithm, a text cleaning algorithm, aAttorney Docket No.690248.0002 / 1WO Page 179 of 189 tokenization and normalization algorithm, a stopword and noise removal algorithm, a text representation algorithm, a miscellaneous NLP-specific algorithm, a basic image cleaning algorithm, an image resizing and cropping algorithm, a pixel normalization algorithm, a noise reduction algorithm, and a data augmentation algorithm.

7. The method of claim 2, further comprising: displaying, by the processing server via the user interface, the one or more data preprocessing algorithms based on the one or more data types included in the dataset and the one or more data featurization algorithms based on the one or more data types included in the dataset, wherein the one or more data preprocessing algorithms and the one or more data featurization algorithms used on the dataset are selected by a user via the user interface.

8. The method of claim 2, further comprising: displaying, by the processing server via the user interface, the one or more adversarial attacks, wherein the one or more adversarial attacks executed on the one or more victim models using one or more adversarial attacks is selected by the user via the user interface.Attorney Docket No.690248.0002 / 1WO Page 180 of 189 9. The method of claim 1, wherein the executed one or more adversarial attack is a membership inference model inversion attack, wherein executing the membership inference model inversion attack includes: generating a model training dataset, the model training dataset being a subset of the dataset; inputting the model training dataset into the victim models, the victim models classifying the model training dataset; training a plurality of shadow models based on a subset of the classified model training dataset; inputting the model training dataset into the plurality of shadow models; receiving, as output from the plurality of shadow models an output probability vector; generating an attack model dataset, the attack model dataset including the output probability vector, the model training dataset; and generating a classification model using the attack model dataset, the classification model being configured to determine whether an observation was utilized in the training of the victim models.

10. The method of claim 2, the method further comprising: generating, by the processing server, a textual explanation of the quantitative value of risk for the second dataset, the textual explanation including a nature of a vulnerability and a reason for the vulnerability,Attorney Docket No.690248.0002 / 1WO Page 181 of 189 wherein the dataset vulnerability report for the second dataset includes the textual explanation of the quantitative value of risk for the second dataset.

11. The method of claim 4, the method comprising: generating, by the processing server, a textual explanation of the hardened second dataset and a textual explanation of the quantitative value of risk that the hardened second dataset, the textual explanation including a summary of changes to the second dataset in generating the hardened second dataset and an explanation of how the quantitative value of risk has changed, wherein the updated dataset vulnerability report for the hardened second dataset includes the textual explanation of the quantitative value of the hardened second dataset and the textual explanation of the quantitative value of risk that the hardened second dataset.

12. A system method for analyzing datasets for vulnerabilities to adversarial attacks, the system comprising a processing server including: a memory configured to store a dataset, the dataset including data of one or more data types; and a processor configured to: preprocess the dataset using one or more data preprocessing algorithms based on the one or more data types included in the dataset;Attorney Docket No.690248.0002 / 1WO Page 182 of 189 perform data featurization of the dataset based on the one or more data types included in the dataset using one or more data featurization algorithms, the data featurization identifying one or more data features of the dataset; generate one or more victim models using the dataset based on the one or more data types included in the dataset, wherein each of the one or more victim models is specific to one of the one or more data types included in the dataset; determine one or more performance metrics of the one or more victim models; generate one or more attack datasets, wherein each of the one or more attack datasets corresponds to one of the one or more data types included in the dataset, and wherein each of the one or more attack datasets includes at least the one or more performance metrics of the one or more victim models, and the one or more data features of the dataset; execute one or more adversarial attacks on the one or more victim models, wherein the one or more adversarial attacks include one or more of: a membership inference model inversion attack, a property inference model inversion attack, and a reconstruction model inversion attack; determine one or more adversarial attack success metrics, the one or more adversarial attack success metrics including one or more of: an accuracy of the adversarial attack, a precision of the adversarial attack, and an F1 score of the adversarial attack;Attorney Docket No.690248.0002 / 1WO Page 183 of 189 update the one or more attack datasets to include the one or more adversarial attack success metrics; and generate vulnerability metric for each of the one or more updated attack datasets, wherein each vulnerability metric is a library of algorithms and models specific to a combination of one of the one or more adversarial attacks performed on one of the one or more data types included in the dataset.

13. The system of claim 12, further comprising: a user computing device; a user interface; and a receiving device of the processing server configured to receive, from a user computing device, a second dataset, the second dataset including at least one data type; wherein the processor is further configured to: preprocess the second dataset using the one or more data preprocessing methods based on the at least one data type; perform data featurization of the second dataset based on the at least one data type to identify one or more data features of the second dataset; identify at least one vulnerability metric of the generated vulnerability metrics, wherein the identified at least one vulnerability metric corresponds to the at least one data type of the second dataset and one of the one or more adversarial attacks;Attorney Docket No.690248.0002 / 1WO Page 184 of 189 apply the identified at least one vulnerability metric to the second dataset; generate a dataset vulnerability report for the second dataset, the dataset vulnerability report including a quantitative value of risk that the second dataset is susceptible to the identified one of the one or more adversarial attacks; and display, via a user interface, dataset vulnerability report for the second dataset.

14. The system of claim 13, wherein identifying at least one vulnerability metric of the generated vulnerability metrics includes: the processor configured to identify a plurality of vulnerability metrics of the generated vulnerability metrics, wherein each the identified plurality of vulnerability metrics corresponds to the at least one data type of the second dataset and different one of the one or more adversarial attacks.

15. The system of claim 13, wherein the database is further configured to store one or more data hardening algorithms, wherein each of the one or more data hardening algorithms is specific to a data type, wherein the processor is further configured to: identify at least one of the one or more data hardening algorithms based on the at least one data type included in the second dataset;Attorney Docket No.690248.0002 / 1WO Page 185 of 189 apply the identified at least one data hardening algorithm to the second dataset; output a hardened second dataset, the hardened second dataset including one or more changes to the dataset; apply the at least one vulnerability metric identified for the second dataset to the hardened second dataset; and update the dataset vulnerability report to include a quantitative value of risk that the hardened second dataset is susceptible to the identified one of the one or more adversarial attacks.

16. The system of claim 12, wherein the data types are one or more of: tabular data, natural language data, computer vision data, and generative artificial intelligence (AI) data.

17. The system of claim 12, wherein the one or more data preprocessing methods includes one or more of: a data cleaning algorithm, a feature scaling and transformation algorithm, a feature engineering algorithm, a text cleaning algorithm, a tokenization and normalization algorithm, a stopword and noise removal algorithm, a text representation algorithm, a miscellaneous NLP-specific algorithm, a basic image cleaning algorithm, an image resizing and cropping algorithm, a pixel normalization algorithm, a noise reduction algorithm, and a data augmentation algorithm.

18. The system of claim 13, wherein the processor is further configured to:Attorney Docket No.690248.0002 / 1WO Page 186 of 189 display, via the user interface, the one or more data preprocessing algorithms based on the one or more data types included in the dataset and the one or more data featurization algorithms based on the one or more data types included in the dataset, wherein the one or more data preprocessing algorithms and the one or more data featurization algorithms used on the dataset are selected by a user via the user interface.

19. The system of claim 13, wherein the processor is further configured to: display, via the user interface, the one or more adversarial attacks, wherein the one or more adversarial attacks executed on the one or more victim models using one or more adversarial attacks is selected by the user via the user interface.

20. The system of claim 12, wherein the executed one or more adversarial attack is a membership inference model inversion attack, wherein executing the membership inference model inversion attack includes the processor is further configured to: generate a model training dataset, the model training dataset being a subset of the dataset; input the model training dataset into the victim models, the victim models classifying the model training dataset; train a plurality of shadow models based on a subset of the classified model training dataset;Attorney Docket No.690248.0002 / 1WO Page 187 of 189 input the model training dataset into the plurality of shadow models; receive as output from the plurality of shadow models an output probability vector; generate an attack model dataset, the attack model dataset including the output probability vector, the model training dataset; and generate a binary classification model using the attack model dataset, the binary classification model being configured to determine whether an observation was utilized in the training of the victim models.

21. The system of claim 13, wherein the processor is further configured to: generate a textual explanation of the quantitative value of risk for the second dataset, the textual explanation including a nature of a vulnerability and a reason for the vulnerability, wherein the dataset vulnerability report for the second dataset includes the textual explanation of the quantitative value of risk for the second dataset.

22. The system of claim 15, wherein the processor is further configured to: generate a textual explanation of the hardened second dataset and a textual explanation of the quantitative value of risk that the hardened second dataset, the textual explanation including a summary of changes to the second dataset in generating the hardened second dataset and an explanation of how the quantitative value of risk has changed,Attorney Docket No.690248.0002 / 1WO Page 188 of 189 wherein the updated dataset vulnerability report for the hardened second dataset includes the textual explanation of the quantitative value of the hardened second dataset and the textual explanation of the quantitative value of risk that the hardened second dataset.

Images