Model detoxification training method and device based on data poisoning and electronic device

Abstract

The invention relates to a model detoxification training method and device based on data poisoning and an electronic device. The method comprises the steps of obtaining original training data, and performing preprocessing operation on the original training data to obtain a pure initial data set; on the basis of a basic training model, performing tiny disturbance on the pure initial data set, and generating a confrontation sample set for the basic training model; based on an injection strategy, injecting the adversarial sample set into the pure initial data set to obtain a mixed training set, and performing model training on the basic training model based on the mixed training set to obtain an anti-interference enhancement model; acquiring a training key parameter of the anti-interference enhancement model, and comparing the training key parameter with a preset safety threshold to obtain an abnormal identifier; and based on the abnormal identifier, correcting the anti-interference enhancement model to obtain a detoxification training model. By adopting the method, the poisoning data can be actively identified, and the safety is improved.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence security technology, and in particular relates to a model detoxification training method, device and electronic device based on data poisoning. Background Technology

[0002] With the in-depth development of artificial intelligence security technology, specialized defense technologies have emerged to address model data poisoning attacks. These technologies aim to systematically address the model security risks caused by malicious contamination of training data. This technological field emphasizes end-to-end protection from the training data input to the model application output, featuring multi-stage and proactive characteristics, thus giving rise to the current mainstream comprehensive detoxification solutions. Traditional defense models typically employ isolated and reactive approaches. For example, they mainly rely on static data cleaning before training to filter outliers, or on simple output threshold monitoring after the model goes live to intercept obvious errors. This model treats security protection as an independent, phased task, failing to deeply embed defense mechanisms into the core processes of model training and inference. However, current traditional protection methods have significant shortcomings. Passive response mechanisms are ill-equipped to handle advanced and covert data poisoning techniques, such as attacks that inject bundled real and fake data, causing defense to lag behind attack evolution; static and isolated defense points cannot form a coordinated security system, failing to cover the entire lifecycle from data preparation and model training to online inference, leaving many exploitable security blind spots. These problems severely restrict the reliable application of artificial intelligence systems in high-risk scenarios. Summary of the Invention

[0003] Therefore, it is necessary to provide a data-based poisoning model detoxification training method, device, and electronic device that can actively counteract the aforementioned technical problems.

[0004] Firstly, this application provides a model detoxification training method based on data poisoning, including:

[0005] Obtain the raw training data and perform preprocessing operations on the raw training data to obtain a clean initial dataset;

[0006] Based on the basic trained model, a set of adversarial examples against the basic trained model is generated by making slight perturbations to the pure initial dataset.

[0007] Based on the injection strategy, the adversarial sample set is injected into the pure initial dataset to obtain a hybrid training set. The basic training model is then trained based on the hybrid training set to obtain an anti-interference enhancement model.

[0008] Obtain the key training parameters of the anti-interference enhancement model, and compare the key training parameters with the preset safety threshold to obtain anomaly indicators;

[0009] Based on the anomaly identifier, the anti-interference enhancement model is modified to obtain the detoxification training model.

[0010] Furthermore, based on the base trained model, a small perturbation is applied to the clean initial dataset to generate a set of adversarial examples against the base trained model, including:

[0011] Extract a batch of clean samples from the clean initial dataset, and input the clean samples into the base training model of the training mode to calculate the gradient of the loss function and obtain the initial gradient direction;

[0012] Based on the gradient direction, a small adversarial perturbation is added to the clean sample to generate candidate adversarial samples;

[0013] The candidate adversarial samples are input into the basic training model for effectiveness verification to obtain effective adversarial samples;

[0014] Based on the clean samples, effective adversarial samples with differences greater than a preset difference threshold are removed from the effective adversarial samples to obtain high-quality adversarial samples.

[0015] By integrating high-quality adversarial samples corresponding to the clean samples in each batch, an adversarial sample set is obtained.

[0016] Furthermore, based on the mixed training set, the basic training model is trained to obtain an anti-interference enhancement model, including:

[0017] Based on the mixed training set, the total loss of the base training model is calculated using the following formula:

[0018]

[0019] in, Where N is the total loss, N is the batch size, and i is the sample index. For clean samples, For adversarial examples, For real labels, For model parameters, For model prediction, Let cross-entropy be the loss function. To counteract the loss of weight;

[0020] Backpropagation of the total loss of the basic training model yields the parameter update model. Based on the validation set, the robustness of the parameter update model is evaluated, and a robustness report is obtained.

[0021] Based on the anti-interference report, the injection strategy is updated to obtain the updated injection strategy; the updated injection strategy is used to update the hybrid training set.

[0022] The parameter update model is trained based on a hybrid training set to obtain an anti-interference enhancement model.

[0023] Furthermore, based on anomaly identification, the anti-interference enhancement model is modified to obtain the detoxification training model, which also includes:

[0024] Based on different data dimensions, the pure initial dataset is sliced ​​to obtain data slices; the data dimensions include at least one of the following: data source, time window, user attributes, and content features.

[0025] Based on the detoxification training model and data slices, key performance indicators are calculated to obtain a detailed performance evaluation report for each data slice.

[0026] Based on the detailed performance evaluation report, identify the data slices where key performance indicators are abnormal, and obtain the abnormal data slices.

[0027] Based on the abnormal data slices, a suspicious data slice location report is generated, and based on the suspicious data slice location report, the poisoned samples in the pure initial dataset are removed to obtain the purified training dataset;

[0028] Based on the purified training dataset, the detoxification training model is fine-tuned to obtain the detoxification model.

[0029] Furthermore, based on the suspicious data slice location report, poisoned samples are removed from the clean initial dataset to obtain the purified training dataset, including:

[0030] Based on suspicious data slice reports and anomaly detection algorithms, poisoning samples are identified for each anomalous data slice to obtain a list of anomalous samples.

[0031] Based on the content moderation model, each poisoning sample in the abnormal sample list is evaluated from multiple perspectives to obtain the moderation results;

[0032] Based on the audit results, confirmed poisoning samples and low-quality samples are generated; and confirmed poisoning samples and low-quality samples are removed from the clean initial dataset to obtain the remaining samples.

[0033] The remaining samples are then subjected to an integrity check to obtain a cleaned training dataset.

[0034] Furthermore, based on the purified training dataset, the detoxification training model is fine-tuned to obtain the detoxification model, which also includes:

[0035] Receive real-time request data and perform preprocessing operations on the real-time request data to obtain standard real-time data;

[0036] Standard real-time data is input into the detoxification model to obtain preliminary inference results;

[0037] By integrating preliminary reasoning results, standard real-time data, and pre-defined domain knowledge, a verification context is obtained; the domain knowledge includes security rules and logical constraints.

[0038] Based on the preset evaluation instructions, the verification context is input into the large language model in the security environment to obtain the auxiliary process verification conclusion; the auxiliary process verification conclusion includes at least one of pass, suspicious, and rejection;

[0039] Based on the verification results of the auxiliary process, the confidence level of the corresponding preliminary reasoning results is calculated to obtain a risk report.

[0040] Secondly, this application also provides a data-based poisoning model detoxification training device, comprising:

[0041] The clean module is used to acquire the raw training data and perform preprocessing operations on the raw training data to obtain a clean initial dataset;

[0042] The perturbation module is used to apply small perturbations to the clean initial dataset based on the base trained model, generating a set of adversarial examples against the base trained model.

[0043] The training module is used to inject the adversarial sample set into the clean initial dataset based on the injection strategy to obtain a hybrid training set, and to train the basic training model based on the hybrid training set to obtain an anti-interference enhancement model.

[0044] The comparison module is used to obtain the key training parameters of the anti-interference enhancement model, and compare the key training parameters with the preset safety threshold to obtain anomaly indicators;

[0045] The correction module is used to correct the anti-interference enhancement model based on anomaly identifiers to obtain the detoxification training model.

[0046] Thirdly, this application also provides an electronic device including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement any step of the method provided in the first aspect of this application.

[0047] Fourthly, this application also provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements any step of the method provided in the first aspect of this application.

[0048] The aforementioned data poisoning-based model detoxification training method, apparatus, and electronic device acquire raw training data and preprocess it to obtain a clean initial dataset. Based on the basic training model, a small perturbation is applied to the clean initial dataset to generate an adversarial example set against the basic training model. Using an injection strategy, the adversarial example set is injected into the clean initial dataset to obtain a hybrid training set. The basic training model is then trained on this hybrid training set to obtain an anti-interference enhanced model. Key training parameters of the anti-interference enhanced model are acquired and compared with preset safety thresholds to obtain anomaly indicators. Based on these anomaly indicators, the anti-interference enhanced model is corrected to obtain a detoxified training model. This method can proactively identify covert data poisoning methods, enabling early defense against data poisoning. It constructs a collaborative security system with multiple defense points, covering the entire lifecycle from data preparation and model training to online inference, effectively protecting and purifying the model training process and improving the accuracy and reliability of model training. Attached Figure Description

[0049] To more clearly illustrate the technical solutions in the embodiments or related technologies of this application, the accompanying drawings used in the description of the embodiments or related technologies will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0050] Figure 1 This is a schematic diagram of the process of a data-based poisoning model detoxification training method provided in an embodiment of the present invention;

[0051] Figure 2 This is a schematic diagram of the structure of a data-based poisoning model detoxification training device provided in an embodiment of the present invention. Detailed Implementation

[0052] To make the objectives, technical solutions, and advantages of this application clearer, the following detailed description is provided in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the scope of this application.

[0053] In one embodiment, such as Figure 1 As shown, a model detoxification training method based on data poisoning is provided. This embodiment illustrates the application of this method to a training terminal. It can be understood that this method can also be applied to a server, and also to a system including a terminal and a server, and implemented through the interaction between the terminal and the server. In this embodiment, the method includes the following steps:

[0054] Step 101: Obtain the original training data and perform preprocessing operations on the original training data to obtain a clean initial dataset.

[0055] The raw training data consists of unprocessed initial data collected from the real world or various channels, containing various noises such as duplicate information, mislabeled data, missing values, inconsistent formats, and potentially malicious data intentionally injected by attackers. Preprocessing is a series of technical operations designed to clean and standardize the raw data, aiming to improve data quality and lay a solid foundation for model training. The clean initial dataset is a high-quality dataset obtained after preprocessing, serving as the cornerstone of all subsequent training steps. "Clean" is relative, meaning that obvious errors and noise have been removed, but there may still be deeply hidden poisoned samples that are difficult to detect using simple rules. The terminal identifies and processes problems in the raw data, deleting completely duplicated records, filling in missing values, or directly deleting samples with excessive missing values. For text data, this includes correcting spelling errors, removing irrelevant characters, checking the accuracy of data labels, identifying inconsistently labeled samples through statistical analysis, transforming data features to a uniform scale or distribution, converting data to a specific format acceptable to the model, uniformly adjusting images to a fixed size, or converting text into sequences required by the model. For example, raw data is pulled from multiple sources, and preliminary exploratory data analysis is immediately conducted. This includes checking the total amount of data, the proportion of missing values ​​in each field, basic statistical information, and the degree of data format disorder to form an overall understanding of data quality and provide a basis for subsequent cleaning strategies. For missing values, the missing pattern is analyzed to determine whether it is completely random. Different strategies are adopted according to the importance of the feature. For key features, missing samples are directly deleted. For numerical features, the mean, median, or value predicted by a regression model from other samples of the same feature may be used for imputation. For categorical features, the mode is used for imputation or a new missing category is created. Statistical methods are used to identify samples that significantly deviate from the main distribution. For outliers, business knowledge is needed to make judgments. If it is a record error, it should be corrected or deleted; if it is a normal but rare case, it should be retained and smoothed in subsequent feature engineering. A set of standardized rules should be established to convert the data into a format more suitable for model learning to eliminate the influence of different feature units. If the data comes from multiple sources, it should be matched and merged according to key fields. Duplicate records should be identified and merged using exact or fuzzy matching algorithms to ensure that each entity appears only once in the dataset. The processed complete dataset should be randomly shuffled and then divided into training set, validation set and test set according to a predetermined ratio. In this embodiment, the clean initial dataset usually specifically refers to the training set part used for subsequent model training.

[0056] Step 102: Based on the basic training model, make a small perturbation to the clean initial dataset to generate a set of adversarial examples against the basic training model.

[0057] Specifically, the base training model is a model that has been initially trained using a clean initial dataset and possesses basic task capabilities. A micro-perturbation is a carefully calculated, artificially added, minute change to the original sample. These changes have minimal perceptual impact, such as imperceptible pixel changes in an image or semantically similar words in text, but are sufficient to cause the model to make incorrect judgments. The adversarial example set is a large collection of effective adversarial examples generated against the base training model, simulating malicious attack data the model might encounter in the real world. The terminal extracts a batch of data from the clean initial dataset and inputs it into the base training model. The model makes predictions and calculates the loss. Through backpropagation, it calculates the gradient of the loss relative to the input data itself. The gradient direction indicates how to subtly change the input data to most effectively increase the model's loss. Along the gradient direction, a very small, controlled perturbation is applied to the original clean samples. The magnitude of the perturbation is strictly limited. The generated candidate adversarial samples are then input into the base training model again to check whether the model makes incorrect predictions. Only samples that successfully deceive the model are considered valid adversarial samples. Valid adversarial samples are further filtered, eliminating those that differ too much from the original samples to ensure that the perturbation is indeed small. All batches of high-quality adversarial samples are integrated to form an adversarial sample set.

[0058] Step 103: Based on the injection strategy, the adversarial sample set is injected into the pure initial dataset to obtain a hybrid training set. The basic training model is then trained based on the hybrid training set to obtain an anti-interference enhancement model.

[0059] Specifically, the injection strategy is a method for mixing adversarial examples with the clean initial dataset, defining the mixing ratio, method, and data augmentation techniques. The mixed training set is a new training dataset formed by merging the adversarial example set and the clean initial dataset according to the injection strategy, containing both normal and adversarial samples. The robustness enhancement model is a new model trained using the mixed training set. This model not only learns the characteristics of normal data but also learns to ignore adversarial perturbations, thus exhibiting stronger resistance to such attacks. The terminal injects the adversarial example set into the clean initial dataset at a certain ratio according to the injection strategy, forming a mixed training set. During training, a special loss function is used that considers both the model's loss on normal samples and its loss on adversarial samples. The mixed training set is used to retrain or continue training the basic trained model. After each training cycle, the model performance is evaluated on a clean validation set and a test set containing adversarial samples. Based on the evaluation results, the injection strategy or training parameters are dynamically adjusted to optimize the training effect.

[0060] Step 104: Obtain the key training parameters of the anti-interference enhancement model, and compare the key training parameters with the preset safety threshold to obtain anomaly indicators.

[0061] Among them, key training parameters are indicators that reflect the learning status and health of the model during training. These include internal model parameters as well as monitoring metrics such as training loss, validation loss, validation accuracy, and the optimizer's state, showing their changes in curves and values ​​during training. Preset safety thresholds are reference standards used to judge whether the training process is normal, including that the validation loss should not exceed a certain upper limit, the accuracy should not fall below a certain lower limit, or the fluctuation of the loss value over several consecutive periods should not exceed a certain range. An anomaly flag is a signal or indicator used to indicate whether the model's training process or current state deviates from the normal trajectory; it can be a Boolean value or a more detailed risk level. During the training of the robust model, the terminal continuously or periodically records key training parameters and compares them with preset safety thresholds. It checks whether the loss value suddenly spikes or drops too quickly, whether the accuracy stagnates or suddenly drops, and whether the weight values ​​overflow or reach extreme values. If a key parameter exceeds the safety threshold range or exhibits an abnormal state, an anomaly flag is generated.

[0062] Step 105: Based on the anomaly identifier, the anti-interference enhancement model is modified to obtain the detoxification training model.

[0063] The correction process involves taking a series of targeted measures to adjust the model or training process based on the problems reflected by the anomaly indicators, in order to eliminate abnormal states. The detoxified training model is a more robust and reliable model obtained after the correction steps, with strong anti-interference capabilities, and the training process itself is considered healthier and cleaner. The terminal performs correction operations depending on the type of problem revealed by the anomaly indicators. If the anomaly is caused by training instability, the model parameters are rolled back to a previous stable checkpoint, and hyperparameters such as the learning rate are adjusted before retraining. If it is merely a local optimum or overfitting, a stronger regularization method or early stopping is used to obtain a better model.

[0064] This embodiment provides a data poisoning-based model detoxification training method. It acquires original training data and preprocesses it to obtain a clean initial dataset. Based on the basic training model, it applies minor perturbations to the clean initial dataset to generate an adversarial example set against the basic training model. Using an injection strategy, it injects the adversarial example set into the clean initial dataset to obtain a hybrid training set. The basic training model is then trained on this hybrid training set to obtain an anti-interference enhanced model. Key training parameters of the anti-interference enhanced model are acquired and compared with preset safety thresholds to obtain anomaly indicators. Based on these anomaly indicators, the anti-interference enhanced model is corrected to obtain a detoxified training model. Through these methods, it is possible to proactively identify covert data poisoning methods and achieve early defense against data poisoning. It constructs a collaborative security system with multiple defense points, covering the entire lifecycle from data preparation and model training to online inference, effectively protecting and purifying the model training process and improving the accuracy and reliability of model training.

[0065] In one embodiment, based on the base trained model, a small perturbation is applied to the clean initial dataset to generate a set of adversarial examples against the base trained model, including:

[0066] Step 201: Extract a batch of clean samples from the clean initial dataset, and input the clean samples into the base training model of the training mode to calculate the gradient of the loss function and obtain the initial gradient direction.

[0067] In this dataset, a batch of clean samples is a small subset of data randomly selected from the clean initial dataset. Batch processing improves computational efficiency and utilizes batch statistics. The base training model in the training mode is a pre-trained model intended as the target of the attack, but its state is set to training mode. This is because some network layers behave differently in training and evaluation modes. To generate effective adversarial examples, the model needs to maintain consistent behavior with training to ensure the correctness of gradient calculations. The gradient of the loss function is the rate of change of the error between the model output and the true label relative to the input data. It is a vector, where each dimension indicates the direction and magnitude of a small change in the corresponding input data point to most quickly increase the model's loss. The initial gradient direction is the calculated gradient vector, indicating the most effective direction to attack the input sample in order to cause the model to misjudge. The terminal retrieves a batch of clean samples and their corresponding real labels from the data loader, ensuring that the basic training model is in training mode. This batch of clean samples is then input into the model, which makes predictions and obtains the prediction results. A loss function is used to calculate the difference between the model's predictions and the real labels, resulting in a scalar loss value. Backpropagation is then performed. The goal of backpropagation is not to calculate the gradient of the loss relative to the model weights, but rather to calculate the gradient of the loss relative to the input data itself. This means that during the calculation process, the model weights are fixed as constants, while the input data is treated as a variable that needs to be differentiated. After backpropagation is complete, the gradient tensor with respect to the input data is extracted from the computation graph; this tensor represents the initial gradient direction.

[0068] Step 202: Based on the gradient direction, add a small adversarial perturbation to the clean sample to generate candidate adversarial samples.

[0069] Specifically, the micro-adversarial perturbation is a tiny tensor with the exact same size as the original sample. This perturbation is calculated based on the initial gradient direction and aims to significantly alter the model's predictions with almost no change to the perception. Candidate adversarial examples are new samples obtained by adding the calculated micro-adversarial perturbation to the original clean sample; they are almost identical to the original samples but designed to deceive the model. The endpoint selects the specific gradient exploitation method based on the attack target. The most classic method is the fast gradient sign method. For example, the perturbation is calculated using the following formula: In this equation, 'a' represents the perturbation, 'u' represents the gradient, and the 'sign()' function takes the sign of the gradient in each dimension, discarding the magnitude information and retaining only the direction. 'ε' is a very small hyperparameter called the perturbation strength, which strictly limits the maximum magnitude of the perturbation to ensure that the perturbation is small. An addition operation is performed, and the candidate adversarial sample is the sum of the clean sample and the perturbation. The addition operation may cause the value of the new sample to exceed the legal range, so each pixel value of the candidate adversarial sample needs to be cropped to the allowed range to ensure that it is still a valid input.

[0070] Step 203: Input the candidate adversarial samples into the basic training model to verify their effectiveness and obtain effective adversarial samples.

[0071] Specifically, validity verification involves inputting candidate adversarial examples into the model and testing whether they can successfully deceive the model. A valid adversarial example is a candidate adversarial example that successfully changes the model's prediction from the original correct label to another incorrect label. The terminal switches the basic trained model from training mode to evaluation mode. When verifying the attack effect, the model needs to exhibit standard, deterministic reasoning behavior, rather than random training behavior. The candidate adversarial examples are input into the model in evaluation mode, and the model outputs a prediction result for each example. For each candidate adversarial example, the category with the highest predicted probability is taken as the model's predicted label. The predicted label is compared with the sample's true label. If the predicted label does not match the true label, the attack is considered successful, and the sample is marked as a valid adversarial example. If the predicted label matches the true label, the attack is considered unsuccessful, and the sample is discarded.

[0072] Step 204: Based on the clean samples, remove the effective adversarial samples whose difference from the clean samples is greater than a preset difference threshold to obtain high-quality adversarial samples.

[0073] Specifically, the difference threshold is a preset value used to measure the upper limit of the allowable difference between the adversarial example and the original sample. The difference is usually measured using norm or cosine distance. A high-quality adversarial example is an effective adversarial example that not only effectively deceives the model, but also whose difference from the original sample is controlled within the difference threshold, emphasizing the stealth of the attack. For each effective adversarial example, the terminal calculates the difference between it and the corresponding clean sample. For example, a commonly used method is to calculate the Euclidean distance between them, which measures the overall energy of the perturbation. The calculated difference is compared with the preset difference threshold. If the difference is less than or equal to the threshold, the sample is considered a high-quality adversarial example and is retained; if the difference is greater than the threshold, the sample is discarded. Excessive perturbation may make the modification obvious, losing the deceptive meaning of the adversarial example and becoming more like an obvious noise sample.

[0074] Step 205: Integrate the high-quality adversarial samples corresponding to the clean samples of each batch to obtain the adversarial sample set.

[0075] Integration is the process of aggregating and merging high-quality adversarial samples generated in different batches. The adversarial sample set is a complete collection containing high-quality adversarial samples from all batches. Each sample in the set has its corresponding original sample and ground truth label, serving as key inputs for subsequent adversarial training. The terminal repeatedly executes steps 201 to 204, traversing the entire training dataset to ensure that corresponding adversarial samples are generated for most of the training data. After generating high-quality adversarial samples in each batch, they are saved in a list or a specific data structure. Once all batches have been processed, all saved high-quality adversarial samples are merged into a large, unified data structure, forming the adversarial sample set, and then shuffled to ensure random use in subsequent training.

[0076] This embodiment generates an adversarial sample set by applying a small perturbation, achieving proactive defense against data poisoning and improving the reliability and security of model training.

[0077] In one embodiment, a model is trained on a base training model based on a hybrid training set to obtain an anti-interference enhancement model, including:

[0078] Step 301: Based on the mixed training set, calculate the total loss of the basic training model using the following formula:

[0079]

[0080] in, Where N is the total loss, N is the batch size, and i is the sample index. For clean samples, For adversarial examples, For real labels, For model parameters, For model prediction, Let cross-entropy be the loss function. To counteract the loss of weights.

[0081] The mixed training set is a training dataset composed of clean samples from the pure initial dataset and adversarial samples from the adversarial sample set, mixed in a certain proportion. The total loss is a composite loss value, a comprehensive quantification of the model's performance in the current training batch. It consists of two parts: the model's predicted loss for all clean samples, and the model's predicted loss for all corresponding adversarial samples multiplied by a weighting coefficient. The core purpose of the loss function is to force the model to simultaneously consider accurate classification of normal data and resistance to adversarial attacks when optimizing parameters. The cross-entropy loss function is a function that measures the difference between the model's predicted probability distribution and the true label distribution; the smaller the value, the more accurate the model's prediction. The adversarial loss weight is a hyperparameter used to balance the importance of clean sample loss and adversarial sample loss in the total loss. The terminal extracts a small batch of data from the mixed training set. For example, the samples in the batch appear in pairs. For index i, the sample pair consists of the i-th clean sample and the corresponding i-th adversarial sample, sharing the same true label. Optionally, the samples in the batch may not appear in pairs. In this case, the total loss is the corresponding independent cross-entropy loss. The batch size refers to the number of sample pairs. The samples in the entire batch are input into the basic training model, and predictions are made for each class of samples to obtain the prediction results. For each clean sample in the batch, the cross-entropy loss between its prediction result and the true label is calculated, which measures the basic classification ability of the model. For each corresponding adversarial sample in the batch, the cross-entropy loss between its prediction result and the true label is calculated, which measures the vulnerability of the model to targeted attacks. The clean loss and the weighted adversarial loss of each sample pair are added together. The loss values ​​of all sample pairs are summed and divided by N to obtain the average total loss of the batch.

[0082] Step 302: Backpropagate the total loss of the basic training model to obtain the parameter update model, and evaluate the anti-interference ability of the parameter update model based on the validation set to obtain an anti-interference report.

[0083] Specifically, backpropagation is an automatic differentiation algorithm that starts by calculating the total loss and, following the chain rule, calculates the gradient of the total loss with respect to each parameter of the model. The gradient indicates how the parameters should be slightly adjusted to reduce the total loss. The parameter update model, after calculating the gradient through backpropagation, is a new model obtained by the optimizer adjusting the existing parameters of the base training model based on the gradient and a preset learning rate. This represents a step forward in the model's robustness learning process. The validation set is a dataset consisting entirely of clean, unused samples, used to objectively evaluate the model's generalization ability on unseen normal data, preventing overfitting. Robustness assessment is a dedicated testing process used to quantify the model's resistance to adversarial attacks. It uses a pre-prepared set of adversarial samples generated by various attack algorithms to challenge the model. The robustness report is a summary document based on the evaluation results, including key metrics such as the model's accuracy on the clean validation set, accuracy on the adversarial test set, and resistance to different attack types. The terminal executes the backpropagation algorithm to calculate the gradient of the total loss with respect to all model parameters. The optimizer uses this gradient information to update the model parameters according to its internal rules. The updated model state is the parameter-updated model. The parameter-updated model is set to evaluation mode and run on the validation set to calculate its classification accuracy and other standard metrics, ensuring that the model's basic performance has not been severely degraded due to adversarial training. Anti-interference capability is evaluated by inputting samples from the adversarial test set into the model and calculating the model's accuracy on these malicious samples. Tests are conducted for attacks of different intensities or types. The performance of the model on the clean validation set and the adversarial test set is combined to generate an anti-interference report. The report clearly depicts the current model's trade-off between accuracy and robustness.

[0084] Step 303: Based on the anti-interference report, update the injection strategy to obtain the updated injection strategy; the updated injection strategy is used to update the hybrid training set.

[0085] Specifically, the injection strategy is a dynamic set of rules that guides the construction of a hybrid training set. It mainly defines aspects such as whether to generate adversarial examples for every clean sample, which algorithm to use to generate adversarial examples, the perturbation strength of the adversarial examples, and the weights in the loss function. It serves as a scheduling strategy for the training process. Updating the injection strategy involves adjusting the parameters or rules of the original injection strategy based on feedback from the anti-interference report, resulting in a new strategy that reflects the adaptive capability of the training system. The terminal analyzes the various metrics in the anti-interference report in depth. For example, it focuses on whether the model's robust accuracy on the adversarial test set meets expectations; whether it performs significantly poorly against a specific attack; and whether the model's accuracy on clean data drops excessively due to adversarial training. Based on the analysis results, it decides how to adjust the injection strategy. Optionally, if robustness is insufficient, the adversarial loss weight is increased to force the model to pay more attention to adversarial examples, or the attack intensity of adversarial examples is enhanced to expose the model to stronger perturbations and improve its resistance. If the accuracy on clean data drops excessively, the adversarial loss weight is appropriately reduced to allow the model to return to normal data, avoiding excessive pursuit of robustness at the expense of basic performance. If the defense against a certain type of attack is weak, the use of such attack algorithms can be prioritized or increased when generating adversarial examples for targeted enhancement. The adjustment scheme is concretized into executable parameters and rules to form an update injection strategy.

[0086] Step 304: Train the parameter update model based on the mixed training set to obtain the anti-interference enhancement model.

[0087] The robustness enhancement model, obtained after multiple training epochs, not only performs well on a clean validation set but, more importantly, demonstrates high and stable robust accuracy on the adversarial example test set, indicating successful and significant robustness enhancement. The terminal applies the update injection strategy to the next training round, reconstructing the hybrid training set using new adversarial loss weights or new adversarial example generation parameters. Using the parameter-updated model as a new starting point, and with data prepared according to the new strategy, the process of calculating the total loss, updating the model, and evaluating and updating the strategy is repeated iteratively. Training stops when the model's performance metrics on the validation and adversarial test sets stabilize and no longer show significant improvement, or when the preset maximum number of training epochs is reached. The model with the best performance is the robustness enhancement model.

[0088] This embodiment introduces a feedback loop, transforming the training process from a static flow into a dynamic adaptive process. Through multiple iterations of the entire dynamic closed-loop process, more efficient and accurate reinforcement training is achieved.

[0089] In one embodiment, after modifying the anti-interference enhancement model based on anomaly identification to obtain the detoxification training model, the method further includes:

[0090] Step 401: Slice the clean initial dataset based on different data dimensions to obtain data slices; the data dimensions include at least one of the following: data source, time window, user attributes, and content features.

[0091] Data dimensions are specific attributes or perspectives used to divide a dataset, enabling the data to be divided into subsets with common characteristics. These include: data source (the origin of data collection, such as different websites, sensor numbers, or third-party data providers); time window (the time period during which the data was generated, such as a day, a week, or a specific event); user attributes (user demographics or behavioral characteristics, such as age range, geographic location, membership level, and historical spending power); and content characteristics (the attributes of the data content itself, such as the topic classification, sentiment polarity, and length of text; and the main objects, color distribution, and sharpness of images). A data slice is a subset of data extracted from a complete, clean initial dataset based on one or more data dimensions. The endpoint selects one or more key dimensions based on business logic and potential attack patterns. For example, if a data provider is suspected of being contaminated, then data source is the primary dimension; if an attack is likely to occur at a specific time, then, combined with the time window, data querying and grouping techniques are used to divide the clean initial dataset into multiple non-overlapping or partially overlapping data slices based on the selected dimensions. Each slice contains all samples that meet the conditions of that dimension. The definition of each slice and the index of the samples it contains are recorded to prepare for subsequent analysis.

[0092] Step 402: Based on the detoxification training model and data slices, calculate key performance indicators to obtain a detailed performance evaluation report for each data slice.

[0093] Specifically, key performance indicators (KPIs) are a series of metrics used to quantify a model's performance on a specific subset of data. These include accuracy, which is the proportion of samples correctly predicted by the model; loss, which is the degree of difference between the model's prediction and the true label (lower values ​​are better); and precision and recall, which, for classification problems, more precisely measure the model's ability to identify a specific category. A detailed performance evaluation report is a quantitative evaluation document for each data slice, showing all the KPIs calculated by the detoxification training model on that slice and comparing them with the model's average performance on the entire dataset or a benchmark slice. The terminal sets the detoxification training model to evaluation mode, inputting each data slice as an independent test set into the model for prediction. For each slice, the model compares the true labels of all samples with its predictions, calculates the preset KPIs, and generates a detailed performance evaluation report for each slice, including the absolute values ​​of the KPIs and their relative values ​​to the global average, to facilitate rapid anomaly identification.

[0094] Step 403: Based on the detailed performance evaluation report, identify the data slices where key performance indicators are abnormal to obtain abnormal data slices.

[0095] Specifically, abnormal data slices are those whose key performance indicators are significantly lower or higher than expected or baseline levels. For example, if the accuracy of a slice is much lower than the overall accuracy, or its loss value is abnormally high, it indicates that the slice may have serious problems. The terminal reviews the detailed performance evaluation reports of all data slices one by one. Based on historical experience or statistical rules, a threshold for judging whether an indicator is abnormal is set. The performance indicators of each slice are compared with the global indicators or the indicators of other similar slices. Special attention is paid to slices whose indicator values ​​deviate too much from the normal range. Slices whose performance indicators trigger the abnormal threshold are marked as abnormal data slices, and their specific abnormal behavior is recorded.

[0096] Step 404: Based on the abnormal data slices, generate a suspicious data slice location report, and based on the suspicious data slice location report, remove the poisoned samples from the pure initial dataset to obtain the purified training dataset.

[0097] The Suspicious Data Slice Location Report is an analysis summary based on anomalous data slices. It summarizes the presence of anomalies in the slices, the type and severity of the anomalies, and preliminarily infers that poisoned samples may be concentrated in certain slices. Poisoned samples are samples hidden in the dataset with malicious intent, designed to damage model performance or induce specific errors in the model. The Clean Training Dataset is a cleaner and more reliable dataset obtained by identifying and removing poisoned samples from the clean initial dataset. Based on the list of anomalous data slices, the terminal generates a Suspicious Data Slice Location Report, clearly indicating the range of slices requiring focused review. Following the report's guidance, each sample within the anomalous data slice is thoroughly examined. Combining anomaly detection algorithms, by analyzing sample content, feature distribution, and their correspondence with model errors, the poisoned samples causing anomalous slice performance are identified. All confirmed poisoned samples are permanently removed from the clean initial dataset. After the removal operation is completed, the remaining data constitutes the Clean Training Dataset.

[0098] Step 405: Based on the purified training dataset, fine-tune the detoxification training model to obtain the detoxification model.

[0099] Fine-tuning is a training technique that involves using a new, higher-quality dataset to further train an already trained model on a small number of epochs with a smaller learning rate. The goal is to adapt the model to the distribution of the new data, solidify knowledge learned from clean data, and discard erroneous patterns learned from poisoned samples. The detoxified model is a model obtained after fine-tuning on a clean training dataset, combining the robustness of adversarial training with the high accuracy provided by clean data. The terminal loads the detoxified training model as the initial model for fine-tuning, setting a smaller learning rate than in the main training phase to prevent the destruction of learned useful features. The model is trained using the clean training dataset in fewer epochs, continuing only until the model's loss function converges on the new data. During fine-tuning, a clean validation set is used to monitor model performance, ensuring that fine-tuning does not lead to performance degradation. The model saved after training is the detoxified model.

[0100] This embodiment, through fine-tuning, allows the detoxification model to completely avoid the influence of the poisoned samples during initial training, building upon its existing anti-interference capabilities. This makes its decisions more reliant on pure and reliable data features, thereby improving the accuracy of the model's detoxification training.

[0101] In one embodiment, based on the suspicious data slice location report, poisoned samples are removed from the clean initial dataset to obtain a cleaned training dataset, including:

[0102] Step 501: Based on the suspicious data slice report and anomaly detection algorithm, identify the poisoning sample for each abnormal data slice to obtain an abnormal sample list.

[0103] The suspicious data slice report is an analysis summary from previous steps, clearly identifying which data slices were identified as anomalous and describing their unusual behavior. Anomaly detection algorithms are machine learning algorithms used to identify outliers or outliers in a dataset that behave significantly differently from the majority of data. Typical algorithms include Isolation Forest and Local Outlier Factor. The anomalous sample list is a list containing specific sample identifiers; these samples are the individuals identified by the anomaly detection algorithm within the anomalous data slice that exhibit the most abnormal behavior and are most likely to be poisoned samples. Based on suspicious data slice reports, the terminal strictly limits the analysis scope to the marked abnormal data slices, greatly narrowing the investigation scope and improving accuracy and efficiency. For each sample in the target slice, a set of features that can characterize its essence is extracted. Optionally, for image samples, this may be the feature vector obtained through an intermediate layer of a model; for text samples, it may be the word vector representation or sentence vector representation. The feature representations of all samples in the slice are input into the selected anomaly detection algorithm. The algorithm analyzes the distribution of all samples in the feature space. For example, the Isolation Forest algorithm is selected. Its core idea is to isolate samples by randomly splitting the feature space. Normal samples are densely distributed and require many splits to be isolated; while abnormal samples, because their characteristics are different from the general population, are distributed in sparse areas at the edge of the space, so they can be isolated with only a few splits. The algorithm calculates an anomaly score for each sample. Based on the anomaly score output by the algorithm, the samples in the slice are sorted, and the top-ranked samples are extracted to form an abnormal sample list.

[0104] Step 502: Based on the content moderation model, evaluate each poisoning sample in the abnormal sample list from multiple perspectives to obtain the moderation results.

[0105] Specifically, a content moderation model is an AI model specifically trained to evaluate content security and quality from multiple dimensions. It's typically an integrated system containing multiple sub-models, each responsible for different moderation angles. Multi-angle evaluation is the process of comprehensively reviewing each sample in the list of abnormal samples from multiple dimensions, aiming to determine whether it's malicious poisoning or simply low-quality data. The moderation result is a structured conclusion output by the content moderation model after evaluating a single sample, including scores or tags across multiple dimensions. The terminal loads a dedicated content moderation model, inputting samples from the abnormal sample list one by one. The moderation model analyzes the samples in parallel, checking whether the sample content conforms to common sense or objective facts. Optionally, it checks whether an image contains illogical splicing elements; whether an event described in a text is physically possible; whether the sample's features are consistent with its labels. Optionally, it checks whether an image labeled as a cat actually contains the main features of a cat; whether a comment marked as a positive review contains positive vocabulary and sentiment; it compares the sample with a database of known poisoning attack patterns. Optionally, it checks whether the image contains adversarial perturbation textures that are difficult to detect with the naked eye; whether the text embeds specific trigger words or syntactic structures used to mislead the model; and it evaluates the quality of the sample itself. Optionally, it checks whether the image is too blurry or noisy; whether the text is grammatically incorrect, full of typos, or meaningless. For each reviewed sample, the content moderation model outputs a comprehensive review result, which can be a vector containing confidence scores for each dimension or a classification suggestion.

[0106] Step 503: Based on the audit results, generate confirmed poisoning samples and low-quality samples; and remove the confirmed poisoning samples and low-quality samples from the clean initial dataset to obtain the remaining samples.

[0107] Specifically, confirmed poisoned samples are those explicitly identified as having malicious intent and aimed at damaging model performance based on the review results. Low-quality samples, while not having explicit malicious intent, are those with serious quality defects that also interfere with the model learning correct features and need to be removed. The remaining samples are the dataset remaining after removing all confirmed poisoned and low-quality samples from the clean initial dataset. The terminal comprehensively interprets the review results for each sample, classifies the samples according to preset rules, summarizes all the identifiers of confirmed poisoned and low-quality samples to form a list to be removed, and operates on the clean initial dataset. Based on the list to be removed, the corresponding sample data and their labels are permanently deleted. This is an irreversible, physical deletion operation. After the deletion operation is completed, the remaining portion of the dataset is the remaining samples.

[0108] Step 504: Perform an integrity check on the remaining samples to obtain a cleaned training dataset.

[0109] The integrity check involves a series of quality verifications performed on the remaining sample set after removing a large number of samples to ensure it remains a valid dataset suitable for model training. The cleaned training dataset is a high-quality dataset confirmed for model fine-tuning after all integrity checks. The process includes checking whether the overall distribution of the remaining samples has shifted significantly compared to before removal; for example, checking if the number of samples in each category remains balanced. If too many samples in a certain category have been removed, it may lead to insufficient representativeness of that category, requiring consideration of data augmentation or resampling. Basic statistics of the dataset are calculated to ensure that data cleaning has not introduced unexpected errors or extreme distributions. A small number of remaining samples are randomly selected for end-to-end verification. Optionally, this ensures that the sample data can be correctly loaded and decoded, and that the correspondence between its labels is correct. After all checks pass, the remaining sample set is officially confirmed as the cleaned training dataset and saved, ready for the next step of model fine-tuning.

[0110] This embodiment introduces a content moderation model for multi-angle evaluation, effectively avoiding misjudgments that may be caused by relying solely on anomaly detection scores, thereby improving the accuracy and reliability of detoxification training.

[0111] In one embodiment, after fine-tuning the detoxification training model based on the purified training dataset to obtain the detoxification model, the method further includes:

[0112] Step 601: Receive real-time request data and perform preprocessing operations on the real-time request data to obtain standard real-time data.

[0113] Real-time request data refers to raw data received in real-time from actual application scenarios after the model is deployed online. Its format, size, and encoding vary greatly and may contain noise. Preprocessing operations are a series of operations identical to those in the training phase, aiming to transform irregular real-time data into a standardized format that the model can process. The preprocessing logic here must be strictly consistent with the logic in step 101 of the training phase. Standard real-time data is data that has been preprocessed to meet the model's input requirements, with the same format and numerical range as the data used in training, making it a valid input that the model can understand. The terminal receives real-time request data from the front end, performs preliminary parsing based on the data type, extracts the core data content, and operates strictly according to the process defined during training. For example, for images, the size is adjusted to a fixed size, color channels are converted, and pixel values ​​are normalized; for text, word segmentation, stop word removal, conversion of words into corresponding ID sequences, and truncation or padding to a fixed length are performed; for numerical features, the same mean and variance as during training are used for standardization. The processed data is then encapsulated into a tensor format required by the model's inference framework.

[0114] Step 602: Input the standard real-time data into the detoxification model to obtain preliminary inference results.

[0115] Specifically, the detoxification model is a highly robust and secure model obtained through adversarial training, parameter correction, data cleaning, and fine-tuning. The preliminary inference result is the raw output of the detoxification model after calculating standard real-time data; it is either a probability distribution or a specific predicted value. The terminal loads the detoxification model into memory and sets it to inference mode. Standard real-time data is passed as input to the model, which internally performs a series of matrix operations and nonlinear transformations to post-process the raw output, obtaining the preliminary inference result.

[0116] Step 603: Integrate the preliminary reasoning results, standard real-time data, and preset domain knowledge to obtain the verification context; the domain knowledge includes security rules and logical constraints.

[0117] Specifically, domain knowledge comprises expert rules, business logic, and security constraints related to the model's application scenarios. It's a common-sense base that places data and results within specific scenarios for reasonable judgment, including security rules and logical constraints. The validation context is a structured information package that integrates preliminary inference results, raw standard real-time data, and relevant domain knowledge, providing all the background information needed for subsequent large language model judgments. The terminal extracts the standard real-time data and preliminary inference results of the current request. Based on the characteristics of the data, it retrieves relevant rules and constraints from the domain knowledge base, organizing the extracted information into a logically clear and easily understood structure, typically a natural language prompt.

[0118] Step 604: Based on the preset evaluation instructions, input the verification context into the large language model in the security environment to obtain the auxiliary process verification conclusion; the auxiliary process verification conclusion includes at least one of pass, suspicious, and rejection.

[0119] The secure environment is a computing environment physically or logically isolated from the public internet. It stores the large language model used for verification and its dedicated, rigorously cleaned dataset. This is to prevent external attacks or data poisoning from contaminating the verifier, ensuring its independence and security. The preset evaluation instructions are a set of explicit instructions written to the large language model, specifying its role, tasks, output format, and decision-making basis. The auxiliary process verification conclusion is a three-way choice, determined by the large language model based on the evaluation instructions after analyzing the verification context, clarifying the security status of the current request. In the secure environment, the terminal combines the assembled verification context and preset evaluation instructions into a complete prompt, which is then input into the large language model. Based on its vast internal knowledge and understanding of the instructions, the large language model reasons about the prompt, considering whether the preliminary result matches data characteristics, whether it violates any security rules, and whether there are logical contradictions or inconsistencies. It then generates a natural language response, from which the structured auxiliary process verification conclusion is parsed, and its rationale is extracted.

[0120] Step 605: Based on the auxiliary process verification conclusion, calculate the confidence level of the corresponding preliminary reasoning result and obtain the risk report.

[0121] The confidence level is a quantified numerical value used to represent the reliability of the preliminary reasoning result. It is calculated based on the auxiliary process verification conclusion, with high confidence corresponding to high confidence, medium or low confidence to suspicious, and extremely low confidence to reject. The risk report is a conclusion that integrates the preliminary reasoning result and the auxiliary process verification conclusion, clearly providing decision recommendations and its uncertainty measurement. The terminal, according to preset mapping rules, converts the semantic auxiliary process verification conclusion into a numerical confidence level, integrates the preliminary reasoning result, the auxiliary process verification conclusion, and the calculated confidence level to generate a structured risk report.

[0122] This embodiment introduces an independent, knowledge-based auxiliary reasoning process to build a safety redundancy. Even if the main model makes a mistake due to extreme circumstances, the large language model can still identify risks based on rules and common sense, greatly reducing the risk of incorrect decisions.

[0123] It should be understood that although the steps in the flowcharts of the embodiments described above are shown sequentially according to the arrows, these steps are not necessarily executed in the order indicated by the arrows. Unless explicitly stated herein, there is no strict order restriction on the execution of these steps, and they can be executed in other orders. Moreover, at least some steps in the flowcharts of the embodiments described above may include multiple steps or multiple stages. These steps or stages are not necessarily completed at the same time, but can be executed at different times. The execution order of these steps or stages is not necessarily sequential, but can be performed alternately or in turn with other steps or at least some of the steps or stages of other steps.

[0124] Based on the same inventive concept, this application also provides a data-based poisoning model detoxification training device for implementing the data-based poisoning model detoxification training method described above. The solution provided by this device is similar to the solution described in the above method. Therefore, the specific limitations in one or more data-based poisoning model detoxification training device embodiments provided below can be found in the limitations of the data-based poisoning model detoxification training method described above, and will not be repeated here.

[0125] In one exemplary embodiment, such as Figure 2 As shown, a data-based poisoning model detoxification training device 700 is provided, comprising:

[0126] The clean module 701 is used to acquire the raw training data and perform preprocessing operations on the raw training data to obtain a clean initial dataset.

[0127] The perturbation module 702 is used to perform minor perturbations on the clean initial dataset based on the basic trained model to generate a set of adversarial examples against the basic trained model.

[0128] Training module 703 is used to inject the adversarial sample set into the clean initial dataset based on the injection strategy to obtain a hybrid training set, and to train the basic training model based on the hybrid training set to obtain an anti-interference enhancement model.

[0129] The comparison module 704 is used to obtain the key training parameters of the anti-interference enhancement model, and compare the key training parameters with the preset safety threshold to obtain anomaly indicators.

[0130] The correction module 705 is used to correct the anti-interference enhancement model based on the anomaly identifier to obtain the detoxification training model.

[0131] Furthermore, the disturbance module 702 is also used for:

[0132] Extract a batch of clean samples from the clean initial dataset, and input the clean samples into the base training model of the training mode to calculate the gradient of the loss function and obtain the initial gradient direction;

[0133] Based on the gradient direction, a small adversarial perturbation is added to the clean sample to generate candidate adversarial samples;

[0134] The candidate adversarial samples are input into the basic training model for effectiveness verification to obtain effective adversarial samples;

[0135] Based on the clean samples, effective adversarial samples with differences greater than a preset difference threshold are removed from the effective adversarial samples to obtain high-quality adversarial samples.

[0136] By integrating high-quality adversarial samples corresponding to the clean samples in each batch, an adversarial sample set is obtained.

[0137] Furthermore, training module 703 is also used for:

[0138] Based on the mixed training set, the total loss of the base training model is calculated using the following formula:

[0139]

[0140] in, Where N is the total loss, N is the batch size, and i is the sample index. For clean samples, For adversarial examples, For real labels, For model parameters, For model prediction, Let cross-entropy be the loss function. To counteract the loss of weight;

[0141] Backpropagation of the total loss of the basic training model yields the parameter update model. Based on the validation set, the robustness of the parameter update model is evaluated, and a robustness report is obtained.

[0142] Based on the anti-interference report, the injection strategy is updated to obtain the updated injection strategy; the updated injection strategy is used to update the hybrid training set.

[0143] The parameter update model is trained based on a hybrid training set to obtain an anti-interference enhancement model.

[0144] Furthermore, the device also includes a detoxification module for:

[0145] Based on different data dimensions, the pure initial dataset is sliced ​​to obtain data slices; the data dimensions include at least one of the following: data source, time window, user attributes, and content features.

[0146] Based on the detoxification training model and data slices, key performance indicators are calculated to obtain a detailed performance evaluation report for each data slice.

[0147] Based on the detailed performance evaluation report, identify the data slices where key performance indicators are abnormal, and obtain the abnormal data slices.

[0148] Based on the abnormal data slices, a suspicious data slice location report is generated, and based on the suspicious data slice location report, the poisoned samples in the pure initial dataset are removed to obtain the purified training dataset;

[0149] Based on the purified training dataset, the detoxification training model is fine-tuned to obtain the detoxification model.

[0150] Furthermore, the detoxification module is also used for:

[0151] Based on suspicious data slice reports and anomaly detection algorithms, poisoning samples are identified for each anomalous data slice to obtain a list of anomalous samples.

[0152] Based on the content moderation model, each poisoning sample in the abnormal sample list is evaluated from multiple perspectives to obtain the moderation results;

[0153] Based on the audit results, confirmed poisoning samples and low-quality samples are generated; and confirmed poisoning samples and low-quality samples are removed from the clean initial dataset to obtain the remaining samples.

[0154] The remaining samples are then subjected to an integrity check to obtain a cleaned training dataset.

[0155] Furthermore, the device also includes an auxiliary module for:

[0156] Receive real-time request data and perform preprocessing operations on the real-time request data to obtain standard real-time data;

[0157] Standard real-time data is input into the detoxification model to obtain preliminary inference results;

[0158] By integrating preliminary reasoning results, standard real-time data, and pre-defined domain knowledge, a verification context is obtained; the domain knowledge includes security rules and logical constraints.

[0159] Based on the preset evaluation instructions, the verification context is input into the large language model in the security environment to obtain the auxiliary process verification conclusion; the auxiliary process verification conclusion includes at least one of pass, suspicious, and rejection;

[0160] Based on the verification results of the auxiliary process, the confidence level of the corresponding preliminary reasoning results is calculated to obtain a risk report.

[0161] In one embodiment, an electronic device is provided, including a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a data-based poisoning model detoxification training method as described above.

[0162] In one embodiment, a computer-readable storage medium is provided having a computer program stored thereon, which, when executed by a processor, implements the steps in the above method embodiments.

[0163] For the electronic device embodiments, since they basically correspond to the method embodiments, the relevant details can be found in the description of the method embodiments. The electronic device embodiments described above are merely illustrative. The components described as separate parts may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of this disclosure according to actual needs. Those skilled in the art can understand and implement this without any inventive effort.

[0164] The above-described embodiments are merely illustrative of several implementation methods of the embodiments of this application, and their descriptions are relatively specific and detailed. However, they should not be construed as limiting the scope of the patent application. It should be noted that those skilled in the art can make various modifications and improvements without departing from the concept of the embodiments of this application, and these modifications and improvements all fall within the protection scope of the embodiments of this application.

Claims

1. A model detoxification training method based on data poisoning, characterized in that, The method includes: Obtain the original training data and perform preprocessing operations on the original training data to obtain a clean initial dataset; Based on the basic training model, a small perturbation is applied to the pure initial dataset to generate a set of adversarial examples against the basic training model; Based on the injection strategy, the adversarial sample set is injected into the pure initial dataset to obtain a hybrid training set, and the basic training model is trained based on the hybrid training set to obtain an anti-interference enhancement model. Obtain the key training parameters of the anti-interference enhancement model, and compare the key training parameters with the preset safety threshold to obtain anomaly indicators; Based on the anomaly identifier, the anti-interference enhancement model is modified to obtain the detoxification training model.

2. The method according to claim 1, characterized in that, The step of generating an adversarial example set against the basic training model by making minor perturbations to the clean initial dataset includes: A batch of clean samples is extracted from the clean initial dataset, and the clean samples are input into the base training model in the training mode to calculate the gradient of the loss function and obtain the initial gradient direction; Based on the gradient direction, a small adversarial perturbation is added to the clean sample to generate candidate adversarial samples; The candidate adversarial samples are input into the basic training model for effectiveness verification to obtain effective adversarial samples; Based on the clean sample, the effective adversarial samples whose difference from the clean sample is greater than a preset difference threshold are removed to obtain high-quality adversarial samples; The high-quality adversarial samples corresponding to each batch of clean samples are integrated to obtain the adversarial sample set.

3. The method according to claim 1, characterized in that, The step of training the base training model based on the hybrid training set to obtain an anti-interference enhancement model includes: Based on the hybrid training set, the total loss of the base training model is calculated using the following formula: in, Where N is the total loss, N is the batch size, and i is the sample index. For clean samples, For adversarial examples, For real labels, For model parameters, For model prediction, Let cross-entropy be the loss function. To counteract the loss of weight; The total loss is backpropagated to the basic training model to obtain the parameter update model. Based on the validation set, the robustness of the parameter update model is evaluated to obtain a robustness report. Based on the anti-interference report, the injection strategy is updated to obtain an updated injection strategy; the updated injection strategy is used to update the hybrid training set. The parameter update model is trained based on the hybrid training set to obtain the anti-interference enhancement model.

4. The method according to claim 1, characterized in that, After modifying the anti-interference enhancement model based on the anomaly identifier to obtain the detoxification training model, the process further includes: Based on different data dimensions, the pure initial dataset is sliced ​​to obtain data slices; the data dimensions include at least one of data source, time window, user attributes, and content features. Based on the detoxification training model and the data slices, key performance indicators are calculated to obtain a detailed performance evaluation report for each data slice. Based on the detailed performance evaluation report, the data slices in which the key performance indicators show abnormalities are identified, and abnormal data slices are obtained. Based on the abnormal data slices, a suspicious data slice location report is generated, and based on the suspicious data slice location report, the poisoned samples in the clean initial dataset are removed to obtain the clean training dataset; Based on the purification training dataset, the detoxification training model is fine-tuned to obtain the detoxification model.

5. The method according to claim 4, characterized in that, The process of removing poisoned samples from the clean initial dataset based on the suspicious data slice location report to obtain a clean training dataset includes: Based on the suspicious data slice report and anomaly detection algorithm, the poisoning sample is identified for each of the abnormal data slices to obtain an abnormal sample list; Based on the content moderation model, each of the poisoned samples in the abnormal sample list is evaluated from multiple perspectives to obtain the moderation results; Based on the audit results, confirmed poisoning samples and low-quality samples are generated; and the confirmed poisoning samples and low-quality samples are removed from the clean initial dataset to obtain the remaining samples. The remaining samples are subjected to an integrity check to obtain the purified training dataset.

6. The method according to claim 5, characterized in that, After fine-tuning the detoxification training model based on the purified training dataset to obtain the detoxification model, the process further includes: Receive real-time request data and perform preprocessing operations on the real-time request data to obtain standard real-time data; The standard real-time data is input into the detoxification model to obtain preliminary inference results; By integrating the preliminary reasoning results, the standard real-time data, and the preset domain knowledge, a verification context is obtained; the domain knowledge includes security rules and logical constraints. Based on preset evaluation instructions, the verification context is input into the large language model in the security environment to obtain the auxiliary process verification conclusion; the auxiliary process verification conclusion includes at least one of pass, suspicious, and reject. Based on the verification conclusion of the auxiliary process, the confidence level corresponding to the preliminary reasoning result is calculated to obtain a risk report.

7. A model detoxification training device based on data poisoning, characterized in that, The device includes: The clean module is used to acquire the original training data and perform preprocessing operations on the original training data to obtain a clean initial dataset. The perturbation module is used to perform minor perturbations on the clean initial dataset based on the base training model, thereby generating a set of adversarial examples against the base training model. The training module is used to inject the adversarial sample set into the clean initial dataset based on the injection strategy to obtain a hybrid training set, and to train the basic training model based on the hybrid training set to obtain an anti-interference enhancement model. The comparison module is used to obtain the key training parameters of the anti-interference enhancement model, and compare the key training parameters with a preset safety threshold to obtain an anomaly identifier. The correction module is used to correct the anti-interference enhancement model based on the anomaly identifier to obtain the detoxification training model.

8. An electronic device comprising a memory and a processor, wherein the memory stores a computer program, characterized in that, When the processor executes the computer program, it implements the steps of the method according to any one of claims 1 to 6.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When the computer program is executed by a processor, it implements the steps of the method according to any one of claims 1 to 6.

Images