A model training method, device, equipment and readable storage medium

Abstract

The specification discloses a model training method and device, equipment and a readable storage medium. According to a to-be-trained transformation network and a source model trained by a poisoned sample, a target model to be trained is determined. A clean sample obtained is input into the target model to obtain a prediction result output by the target model. According to the prediction result and a label of the clean sample, parameters of the transformation network are adjusted. When a prediction request is received, to-be-predicted data is input into the trained target model to obtain a prediction result of the to-be-predicted data. It can be seen that the reconstruction of the source model based on the transformation network avoids the input and / or output of the source model attacked by the poisoned sample being transformed by the transformation network without adjusting the model parameters of the source model, so as to avoid the source model obtaining an incorrect prediction result for the sample preset by the attacker, thereby ensuring the model performance and realizing a high defense effect and protecting the safety of private data.

Description

Technical Field

[0001] This specification relates to the field of computer technology, and in particular to a model training method, apparatus, device, and readable storage medium. Background Technology

[0002] With the development of artificial intelligence technology and increased public awareness of data privacy, machine learning models have found widespread application, such as facial recognition models, speech recognition models, and natural language processing models. However, machine learning models themselves face various security challenges, such as data poisoning and backdoor attacks. Attackers inject malicious samples into the model's training dataset to disrupt the learning process, causing the trained machine learning model to produce incorrect predictions for pre-selected clean samples, thereby reducing the model's performance.

[0003] Based on this, this specification provides a model training method. Summary of the Invention

[0004] This specification provides a model training method, apparatus, device, and readable storage medium to partially solve the aforementioned problems existing in the prior art.

[0005] The following technical solution is adopted in this specification:

[0006] This manual provides a model training method, including:

[0007] Obtain the source model pre-trained using poisoned samples;

[0008] Obtain the transformation network to be trained, and determine the target model to be trained based on the transformation network and the source model;

[0009] Obtain clean samples and their labels;

[0010] The clean sample is input into the target model to obtain the prediction result of the clean sample output by the target model;

[0011] The parameters of the transformation network are adjusted with the optimization objective of minimizing the difference between the prediction results of the clean samples and the labels of the clean samples, so as to obtain the trained target model.

[0012] When a prediction request is received, the data to be predicted corresponding to the prediction request is input into the trained target model to obtain the prediction result of the data to be predicted output by the target model.

[0013] This specification provides a model training apparatus, including:

[0014] The source model acquisition module is used to acquire the source model that has been pre-trained using poisoned samples;

[0015] The target model determination module is used to obtain the transformation network to be trained and determine the target model to be trained based on the transformation network and the source model.

[0016] A clean sample acquisition module is used to acquire clean samples and the labels of the clean samples;

[0017] The input module is used to input the clean sample into the target model and obtain the prediction result of the clean sample output by the target model;

[0018] The first training module is used to adjust the parameters of the transformation network with the optimization objective of minimizing the difference between the prediction result of the clean sample and the label of the clean sample, so as to obtain the trained target model.

[0019] The prediction module is used to input the data to be predicted corresponding to the prediction request into the trained target model when a prediction request is received, and to obtain the prediction result of the data to be predicted output by the target model.

[0020] This specification provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the above-described model training method.

[0021] This specification provides an electronic device, including a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the model training method described above.

[0022] The above-mentioned technical solutions adopted in this specification can achieve the following beneficial effects:

[0023] The model training method provided in this manual determines the target model to be trained based on the transformation network to be trained and the source model trained using poisoned samples. Clean samples are then input into the target model to obtain the prediction results output by the target model. Based on the prediction results and the labels of the clean samples, the parameters of the transformation network are adjusted to obtain the trained target model. This allows the model to be input into the target model when a prediction request is received, thus obtaining the prediction results for the data to be predicted. Therefore, this method of reconstructing the source model based on the transformation network transforms the input and / or output of the source model attacked by poisoned samples without adjusting the model parameters. This prevents the source model from obtaining incorrect prediction results for samples preset by the attacker, thereby achieving a high level of defense while maintaining model performance. Attached Figure Description

[0024] The accompanying drawings, which are included to provide a further understanding of this specification and form part of this specification, illustrate exemplary embodiments and their descriptions, serving to explain this specification and do not constitute an undue limitation thereof.

[0025] In the picture:

[0026] Figure 1 This is a flowchart illustrating one model training method described in this specification.

[0027] Figure 2 This is a flowchart illustrating one model training method described in this specification.

[0028] Figure 3 This is a schematic diagram of a model training device provided in this specification;

[0029] Figure 4 The corresponding information provided in this specification Figure 1 A schematic diagram of an electronic device. Detailed Implementation

[0030] To make the objectives, technical solutions, and advantages of this specification clearer, the technical solutions of this specification will be clearly and completely described below in conjunction with specific embodiments and corresponding drawings. Obviously, the described embodiments are only a part of the embodiments of this specification, and not all of them. Based on the embodiments in this specification, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this specification.

[0031] Additionally, it should be noted that all actions involving the acquisition of signals, information, or data in this manual are performed in accordance with the relevant data protection regulations and policies of the locality and with authorization from the owner of the relevant device.

[0032] The technical solutions provided in the various embodiments of this specification are described in detail below with reference to the accompanying drawings.

[0033] Figure 1 This is a flowchart illustrating a model training method provided in this specification.

[0034] S100: Obtain the source model that has been pre-trained using poisoned samples.

[0035] The model training method provided in the embodiments of this specification can be executed by electronic devices such as servers used for model training (the process of optimizing network parameters). Furthermore, this specification also involves the usage of the source model and the target model; the electronic devices used in these processes and the electronic devices executing the model training method can be the same or different, and this specification does not impose any limitations on this.

[0036] In recent years, with the development of machine learning, machine learning models have been widely applied in various practical scenarios, such as autonomous driving, facial recognition, and speech recognition. Especially in risk control scenarios, pre-trained risk identification models can be used to identify user transaction data to determine whether a user's transaction is risky, thereby providing risk warnings to the user and preventing asset losses from unsafe transactions.

[0037] However, with the development of artificial intelligence technology, machine learning also faces various types of security issues. For example, attackers can construct poisoned samples by intentionally feeding incorrect or biased samples during the training sample acquisition phase, i.e., data poisoning or label poisoning. The model uses the poisoned samples during training or retraining, resulting in the embedding of backdoor triggers in the trained model. This causes the poisoned model to ignore the features of the input data and directly output the attacker's preset incorrect output results when judging samples or triggers designed by the attacker, thereby reducing the accuracy of the model's output.

[0038] The target model obtained using the model training method provided in this specification is a defense against the aforementioned attack methods. Therefore, the source model obtained in this step is pre-trained based on poisoned samples. The attacker can poison the training samples of the source model during either the initial training process or the retraining process (i.e., updating and iterating the source model after training and deployment by collecting user input and feedback on the output as retraining samples). Alternatively, the attacker can poison the training samples of the source model during both training processes; this specification does not limit this.

[0039] Furthermore, this manual does not limit the data processing tasks performed by the source model or the applicable application scenarios. The source model can be a pedestrian intent recognition model or route planning model in the field of autonomous driving, a face recognition model or liveness detection model in the field of face recognition, a speech-to-text model in the field of speech recognition, or a transaction risk recognition model in the risk control scenario. Therefore, this manual does not limit the model structure, accuracy, and performance of the source model.

[0040] The poisoned sample can be either a sample modified and poisoned by an attacker, or a sample whose labels are modified and poisoned by an attacker, specifically the training samples of the source model. Modifying and poisoning a sample means altering or adding triggers to the training samples of the source model to construct a poisoned sample, without changing the labels of the original training samples; that is, modifying the original sample x... originalModified to poisoned sample x poison Afterwards, poisoned sample x poison The corresponding label is the original sample x. original The original label original The method of "poisoning" by modifying the labels of samples refers to modifying or changing the labels of the original training samples of the source model. For example, for the original training sample 'a', its original label is 'label'. a To poison the label of the original training sample a, we can modify the label of the original training sample a from the original label. a Modify the label to the training sample b of the source model. b Or a label constructed by the attacker. c Among them, label a With label b and label c They are all different. This instruction manual does not limit the type, quantity, or construction method of the poisoned samples.

[0041] For example, in the field of transaction risk control, the source model is a machine learning model used to identify the risk of a user's transactions. The input to the source model can be transaction data for a single user transaction, including the transaction time, the name and type of the goods traded, the transaction amount, and the merchant's qualifications. The output of the source model can be the probability that the user's transaction carries risk. An attacker could poison this source model by reporting that the transaction is risk-free when the source model outputs a high probability of risk, or by reporting that the transaction is risky when the source model outputs a low probability of risk. In this way, during the process of collecting user feedback as retraining samples for model parameter updates, the source model might use samples with incorrect feedback results, leading to parameter shifts and errors, and thus incorrectly identifying the risk probability of the transaction data.

[0042] Furthermore, in practical applications, to promptly identify whether a trained source model has been poisoned during training due to the introduction of poisoned samples, the training samples can be thoroughly checked. If a mismatch exists between the training sample and its label, the source model is determined to have been trained using poisoned samples. Alternatively, the accuracy or performance (the degree of matching between the model's input and output) of the already deployed source model can be monitored. If the accuracy of the source model's output differs significantly from the accuracy during the training and testing phases, or if the accuracy for feature inputs is low, the source model is determined to have been trained using poisoned samples. Of course, any existing poisoning attack detection method can also be used; this specification does not limit this approach.

[0043] S102: Obtain the transformation network to be trained, and determine the target model to be trained based on the transformation network and the source model.

[0044] In practical applications, to prevent attacked models from being used for extended periods and causing large-scale output errors, defenses can be implemented against the attacked models. This specification employs data preprocessing and output defense methods, combining a transformation network and the source model to obtain the target model. By transforming the inputs of the source model, the author prevents the inputs from triggering the triggers that would otherwise be inserted into the source model. Furthermore, by transforming the outputs of the source model, the author corrects any potential erroneous outputs.

[0045] The transformation network can transform the input and / or output of the source model. By transforming the source model, it can prevent the source model from having pre-set triggers in its input, or correct the source model's output for erroneous results that may be obtained by the source model based on the attacker's pre-set triggers.

[0046] In this specification, the purpose of adding a transformation network to the source model is to preprocess the input of the source model and / or post-process the output of the source model, thereby avoiding triggering in the source model or correcting erroneous output results. The transformation network can perform preprocessing on the input, such as rotation, noise addition / denoising, and resolution adjustment. It can also extract features from the input data and regenerate transformed data that conforms to the input format of the source model based on these features. Similarly, the transformation network can perform post-processing on the output, such as rotation, noise addition / denoising, and resolution adjustment. It can also extract features from the output data of the source model and regenerate transformed data based on these features.

[0047] Optionally, the data processing task performed by the target model obtained based on the transformation network and the source model can be similar to or different from the data processing task performed by the source model. That is, the target domain where the input and output of the target model reside can be the same as or different from the source domain where the input and output of the source model reside; this specification does not impose any limitations on this. Furthermore, when the source domain and the target domain are different, the model training method provided in this specification, in addition to achieving the purpose of resisting poisoning attacks and improving model performance, can also be applied to cross-domain machine learning. Compared with transfer learning, it greatly reduces the scale of model parameters that need to be adjusted and improves the training speed of the model.

[0048] For example, a transformation network is used to transform the input and output of the source model; that is, the target model's structure is a concatenation of a first transformation network, the source model, and a second transformation network. Assuming the source model performs a data processing task of generating images from text, and the target model performs a data processing task of generating images from speech, when a prediction request is received, the data to be predicted corresponding to the prediction request is input into the trained target model to obtain the prediction result of the data to be predicted output by the target model. Specifically, this includes:

[0049] When a prediction request is received, the speech to be predicted is parsed from the prediction request, the speech to be predicted is input into the target model, and the text corresponding to the speech to be predicted is obtained through the first transformation network included in the target model; the text corresponding to the speech to be predicted is input into the source model to obtain the initial image corresponding to the speech to be predicted output by the source model; the initial image corresponding to the speech to be predicted output by the source model is input into the second transformation network included in the target model to obtain the target image corresponding to the speech to be predicted output by the second transformation network.

[0050] Furthermore, it is understandable that the model parameters of the transform network obtained in this step are initialized and unoptimized, i.e., the transform network to be trained. By using the transform network to be trained and the pre-trained source model to determine the target model, the parameters of the transform network and the data processing tasks it can perform can be dynamically adjusted based on the data processing tasks (face recognition, speech recognition, risk recognition, etc.) performed by the source model. Compared to readjusting the parameters of the source model, this method of defending against attacks can significantly reduce the scale of model parameters that need to be adjusted, thereby improving the efficiency of defense.

[0051] S104: Obtain a clean sample and the label of the clean sample.

[0052] In this step, the obtained clean samples and their labels form the training set for training the transformation network. Generally, the transformation network transforms the input and / or output of the source model. Therefore, based on the data processing task implemented by the transformation network and the source model, the data processing task implemented by the target model can be determined. Based on the data processing task implemented by the target model, the input and output formats of the target model can be determined, and thus, based on the input and output formats of the target model, clean samples and their labels can be obtained.

[0053] In practical applications, the data processing tasks implemented by the target model generally fall into the following two categories:

[0054] One reason is that the data processing tasks implemented by the target model are different from those implemented by the source model.

[0055] For example, the source model is used to generate images from text, and the transform network is used to obtain text from speech recognition. That is, the transform network is used to transform the input of the source model. The data processing task achieved by the target model based on the transform network and the source model is to generate images from speech. The input format of the target model is speech, and the output format is image. Therefore, the clean sample is speech, and the label of the clean sample is image.

[0056] Secondly, the data processing tasks implemented by the target model are the same as those implemented by the source model.

[0057] For example, the source model is used to perform face recognition on an image, and the transform network is used to denoise the image. That is, the transform network is used to transform the input of the source model. The data processing task achieved by the target model based on the transform network and the source model is face recognition based on an image. The input format of the target model is an image, and the output format is the face recognition result. Therefore, the clean sample is the image, and the label of the clean sample is the face recognition result.

[0058] S106: Input the clean sample into the target model to obtain the prediction result of the clean sample output by the target model.

[0059] Generally, if the transformation network transforms the input of the source model, the target model's structure is a concatenation of the transformation network and the source model. In this case, when clean samples are input into the target model, the clean samples become the input of the transformation network. By passing through the transformation network and the source model in sequence, the prediction result output by the target model is obtained.

[0060] If the transformation network transforms the output of the source model, the target model's structure is a concatenation of the source model and the transformation network. When clean samples are input into the target model, they become the input of the source model. The target model's prediction results are obtained by passing the clean samples through the source model and the transformation network in sequence.

[0061] If the transformation network transforms both the input and output of the source model, then the target model's structure is a series structure of the first transformation network, the source model, and the second transformation network. In this case, when a clean sample is input into the target model, the clean sample becomes the input of the first transformation network. The sample passes through the first transformation network, the source model, and the second transformation network in sequence to obtain the prediction result output by the target model.

[0062] Furthermore, clean samples refer to training samples that have not undergone data poisoning; that is, clean samples and their labels are mutually matched. To avoid contaminating the clean samples obtained in this step with poisoned samples, the obtained samples can be tested to screen out poisoned samples. The detection method can be to input the obtained samples into a reference model without backdoors, obtain the results corresponding to the samples output by the reference model, and based on the differences between the reference model's output and the sample labels, samples with significant differences are considered poisoned samples and removed from the obtained samples, leaving the remaining samples as clean samples. Of course, the method for removing poisoned samples from the obtained samples can also be based on manual screening, which is not limited in this specification.

[0063] S108: Based on the prediction results and labels of the clean samples, adjust the parameters of the transformation network to obtain the trained target model.

[0064] Specifically, the loss can be determined based on the difference between the prediction results of clean samples and the labels of clean samples, and the parameters of the transformation network can be adjusted by existing gradient optimization methods such as stochastic gradient descent with the goal of minimizing the loss.

[0065] However, in practical applications, the source model, as a pre-trained model provided by a third party, can be a black box model. The gradient-based optimization method mentioned above may not be applicable to the black box model. In this case, the zero-order optimization method can be used to obtain the estimated gradient through sampling and perturbation. Then, the gradient optimization method can be used based on the estimated gradient to optimize the parameters of the transformation network.

[0066] The parameters of the transformation network can be obtained through multiple iterations of optimization. The condition for stopping the iteration optimization can be that the loss is less than a preset loss threshold, or the number of iterations is higher than a preset number threshold. Of course, there can be other conditions as well, but this manual does not limit them.

[0067] S110: When a prediction request is received, the data to be predicted corresponding to the prediction request is input into the trained target model to obtain the prediction result of the data to be predicted output by the target model.

[0068] After the target model has been trained (and the parameters of the transformation network have been optimized), it can be deployed and run. When a prediction request is received, a prediction task can be performed based on the trained target model. Specifically, the data to be predicted is parsed from the prediction request, input into the target model, and based on the structure of the target model, the prediction result of the data to be predicted is obtained by transforming the network and the source model.

[0069] Since the target model contains a transformation network, even if the source model is trained based on poisoned samples (subjected to poisoning attacks), the transformation network can transform the input and / or output of the source model to eliminate triggers that may exist in the input of the source model, avoid triggering backdoors implanted in the source model, or correct erroneous results that may exist in the output of the source model, and avoid directly outputting erroneous results obtained from the source model.

[0070] The model training method provided in this specification determines the target model to be trained based on the transformation network to be trained and the source model trained with poisoned samples. The obtained clean samples are input into the target model to obtain the prediction results output by the target model. Based on the prediction results and the labels of the clean samples, the parameters of the transformation network are adjusted so that when a prediction request is received, the data to be predicted is input into the trained target model to obtain the prediction results for the data to be predicted.

[0071] It is evident that the method of reconstructing the source model based on the transformation network transforms the input and / or output of the source model attacked by the poisoned sample without adjusting the model parameters of the source model. This prevents the source model from obtaining incorrect prediction results for the samples preset by the attacker, thereby achieving a high defense effect while ensuring model performance. Furthermore, training only the transformation network without adjusting the model parameters of the source model greatly reduces the scale of parameters that need to be adjusted, thus reducing the model training time and improving defense efficiency.

[0072] In one or more embodiments of this specification, the data processing tasks performed by the target model obtained based on the transformation network and the source model may be different from those performed by the source model. That is, the target model trained by the model training method provided in this specification can be applied to cross-domain machine learning. Compared with transfer learning, it greatly reduces the scale of model parameters that need to be adjusted and improves the training speed of the model.

[0073] Specifically, in cases where a transformation network transforms the input of the source model, the input of the target model is the input of the transformation network, and the output of the target model is the output of the source model. The transformation network transforms the input of the target model into data that can be input into the source model (including transforming the input format and scale). For example, suppose the data processing task performed by the source model is a text-to-image model; then the input format of the source model is text, and the output format is an image. To avoid triggers (character triggers, sentence triggers, grammatical triggers, or semantic style triggers) in the text input to the source model, a transformation network can be introduced that transforms the data processing task into speech-to-text generation. By generating text from the input speech, triggers are avoided in the generated text, thus resisting poisoning attacks and enabling a cross-domain transition from text-to-image generation to speech-to-image generation.

[0074] In the case of a transformation network that transforms the output of a source model, the input of the source model is the input of the target model, and the output of the transformation network is the output of the target model. By transforming the output of the source model into data applicable to other domains, the data processing tasks performed by the target model differ from those performed by the source model.

[0075] In the case where the transformation network transforms the input and output of the source model, the target model consists of a first transformation network, a source model, and a second transformation network. The function of the first transformation network is similar to that of the transformation network transforming the input of the source model, and the function of the second transformation network is similar to that of the transformation network transforming the output of the source model. These will not be elaborated here.

[0076] In one or more embodiments of this specification, in Figure 1 In step S102, the acquired transformation network can transform the input and / or output of the source model to eliminate attacker-preset triggers that may exist in the input of the source model, or to correct erroneous results obtained by the source model based on attacker-preset triggers that may exist in the output of the source model. Therefore, the transformation network includes a first transformation network and / or a second transformation network, wherein the first transformation network is used to transform the input of the target model, and the second transformation network is used to transform the output of the source model. In this specification, target models with different structures can be obtained according to different transformation networks, and further, based on target models with different structures... Figure 1 The training process of the target model shown in step S06 also varies, and can be divided into the following three cases.

[0077] The first case: The transformation network includes the first transformation network.

[0078] In this case, step S102 specifically involves: concatenating the first transformation network and the source model to obtain the target model to be trained; wherein the output of the first transformation network is the input of the source model, and the output of the source model is the output of the target model.

[0079] Furthermore, step S106 specifically involves: taking the minimization of the difference between the prediction result of the clean sample and the label of the clean sample as the optimization objective, adjusting the parameters of the first transformation network to obtain the trained target model.

[0080] Poisoning attacks can implant backdoors into the source model during training, causing the trained model to produce specific (incorrect) predictions when triggers are present in the input, and output normal predictions when no triggers are present. In this case, a transformation network preprocesses the input that should have been directly input into the source model. Generally, the data obtained through the transformation network does not contain artificially set triggers, thus rendering the poisoning attack ineffective; that is, eliminating the possibility of triggers in the input that should have been directly input into the source model.

[0081] For example, the source model is used to recognize text in an image. During training, a backdoor is implanted in this model. When the input image contains the asterisk (*), the model ignores the image's features and directly outputs the text "8". A transform network is then added before the source model. The transform network extracts image features and regenerates an image containing the printed text. In this case, the input image to the transform network contains the asterisk (*) trigger. After the transform network regenerates the image, it may not contain the asterisk. Consequently, the input to the source model does not contain the trigger, thus preventing the implanted backdoor from being triggered. Based on this method, even if the source model is trained on poisoned samples, the transform network can eliminate any potential triggers in the input, thereby bypassing the backdoor and resisting poisoning attacks.

[0082] The second case: The transformation network includes a second transformation network.

[0083] In this case, step S102 specifically involves: concatenating the source model and the second transformation network to obtain the target model to be trained; wherein the output of the source model is the input of the second transformation network. The input of the source model is the input of the target model, i.e., clean samples.

[0084] Furthermore, step S106 specifically involves: taking the minimization of the difference between the prediction result of the clean sample and the label of the clean sample as the optimization objective, adjusting the parameters of the second transformation network to obtain the trained target model.

[0085] In practical applications, poisoning attacks refer to attackers modifying the training samples (or labels) of the source model to construct poisoned samples, causing the trained source model to produce incorrect predictions for certain samples pre-set by the attacker. To address this, a transformation network can be added after the source model's output. The source model's output is used as input to the transformation network to transform the output and correct any potential errors. Even if the source model produces an incorrect output based on inputs with triggers, this incorrect output will not be directly used as the final prediction. Instead, the transformation network will correct the incorrect output to a correct one, thus preventing a decrease in model accuracy.

[0086] The third scenario: The transformation network includes a first transformation network and a second transformation network.

[0087] In this case, step S102 specifically involves: sequentially connecting the first transformation network, the source model, and the second transformation network to obtain the target model to be trained;

[0088] Furthermore, step S106 specifically involves: taking the minimization of the difference between the prediction result of the clean sample and the label of the clean sample as the optimization objective, adjusting the parameters of the first transformation network and the second transformation network to obtain the trained target model.

[0089] Specifically, in this case, the target model's structure consists of a first transformation network, a source model, and a second transformation network connected in series. That is, the input to the target model is the input to the first transformation network, the input to the source model is the output of the first transformation network, the input to the second transformation network is the output of the source model, and the output of the second transformation network is the output of the target model. Based on this target model structure, the input to the source model is essentially a clean sample transformed by the first transformation network, and the output of the source model still needs to be transformed by the second transformation network. Thus, the first transformation network can eliminate triggers that may be present in the source model input, while the second transformation network can correct errors that may exist in the source model output, thereby further improving model accuracy and resisting poisoning attacks.

[0090] Furthermore, the first transformation network transforms clean samples to obtain data conforming to the input format of the source model, and the second transformation network transforms the output of the source model to obtain the output of the target model. Therefore, the data processing task performed by the target model, which includes both the first and second transformation networks, can differ from the data processing task performed by the source model. That is, the target domain of the target model's input and output differs from the source domain of the source model's input and output. Based on this, constructing a target model not only solves the problem of the source model being susceptible to poisoning attacks but also enables cross-domain tasks. Without retraining the model, cross-domain machine learning tasks can be performed based on a pre-trained source model.

[0091] Furthermore, in the third scenario described above, during the adjustment of the parameters of the first and second transformation networks, source samples used for training the source model can be introduced. The first and second transformation networks are then treated as independent transformation networks, and their losses are determined separately. Subsequently, based on the losses determined by the first and second transformation networks, the parameters of the first and second transformation networks are adjusted respectively to obtain the target model with optimized parameters. Figure 2 As shown, the specific solution is as follows:

[0092] S200: Obtain source samples for training the source model.

[0093] In this step, the source samples obtained are used to train the source model. The source samples can be a training set containing poisoned samples or a training set that has removed poisoned samples. This specification does not limit this.

[0094] Generally, the type of source samples used to train the source model is determined based on the data processing task performed by the source model. For example, if the source model is an image processing model, then the source samples are images; if the source model is a speech recognition model, then the source samples are speech.

[0095] S202: Input the clean sample into the first transformation network to obtain the transformation result output by the first transformation network.

[0096] S204: Input the first transformation result into the source model to obtain the reference result output by the source model.

[0097] S206: Input the reference result into the second transformation network to obtain the prediction result of the clean sample output by the second transformation network.

[0098] S208: Determine the first loss based on the difference between the transformation result and the source sample.

[0099] In the target model, the model parameters that actually need to be adjusted are the first transformation network and the second transformation network. Therefore, the source samples used to train the source model can be used as the labels for training the first transformation network, and the labels of the clean samples can be used as the labels for training the second transformation network.

[0100] Therefore, in this step, the first loss is determined based on the transformation result (output of the first transformation network) and the source samples (labels used to train the first transformation network), and in subsequent steps, the second loss is determined based on the prediction result (output of the second transformation network) and the labels of the clean samples (labels used to train the second transformation network).

[0101] S210: Determine a second loss based on the difference between the prediction result and the label of the clean sample.

[0102] S212: Based on the first loss and the second loss, adjust the parameters of the first transformation network and the second transformation network respectively to obtain the trained target model.

[0103] Specifically, the total loss can be determined based on the first loss and the second loss, and the optimization objective is to minimize the total loss. The first and second transformation networks are then trained jointly; that is, the parameters of both networks are adjusted simultaneously to minimize the total loss. Alternatively, the parameters of the first transformation network can be adjusted to minimize the first loss, while the parameters of the second transformation network can be adjusted to minimize the second loss; in other words, the first and second transformation networks are trained separately.

[0104] Optionally, the labels of the source samples can also be obtained. During training, based on the reference results obtained by the source model when the input is the transformation result of the first transformation network, and the differences between the labels of the source samples, a third loss is determined. Then, the total loss is determined based on the first loss, the second loss, and the third loss, and the parameters of the first transformation network and the second transformation network are adjusted with the minimization of the total loss as the training objective. Since the source model is pre-trained, its accuracy is guaranteed in most cases even if it is subjected to a poisoning attack. Therefore, the source model can be used as the model with higher accuracy during the training of the first transformation model and the second transformation model.

[0105] Furthermore, in practical applications, the source model can be a large pre-trained model (from a third party), such as LLM, LIM, GPT, etc. However, the model structure and training process of third-party pre-trained models are opaque, meaning the source model is a black box. In this case, the source model is usually provided in the form of an interface; that is, the user of the source model can only input data into the source model to obtain its output, without knowing the intermediate results. Therefore, it is impossible to defend against security threats during the training process by modifying the model, nor is it possible to obtain the gradient information in the intermediate steps of the source model. To address this, this specification employs zeroth-order optimization (ZO) to estimate the gradient by analyzing the differences between the outputs of the source model for different inputs, and then uses gradient descent to optimize the parameters of the transformation network based on the estimated gradient.

[0106] Based on this, the transformation network in the target model can be further optimized and adjusted using the following methods:

[0107] Step 1: Obtain multiple perturbation values.

[0108] Step 2: Determine each perturbation sample based on the clean sample and each perturbation value.

[0109] Step 3: Input each perturbation sample into the target model to obtain the prediction results of each perturbation sample output by the target model.

[0110] Step 4: Determine the gradient based on the differences between the prediction results of each perturbation sample.

[0111] Step 5: Adjust the parameters of the transformation network according to the gradient to obtain the trained target model.

[0112] Figure 3 A schematic diagram of a model training device provided in this specification specifically includes:

[0113] The source model acquisition module 300 is used to acquire the source model that has been pre-trained using poisoned samples;

[0114] The target model determination module 302 is used to obtain the transformation network to be trained and determine the target model to be trained based on the transformation network and the source model.

[0115] Clean sample acquisition module 304 is used to acquire clean samples and the labels of the clean samples;

[0116] Input module 306 is used to input the clean sample into the target model and obtain the prediction result of the clean sample output by the target model;

[0117] The first training module 308 is used to adjust the parameters of the transformation network with the optimization objective of minimizing the difference between the prediction result of the clean sample and the label of the clean sample, so as to obtain the trained target model.

[0118] The prediction module 310 is used to input the data to be predicted corresponding to the prediction request into the trained target model when a prediction request is received, and to obtain the prediction result of the data to be predicted output by the target model.

[0119] Optionally, the transform network includes a first transform network;

[0120] Optionally, the target model determination module 302 is specifically used to concatenate the first transformation network and the source model to obtain the target model to be trained; wherein the output of the first transformation network is the input of the source model;

[0121] Optionally, the first training module 308 is specifically used to adjust the parameters of the first transformation network with the optimization objective of minimizing the difference between the prediction result of the clean sample and the label of the clean sample, so as to obtain the trained target model.

[0122] Optionally, the transformation network includes a second transformation network;

[0123] Optionally, the target model determination module 302 is specifically used to concatenate the source model and the second transformation network to obtain the target model to be trained; wherein the output of the source model is the input of the second transformation network;

[0124] Optionally, the first training module 308 is specifically used to adjust the parameters of the second transformation network with the optimization objective of minimizing the difference between the prediction result of the clean sample and the label of the clean sample, so as to obtain the trained target model.

[0125] Optionally, the transformation network includes a first transformation network and a second transformation network;

[0126] Optionally, the target model determination module 302 is specifically used to connect the first transformation network, the source model and the second transformation network in series to obtain the target model to be trained;

[0127] Optionally, the first training module 308 is specifically used to adjust the parameters of the first transformation network and the second transformation network with the optimization objective of minimizing the difference between the prediction result of the clean sample and the label of the clean sample, so as to obtain the trained target model.

[0128] Optionally, the device further includes:

[0129] The second training module 312 is specifically used to acquire source samples for training the source model; input the clean samples into the first transformation network to obtain the transformation result output by the first transformation network; input the transformation result into the source model to obtain the reference result output by the source model; input the reference result into the second transformation network to obtain the prediction result of the clean samples output by the second transformation network; determine a first loss based on the difference between the transformation result and the source samples; determine a second loss based on the difference between the prediction result and the label of the clean samples; and adjust the parameters of the first transformation network and the second transformation network according to the first loss and the second loss respectively to obtain the trained target model.

[0130] Optionally, the source model is a black-box model;

[0131] Optionally, the device further includes:

[0132] The third training module 314 is specifically used to acquire multiple perturbation values; determine each perturbation sample based on the clean sample and each perturbation value; input each perturbation sample into the target model respectively to obtain the prediction result of each perturbation sample output by the target model; determine the gradient based on the difference between the prediction results of each perturbation sample; adjust the parameters of the transformation network based on the gradient to obtain the trained target model.

[0133] Optionally, the data processing task performed by the target model is different from the data processing task performed by the source model.

[0134] This specification also provides a computer-readable storage medium storing a computer program that can be used to execute the above-described... Figure 1 The model training method shown.

[0135] This instruction manual also provides Figure 4 The diagram shows a schematic structural representation of the electronic device. Figure 4 At the hardware level, the electronic device includes a processor, internal bus, network interface, memory, and non-volatile memory, and may also include other hardware required for the business operations. The processor reads the corresponding computer program from the non-volatile memory into memory and then runs it to achieve the above-mentioned functions. Figure 1 The model training method is shown. Of course, in addition to the software implementation, this specification does not exclude other implementation methods, such as logic devices or a combination of hardware and software, etc. In other words, the execution subject of the following processing flow is not limited to individual logic units, but can also be hardware or logic devices.

[0136] In the 1990s, improvements to a technology could be clearly distinguished as either hardware improvements (e.g., improvements to the circuit structure of diodes, transistors, switches, etc.) or software improvements (improvements to the methodology). However, with technological advancements, many methodological improvements today can be considered direct improvements to the hardware circuit structure. Designers almost always obtain the corresponding hardware circuit structure by programming the improved methodology into the hardware circuit. Therefore, it cannot be said that a methodological improvement cannot be implemented using hardware physical modules. For example, a Programmable Logic Device (PLD) (such as a Field Programmable Gate Array (FPGA)) is such an integrated circuit whose logic function is determined by the user programming the device. Designers can program and "integrate" a digital system onto a PLD themselves, without needing chip manufacturers to design and manufacture dedicated integrated circuit chips. Furthermore, nowadays, instead of manually manufacturing integrated circuit chips, this programming is mostly implemented using "logic compiler" software. Similar to the software compiler used in program development, the original code before compilation must be written in a specific programming language, called a Hardware Description Language (HDL). There are many HDLs, such as ABEL (Advanced Boolean Expression Language), AHDL (Altera Hardware Description Language), Confluence, CUPL (Cornell University Programming Language), HDCal, JHDL (Java Hardware Description Language), Lava, Lola, MyHDL, PALASM, and RHDL (Ruby Hardware Description Language). Currently, the most commonly used are VHDL (Very-High-Speed ​​Integrated Circuit Hardware Description Language) and Verilog. Those skilled in the art should understand that by simply performing some logic programming on the method flow using one of these hardware description languages ​​and programming it into an integrated circuit, the hardware circuit implementing the logical method flow can be easily obtained.

[0137] The controller can be implemented in any suitable manner. For example, it can take the form of a microprocessor or processor and a computer-readable medium storing computer-readable program code (e.g., software or firmware) executable by the (micro)processor, logic gates, switches, application-specific integrated circuits (ASICs), programmable logic controllers, and embedded microcontrollers. Examples of controllers include, but are not limited to, the following microcontrollers: ARC 625D, Atmel AT91SAM, Microchip PIC18F26K20, and Silicon Labs C8051F320. A memory controller can also be implemented as part of the control logic of the memory. Those skilled in the art will also recognize that, in addition to implementing the controller in purely computer-readable program code form, the same functionality can be achieved by logically programming the method steps to make the controller take the form of logic gates, switches, application-specific integrated circuits, programmable logic controllers, and embedded microcontrollers. Therefore, such a controller can be considered a hardware component, and the means included therein for implementing various functions can also be considered as structures within the hardware component. Alternatively, the means for implementing various functions can be considered as both software modules implementing the method and structures within the hardware component.

[0138] The systems, devices, modules, or units described in the above embodiments can be implemented by computer chips or entities, or by products with certain functions. A typical implementation device is a computer. Specifically, a computer can be, for example, a personal computer, laptop computer, cellular phone, camera phone, smartphone, personal digital assistant, media player, navigation device, email device, game console, tablet computer, wearable device, or any combination of these devices.

[0139] For ease of description, the above devices are described in terms of function, divided into various units. Of course, in implementing this specification, the functions of each unit can be implemented in one or more software and / or hardware components.

[0140] Those skilled in the art will understand that embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0141] This specification is described with reference to flowchart illustrations and / or block diagrams of methods, apparatus (systems), and computer program products according to embodiments of this specification. It will be understood that each block of the flowchart illustrations and / or block diagrams, and combinations of blocks in the flowchart illustrations and / or block diagrams, can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, special-purpose computer, embedded processor, or other programmable data processing apparatus to produce a machine, such that the instructions, which execute via the processor of the computer or other programmable data processing apparatus, create a machine for implementing the flowchart illustrations and / or block diagrams. Figure 1 One or more processes and / or boxes Figure 1 A device that provides the functions specified in one or more boxes.

[0142] These computer program instructions may also be stored in a computer-readable storage medium that can direct a computer or other programmable data processing device to function in a particular manner, such that the instructions stored in the computer-readable storage medium produce an article of manufacture including instruction means, which are implemented in a process Figure 1 One or more processes and / or boxes Figure 1 The function specified in one or more boxes.

[0143] These computer program instructions may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps of the function specified in one or more boxes.

[0144] In a typical configuration, a computing device includes one or more processors (CPU), input / output interfaces, network interfaces, and memory.

[0145] Memory may include non-persistent storage in computer-readable media, such as random access memory (RAM) and / or non-volatile memory, such as read-only memory (ROM) or flash RAM. Memory is an example of computer-readable media.

[0146] Computer-readable media includes both permanent and non-permanent, removable and non-removable media that can store information using any method or technology. Information can be computer-readable instructions, data structures, modules of programs, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic magnetic disk storage or other magnetic storage devices, or any other non-transferable medium that can be used to store information accessible by a computing device. As defined herein, computer-readable media does not include transient computer-readable media, such as modulated data signals and carrier waves.

[0147] It should also be noted that the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitation, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0148] Those skilled in the art will understand that the embodiments of this specification can be provided as methods, systems, or computer program products. Therefore, this specification may take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Furthermore, this specification may take the form of a computer program product embodied on one or more computer-usable storage media (including, but not limited to, disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0149] This specification can be described in the general context of computer-executable instructions that are executed by a computer, such as program modules. Generally, program modules include routines, programs, objects, components, data structures, etc., that perform a specific task or implement a specific abstract data type. This specification can also be practiced in distributed computing environments, where tasks are performed by remote processing devices connected via a communication network. In distributed computing environments, program modules can reside in local and remote computer storage media, including storage devices.

[0150] The various embodiments in this specification are described in a progressive manner. Similar or identical parts between embodiments can be referred to interchangeably. Each embodiment focuses on describing the differences from other embodiments. In particular, the system embodiments are basically similar to the method embodiments, so the description is relatively simple; relevant parts can be referred to the descriptions in the method embodiments.

[0151] The above description is merely an embodiment of this specification and is not intended to limit this specification. Various modifications and variations can be made to this specification by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of this specification should be included within the scope of the claims of this specification.

Claims

1. A model training method, the method comprising: Obtain the source model pre-trained using poisoned samples; A transformation network to be trained is obtained, and a target model to be trained is determined based on the transformation network and the source model; the transformation network is used to transform the format of the input of the source model to avoid the presence of attacker-preset triggers in the input of the source model. Obtain clean samples and their labels; The clean sample is input into the target model to obtain the prediction result of the clean sample output by the target model; Based on the prediction results and labels of the clean samples, the parameters of the transformation network are adjusted to obtain the trained target model. When a prediction request is received, the data to be predicted corresponding to the prediction request is input into the trained target model to obtain the prediction result of the data to be predicted output by the target model.

2. The method of claim 1, wherein the transformation network comprises a first transformation network; The target model to be trained is determined based on the transformation network and the source model, specifically including: The first transformation network and the source model are concatenated to obtain the target model to be trained; wherein the output of the first transformation network is the input of the source model. Based on the prediction results and labels of the clean samples, the parameters of the transformation network are adjusted to obtain the trained target model, specifically including: With the optimization objective of minimizing the difference between the prediction result of the clean sample and the label of the clean sample, the parameters of the first transformation network are adjusted to obtain the trained target model.

3. The method of claim 1, wherein the transformation network comprises a second transformation network; The target model to be trained is determined based on the transformation network and the source model, specifically including: The source model and the second transformation network are concatenated to obtain the target model to be trained; wherein the output of the source model is the input of the second transformation network. Based on the prediction results and labels of the clean samples, the parameters of the transformation network are adjusted to obtain the trained target model, specifically including: With the optimization objective of minimizing the difference between the prediction result of the clean sample and the label of the clean sample, the parameters of the second transformation network are adjusted to obtain the trained target model.

4. The method as described in claim 1, wherein the transformation network comprises a first transformation network and a second transformation network; The target model to be trained is determined based on the transformation network and the source model, specifically including: The first transformation network, the source model, and the second transformation network are sequentially connected in series to obtain the target model to be trained. Based on the prediction results and labels of the clean samples, the parameters of the transformation network are adjusted to obtain the trained target model, specifically including: With the optimization objective of minimizing the difference between the prediction results of the clean samples and the labels of the clean samples, the parameters of the first transformation network and the second transformation network are adjusted to obtain the trained target model.

5. The method of claim 4, further comprising: Obtain source samples for training the source model; The clean sample is input into the first transformation network to obtain the transformation result output by the first transformation network; The transformation result is input into the source model to obtain the reference result output by the source model; The reference result is input into the second transformation network to obtain the prediction result of the clean sample output by the second transformation network; The first loss is determined based on the difference between the transformation result and the source sample; A second loss is determined based on the difference between the prediction result and the label of the clean sample; Based on the first loss and the second loss, the parameters of the first transformation network and the second transformation network are adjusted respectively to obtain the trained target model.

6. The method as described in claim 1, wherein the source model is a black-box model; The method further includes: Obtain multiple perturbation values; Based on the clean sample and each perturbation value, each perturbation sample is determined; Each perturbation sample is input into the target model to obtain the prediction results of each perturbation sample output by the target model; The gradient is determined based on the differences between the prediction results of each perturbation sample; The parameters of the transformation network are adjusted according to the gradient to obtain the trained target model.

7. The method as described in claim 1, wherein the data processing task performed by the target model is different from the data processing task performed by the source model.

8. A model training apparatus, the apparatus comprising: The source model acquisition module is used to acquire the source model that has been pre-trained using poisoned samples; The target model determination module is used to acquire the transformation network to be trained, and determine the target model to be trained based on the transformation network and the source model; the transformation network is used to perform format transformation on the input of the source model to avoid the presence of attacker-preset triggers in the input of the source model; A clean sample acquisition module is used to acquire clean samples and the labels of the clean samples; The input module is used to input the clean sample into the target model and obtain the prediction result of the clean sample output by the target model; The first training module is used to adjust the parameters of the transformation network with the optimization objective of minimizing the difference between the prediction result of the clean sample and the label of the clean sample, so as to obtain the trained target model. The prediction module is used to input the data to be predicted corresponding to the prediction request into the trained target model when a prediction request is received, and to obtain the prediction result of the data to be predicted output by the target model.

9. A computer-readable storage medium storing a computer program that, when executed by a processor, implements the method described in any one of claims 1 to 7.

10. An electronic device comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the program to implement the method described in any one of claims 1 to 7.

Images