Adversarial training method and device of model, electronic equipment and storage medium

Abstract

The invention provides an adversarial training method and device of a model, electronic equipment and a storage medium, and relates to the field of artificial intelligence, and the method is mainly and technically characterized by comprising the steps: carrying out the multi-dimensional risk assessment of a training sample, and obtaining the risk degrees of different dimensions of the training sample; wherein the multiple dimensions comprise at least one of a model parameter gradient, a similarity with a historical sample and a similarity with a knowledge base, and the training sample is an evaluation and question-answer pair in a large language model interaction process; performing updating calculation on the weight of the training sample according to the risk degrees of the different dimensions to obtain a target training sample; training a scoring model based on the target training sample to obtain a target scoring model; and scoring the generated content of the large language model in the interaction process based on the target scoring model to obtain a scoring result, and adjusting the large language model based on the scoring result until a preset ending condition of large language model training is met.

Description

Technical Field

[0001] This disclosure relates to the field of artificial intelligence, and more particularly to an adversarial training method, apparatus, electronic device, and storage medium for a model. Background Technology

[0002] With the development of artificial intelligence technology, large language models are widely used in various fields due to their natural human-computer interaction capabilities. To adapt to user needs, large language models need to improve their output performance through adaptive updates, and scoring models are the key support for achieving adaptive updates of large language models.

[0003] However, since the training samples of the scoring model are collected from the dialogue between the user and the large language model, they contain contaminated samples after malicious attacks. If the scoring model is trained with contaminated samples, it will reduce the robustness of the scoring model, making its scoring of the content generated by the large language model inaccurate, and thus affecting the adaptive update of the large language model. Summary of the Invention

[0004] This disclosure provides an adversarial training method, apparatus, electronic device, and storage medium for a model. Its main objective is to address the problem in existing technologies where scoring models used for adaptive updates of large language models suffer from low robustness due to contaminated samples in the training dataset, thereby reducing the overall performance of large language models adaptively updated based on the scoring model's results.

[0005] According to a first aspect of this disclosure, an adversarial training method for a model is provided, comprising: A multi-dimensional risk assessment is performed on the training samples to obtain the risk level of the training samples in different dimensions; wherein, the multi-dimensional risk level includes at least one of model parameter gradient, similarity with historical samples, and similarity with the knowledge base, and the training samples are evaluation and question-answer pairs in the interaction process of the large language model; The weights of the training samples are updated and calculated based on the risk levels of the different dimensions to obtain the target training samples; The scoring model is trained based on the target training samples to obtain the target scoring model; The large language model generates content during the interaction process based on the target scoring model to obtain a scoring result. The large language model is then adjusted based on the scoring result until the preset termination condition of the large language model training is met.

[0006] In some embodiments, the different dimensions of risk include at least one of: model parameter gradient risk, similarity risk with historical samples, and similarity risk with the knowledge base; The multi-dimensional risk assessment of the training samples, to obtain the risk levels of the training samples in different dimensions, includes: Calculate the gradient risk of the model parameters corresponding to the training sample based on the gradient of the model parameters and the loss function; The similarity risk with historical samples is calculated based on the similarity between the training samples and each sample in the most recent preset number of historical training samples. The similarity risk level with the knowledge base is calculated based on the similarity between the training sample and each reference sample in the preset knowledge base.

[0007] In some embodiments, updating the weights of the training samples based on the risk levels of the different dimensions to obtain the target training samples includes: The risk levels of the different dimensions are combined to obtain the comprehensive risk level of the training sample; The risk weight of the training sample is calculated based on the comprehensive risk level; wherein the risk weight is negatively correlated with the processed total risk level. The training samples are updated according to the risk weights to obtain the target training samples.

[0008] In some embodiments, after performing a multi-dimensional risk assessment on the training samples to obtain the risk levels of the training samples in different dimensions, the method further includes: The feature overlap and class discrimination of the training samples are calculated to obtain the diversity weight of the training samples.

[0009] In some embodiments, calculating the feature overlap and class discriminancy of the training samples to obtain the diversity weights of the training samples includes: The training samples are mapped to a low-dimensional feature space composed of pre-trained models to obtain sample features; Based on the sample features, the degree of feature overlap of the training samples is determined by calculating the overlapping region of feature values ​​of different categories of samples; The class discrimination of the training samples is determined by calculating the clarity of the classification boundaries of the samples of different categories; The diversity weights of the training samples are determined based on the degree of feature overlap and the class discrimination.

[0010] In some embodiments, updating the training samples according to the risk weights to obtain the target training samples includes: The updated weights of the training samples are obtained by calculating the risk weights and the diversity weights. The training samples are updated based on the updated weights to obtain the target training samples.

[0011] In some embodiments, the step of scoring the content generated by the large language model during the interaction process based on the target scoring model to obtain a scoring result, and adjusting the large language model based on the scoring result until a preset termination condition for training the large language model is met, includes: Obtain the generated content of the large language model during the interaction process; The generated content is input into the target scoring model for scoring, and the scoring result is obtained; The parameters of the large language model are adjusted based on the scoring results; Repeat the steps of performing multi-dimensional risk assessment on training samples to adjusting the large language model based on the scoring results, until the scoring results reach a preset score threshold and / or the number of training iterations of the large language model reaches a preset number.

[0012] In some embodiments, mapping the training samples to a low-dimensional feature space composed of a pre-trained model to obtain sample features includes: The training samples are input into a pre-trained text semantic classification model and a sentiment classification model; Extract the output feature representations of the hidden layers of the text semantic classification model and the sentiment classification model; The output feature representation is determined as the sample feature of the training sample in the low-dimensional feature space.

[0013] According to a second aspect of this disclosure, an adversarial training apparatus for a model is provided, comprising: An evaluation unit is used to perform multi-dimensional risk assessment on training samples to obtain risk levels of the training samples in different dimensions; wherein, the multi-dimensional risk level includes at least one of model parameter gradient, similarity with historical samples, and similarity with the knowledge base, and the training samples are evaluation and question-answer pairs in the interaction process of the large language model. The first calculation unit is used to update the weights of the training samples according to the risk levels of the different dimensions to obtain the target training samples. The training unit is used to train the scoring model based on the target training samples to obtain the target scoring model; The adjustment unit is used to score the content generated by the large language model during the interaction process based on the target scoring model, obtain the scoring result, and adjust the large language model based on the scoring result until the preset termination condition of the large language model training is met.

[0014] In some embodiments, the different dimensions of risk include at least one of: model parameter gradient risk, similarity risk with historical samples, and similarity risk with the knowledge base; The evaluation unit includes: The first calculation module is used to calculate the risk level of the model parameter gradient corresponding to the training sample based on the model parameter gradient and the loss function. The second calculation module is used to calculate the similarity risk degree with historical samples based on the similarity between the training samples and each sample in the most recent preset number of historical training samples. The third calculation module is used to calculate the similarity risk degree with the knowledge base based on the similarity between the training sample and each reference sample in the preset knowledge base.

[0015] In some embodiments, the first computing unit includes: The fourth calculation module is used to perform combined calculations on the risk levels of the different dimensions to obtain the comprehensive risk level of the training sample; The fifth calculation module is used to calculate the risk weight of the training sample based on the comprehensive risk level; wherein the risk weight is negatively correlated with the processed total risk level; An update module is used to update the training samples according to the risk weights to obtain the target training samples.

[0016] In some embodiments, the apparatus further includes: The second calculation unit is used to calculate the feature overlap and category discrimination of the training samples after the evaluation unit performs multi-dimensional risk assessment on the training samples and obtains the risk levels of the training samples in different dimensions, so as to obtain the diversity weight of the training samples.

[0017] In some embodiments, the second computing unit includes: The mapping module is used to map the training samples to a low-dimensional feature space composed of pre-trained models to obtain sample features; The first determining module is used to determine the degree of feature overlap of the training samples by calculating the overlapping region of feature values ​​of different categories of samples based on the sample features. The second determining module is used to determine the class discrimination of the training samples by calculating the classification boundary clarity of the samples of different categories; The third determining module is used to determine the diversity weights of the training samples based on the degree of feature overlap and the class discrimination.

[0018] In some embodiments, the updating module includes: The calculation submodule is used to calculate the risk weight and the diversity weight to obtain the updated weight of the training sample; The update submodule is used to update the training samples based on the update weights to obtain the target training samples.

[0019] In some embodiments, the adjustment unit includes: The acquisition module is used to acquire the generated content of the large language model during the interaction process; The scoring module is used to input the generated content into the target scoring model for scoring and to obtain the scoring result; An adjustment module is used to adjust the parameters of the large language model based on the scoring results; The execution module is used to repeatedly execute the steps of performing multi-dimensional risk assessment on training samples to adjusting the large language model based on the scoring results, until the scoring results reach a preset score threshold, and / or the number of training iterations of the large language model reaches a preset number.

[0020] In some embodiments, the mapping module includes: The input submodule is used to input the training samples into the pre-trained text semantic classification model and sentiment classification model; The extraction submodule is used to extract the output feature representations of the hidden layers of the text semantic classification model and the sentiment classification model; A determination submodule is used to determine the output feature representation as the sample feature of the training sample in the low-dimensional feature space.

[0021] According to a third aspect of this disclosure, an electronic device is provided, comprising: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method described in the first aspect above.

[0022] According to a fourth aspect of this disclosure, a non-transitory computer-readable storage medium is provided storing computer instructions, wherein the computer instructions are configured to cause the computer to perform the method described in the first aspect above.

[0023] According to a fifth aspect of this disclosure, a computer program product is provided, comprising a computer program that, when executed by a processor, implements the method described in the first aspect above.

[0024] In summary, compared with related technologies, the solution disclosed herein can obtain risk levels in different dimensions by performing multi-dimensional risk assessment on the evaluation and question-answer pairs (training samples) during the interaction process of a large language model. Based on the risk levels in different dimensions, the weights of the training samples are updated to obtain target training samples. A scoring model is trained based on the target training samples to obtain a target scoring model. The generated content of the large language model during the interaction process is scored based on the target scoring model, and the large language model is adjusted according to the scoring results until the preset termination conditions are met. This improves the robustness of the scoring model and enhances the overall performance of the large language model after adjustment based on the scoring results of the scoring model during the interaction process.

[0025] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of this application, nor is it intended to limit the scope of this application. Other features of this application will become readily apparent from the following description. Attached Figure Description

[0026] The accompanying drawings are provided to better understand this solution and do not constitute a limitation of this disclosure. Wherein: Figure 1 A flowchart illustrating an adversarial training method for a model provided in an embodiment of this disclosure; Figure 2 A flowchart illustrating an adversarial training method for another model provided in an embodiment of this disclosure; Figure 3 A flowchart illustrating an adversarial training method for another model provided in an embodiment of this disclosure; Figure 4 A flowchart illustrating an adversarial training method for another model provided in an embodiment of this disclosure; Figure 5 A flowchart illustrating an adversarial training method for another model provided in an embodiment of this disclosure; Figure 6 A flowchart illustrating an adversarial training method for another model provided in an embodiment of this disclosure; Figure 7 A flowchart illustrating an adversarial training method for another model provided in an embodiment of this disclosure; Figure 8 A schematic diagram of the overall process of an adversarial training method for a model provided in an embodiment of this disclosure; Figure 9 A schematic diagram of the structure of an adversarial training device for a model provided in an embodiment of this disclosure; Figure 10 A schematic diagram of the structure of an adversarial training device for another model provided in an embodiment of this disclosure; Figure 11 This is a schematic block diagram of an example electronic device provided in an embodiment of this disclosure. Detailed Implementation

[0027] The exemplary embodiments of this disclosure are described below with reference to the accompanying drawings, including various details of the embodiments to aid understanding, and should be considered merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications can be made to the embodiments described herein without departing from the scope and spirit of this disclosure. Similarly, for clarity and brevity, descriptions of well-known functions and structures are omitted in the following description.

[0028] The adversarial training method, apparatus, electronic device, and storage medium of the model according to embodiments of the present disclosure are described below with reference to the accompanying drawings.

[0029] Figure 1 This is a flowchart illustrating an adversarial training method for a model provided in an embodiment of this disclosure.

[0030] like Figure 1 As shown, the method includes steps 101-104.

[0031] Step 101: Perform a multi-dimensional risk assessment on the training samples to obtain the risk level of the training samples in different dimensions; wherein, the multi-dimensional risk level includes at least one of the following: model parameter gradient, similarity with historical samples, and similarity with the knowledge base, and the training samples are evaluation and question-answer pairs in the interaction process of the large language model.

[0032] In some embodiments, training samples originate from the interaction between the user and the large language model question-answering system, including the user's input question, the large language model's generated answer, and the user's evaluation of the answer. These training samples are used for subsequent scoring model training. Multi-dimensional risk assessment aims to evaluate the potential risks of each training sample in multiple aspects to ensure the stability and safety of the scoring model training. The model parameter gradient reflects the degree of influence of the training sample on model parameter updates; a larger gradient value indicates a greater influence of the training sample on the model and a higher risk. Similarity with historical samples refers to the similarity distance between the training sample and each sample in the most recent preset number of historical training samples. A larger similarity distance indicates a greater difference between the training sample and recent data and a higher risk. Similarity with the knowledge base refers to the similarity distance between the training sample and each reference sample in the preset knowledge base. A larger similarity distance indicates a greater difference between the training sample and the reference samples and a higher risk. The calculation of similarity distance can include, but is not limited to, using metrics such as Euclidean distance and cosine similarity.

[0033] The above methods can comprehensively assess the risks of training samples, identify training samples that may adversely affect the scoring model, and provide a basis for subsequent weight updates.

[0034] Step 102: Update the weights of the training samples according to the risk levels of the different dimensions to obtain the target training samples.

[0035] In some embodiments, the weights of training samples are calculated based on the risk levels of different dimensions obtained in step 101. This can be achieved through methods including, but not limited to, directly linking risk level to weight. If a training sample has a high risk level in the model parameter gradient dimension, it indicates that the training sample may cause abnormal changes in the scoring model parameters, thus reducing the weight of the training sample in subsequent training. Conversely, if a training sample has a low risk level in the similarity dimension with the knowledge base, it indicates that the training sample has a high degree of matching with the correct question-answer content, thus increasing the weight of the training sample in subsequent training. After the weight calculation is completed, the training samples are updated based on the calculated weights to obtain the target training sample.

[0036] By using the above method, training samples with lower risk can be selected as target training samples, avoiding the adverse effects of high-risk training samples on subsequent scoring model training and ensuring the quality of training data.

[0037] Step 103: Train the scoring model based on the target training samples to obtain the target scoring model.

[0038] In some embodiments, the scoring model is used to assess the quality of content generated by a large language model. Its types include, but are not limited to, neural network models and logistic regression models. Taking a neural network model as an example, when training the scoring model based on target training samples, the target training samples are first input into the initial neural network model. The large language model's answers in the training samples serve as the model input, and user evaluations serve as labels (e.g., marking "meets expectations" as 1 and "does not meet expectations" as 0). The parameters of the initial neural network model are adjusted using a backpropagation algorithm, such as adjusting the weights of neurons in the hidden layers, so that the error between the scoring result output by the neural network model and the label gradually decreases. When the neural network model reaches a preset number of training rounds (e.g., 50 rounds) or the neural network model error is lower than a preset error threshold (e.g., 0.01), training is stopped. The neural network model obtained at this point is the target scoring model. The above is only an illustrative example and not a limitation on the specific training process of the scoring model.

[0039] By using the above methods, the scoring model can be equipped with the ability to accurately assess content quality using target training samples, providing a reliable basis for subsequent adjustments to the large language model.

[0040] Step 104: Based on the target scoring model, score the content generated by the large language model during the interaction process to obtain the scoring result, and adjust the large language model based on the scoring result until the preset termination condition of the large language model training is met.

[0041] In some embodiments, the generated content of the large language model during the interaction process is obtained, specifically the answer generated by the large language model in response to the user's input question during real-time interaction with the user. The generated content of the large language model during the interaction process is input into a target scoring model, which scores the content to obtain a score result. The large language model is adjusted based on the score result. If the score result is lower than a preset score threshold, the generation parameters of the large language model are adjusted, such as the parameters of the word embedding layer; if the score result is higher than the preset score threshold, the current parameters of the large language model remain unchanged. Preset termination conditions include, but are not limited to, the score result reaching the preset score threshold and the number of training iterations of the large language model reaching a preset number. When any one of these conditions is met, the adjustment of the large language model is stopped.

[0042] Using the above methods, the parameters of the large language model can be continuously optimized based on the scoring results of the target scoring model, ensuring that the quality of the content generated by the large language model during the interaction process meets expectations.

[0043] In summary, the adversarial training method for the model provided in this disclosure can obtain risk levels in different dimensions by performing multi-dimensional risk assessment on the evaluation and question-answer pairs (training samples) during the interaction process of the large language model. Based on the risk levels in different dimensions, the weights of the training samples are updated to obtain target training samples. A scoring model is trained based on the target training samples to obtain a target scoring model. The generated content of the large language model in the interaction process is scored based on the target scoring model, and the large language model is adjusted according to the scoring results until the preset termination conditions are met. This improves the robustness of the scoring model and enhances the overall performance of the large language model after adjustment based on the scoring results of the scoring model in the interaction process.

[0044] Figure 2 A flowchart illustrating an adversarial training method for a model provided in this disclosure is further illustrated. Based on Figure 1 The embodiment shown further explains step 101: performing a multi-dimensional risk assessment on the training samples to obtain the risk levels of the training samples in different dimensions. These different risk levels include at least one of: model parameter gradient risk level, similarity risk level with historical samples, and similarity risk level with the knowledge base. Figure 2 This may include the following steps: Step 201: Calculate the gradient risk of the model parameters corresponding to the training sample based on the gradient of the model parameters and the loss function.

[0045] In some embodiments, the gradient risk of model parameters It is obtained by calculating the gradient norm of the loss function L(x) with respect to the scoring model parameters w on the training samples x, specifically expressed as: Where L(x) is the loss value of the scoring model on the training sample x, and ▽ represents the gradient operator. The gradient risk of the model parameters reflects the strength of the influence of the current training sample on the parameters of the scoring model. When the gradient risk of the model parameters is large, it indicates that the training sample differs significantly from the pattern already learned by the scoring model, which may lead to significant updates to the parameters of the scoring model, and is therefore considered to have a high risk of model update.

[0046] Step 202: Calculate the similarity risk degree with historical samples based on the similarity between the training samples and each sample in the most recent preset number of historical training samples.

[0047] In some embodiments, the similarity risk with historical samples It is obtained by calculating the minimum similarity distance between the training sample x and all samples in the sample set T used in the most recent t training iterations, specifically expressed as: Where dist(x,x') is the similarity distance function between samples x and x', and t is a preset positive integer representing the historical training rounds. The similarity risk with historical samples reflects the degree of difference between the current training sample and recent training samples. A large value indicates that the training sample deviates significantly from recent training patterns, posing a higher risk of short-term updates. Similarity distance can be calculated using, but is not limited to, cosine distance or Euclidean distance from word embedding vectors.

[0048] Step 203: Calculate the similarity risk with the knowledge base based on the similarity between the training sample and each reference sample in the preset knowledge base.

[0049] In some embodiments, the similarity risk level with the knowledge base This is obtained by calculating the minimum similarity distance between the training sample x and all reference samples in the preset knowledge base D, specifically expressed as: The knowledge base D contains verified, uncontaminated question-answer pairs. The similarity risk with the knowledge base reflects the degree of difference between the current training sample and sample x' in the knowledge base D. A large value indicates a significant difference between the training sample and the samples in the knowledge base, potentially containing poisoned content or anomalous patterns, posing a high risk of knowledge updates. The construction of the knowledge base can include, but is not limited to, using high-quality question-and-answer pairs that have been manually reviewed or secure data that has undergone multiple rounds of verification.

[0050] The above method can assess the risk of training samples from three dimensions, providing a multi-dimensional risk assessment basis for subsequent weight adjustments.

[0051] Figure 3 A flowchart illustrating an adversarial training method for a model provided in this disclosure is further illustrated. Based on Figure 1 The illustrated embodiment further explains step 102. Figure 3 This may include the following steps: Step 301: Perform a combined calculation on the risk levels of the different dimensions to obtain the comprehensive risk level of the training sample.

[0052] In some embodiments, the risk scores obtained in step 101 are combined to obtain the comprehensive risk score s(x) of the training samples. The combination operation includes, but is not limited to, multiplication, weighted summation, or other mathematical combination functions. In one possible implementation of this disclosure, multiplication is used to combine the gradient risk scores of the model parameters. Similarity risk with knowledge base and the risk of similarity with historical samples. Multiplication, that is This combination method can balance the risk contribution of each dimension. When any one risk level increases, the overall risk level will increase accordingly, thus comprehensively reflecting the overall risk level of the training samples.

[0053] Step 302: Calculate the risk weight of the training sample based on the comprehensive risk level; wherein the risk weight is negatively correlated with the processed total risk level.

[0054] In some embodiments, the comprehensive risk degree s(x) is normalized, scaling its value range to the [0,1] interval. The specific method of normalization is as follows: Where max and min are preset threshold parameters used to control the range of values ​​after normalization. Based on the normalized comprehensive risk level... Calculate risk weights The calculation formula is: Where ε is a very small positive number used to prevent the denominator from being zero. Using this calculation method, the normalized overall risk level... When the risk weight increases, The decrease reflects the negative correlation between risk weights and the normalized overall risk level, reducing the impact of high-risk training samples in subsequent training.

[0055] Step 303: Update the training samples according to the risk weights to obtain the target training samples.

[0056] In some embodiments, the methods for updating training samples based on risk weights to obtain target training samples include, but are not limited to, sample selection and sample weighting. If sample selection is used, a risk weight threshold must be preset, retaining training samples with risk weights greater than or equal to the threshold and discarding those with risk weights less than the threshold. If sample weighting is used, the risk weight is used as the contribution coefficient of the training sample in the subsequent training of the scoring model; samples with higher risk weights have a greater impact on the parameter update process of the scoring model.

[0057] The above method can transform risk levels of different dimensions into a unified comprehensive risk level through combined operations. Then, risk weights are determined based on negative correlations, ultimately updating the training samples. This effectively filters out low-risk, high-value training samples, providing high-quality data support for the subsequent training of the scoring model and reducing the interference of high-risk samples on the performance of the scoring model.

[0058] Figure 4 This is a schematic flowchart illustrating an adversarial training method for a model provided in an embodiment of this disclosure. Figure 4 As shown, the adversarial training method of this model includes steps 401-406.

[0059] Step 401: Perform a multi-dimensional risk assessment on the training samples to obtain the risk level of the training samples in different dimensions; wherein, the multi-dimensional risk level includes at least one of the following: model parameter gradient, similarity with historical samples, and similarity with the knowledge base, and the training samples are evaluation and question-answer pairs in the interaction process of the large language model.

[0060] For an explanation of step 401, please refer to [link / reference]. Figure 1 The detailed description of the relevant steps will not be repeated here.

[0061] Step 402: Calculate the feature overlap and category discrimination of the training samples to obtain the diversity weight of the training samples.

[0062] In some embodiments, diversity weight The calculations are performed on the mini-batch sample set D1. Training samples are mapped to a low-dimensional feature space constructed by pre-trained models, including but not limited to pre-trained text semantic classification and sentiment classification models. The output feature representation f(x) of the last hidden layer of the text semantic classification and sentiment classification models is extracted as sample features. Based on the sample features, the feature overlap and class discrimination of the training samples are calculated. The feature overlap is obtained by calculating the overlapping regions of different categories of training samples in each feature dimension of the low-dimensional feature space. The specific calculation formula is as follows: ,in and They represent the categories respectively. Eigenvalues The maximum and minimum values; the class discrimination is obtained by solving the minimum error L1 of the classifier using linear programming; diversity weights. The F2 and L1 values ​​of all samples in the mini-batch sample set D1 are obtained by summing them. , This is a sample in the mini-batch sample set D1.

[0063] Step 403: Calculate the risk weight and the diversity weight to obtain the updated weight of the training sample.

[0064] In some embodiments, risk weights With diversity weight A linear combination is performed to obtain the final updated weight W. The specific calculation formula is as follows: Where α and β are preset weighting factors used to control risk weights. With diversity weight The degree of influence on the overall weights. The weight factors α and β range from [0,1] and satisfy α+β=1. Different training strategies can be achieved by adjusting the values ​​of these two factors. For example, when focusing on risk control, a larger α value can be set; when focusing on sample diversity, a larger β value can be set.

[0065] Step 404: Update the training samples based on the updated weights to obtain the target training samples.

[0066] In some embodiments, the updated weights W calculated in step 403 are used to weight the training samples. Specific implementations include, but are not limited to: multiplying each training sample by its corresponding updated weight in the training loss function of the scoring model to form a weighted loss function; or adjusting the sampling probability of the training samples according to the updated weights. The sample set obtained after processing in the above manner is the target training sample, which considers both the risk level and the diversity of the training samples.

[0067] Step 405: Train the scoring model based on the target training samples to obtain the target scoring model.

[0068] Step 406: Based on the target scoring model, score the content generated by the large language model during the interaction process to obtain a scoring result, and adjust the large language model based on the scoring result until the preset termination condition of the large language model training is met.

[0069] For explanations of steps 405 and 406, please refer to [link / reference needed]. Figure 1The detailed description of the relevant steps will not be repeated here.

[0070] The above method ensures the safety of the scoring model training and avoids model overfitting caused by sampling bias.

[0071] Figure 5 A flowchart illustrating an adversarial training method for a model provided in this disclosure is further illustrated. Based on Figure 4 The illustrated embodiment further explains step 402. Figure 5 This may include the following steps: Step 501: Map the training samples to a low-dimensional feature space composed of pre-trained models to obtain sample features.

[0072] In some embodiments, training samples are input into pre-trained text semantic classification models and sentiment classification models. The output feature representation f(x) of the last hidden layer of the pre-trained text semantic classification models and sentiment classification models is extracted as the sample features of the training samples in a low-dimensional feature space. The text semantic classification model is used to extract the topic and semantic information of the input text, and the sentiment classification model is used to extract the emotional color features of the text. By mapping the text data in the high-dimensional training samples to the low-dimensional feature space generated by these pre-trained models, the distribution characteristics of the training samples are analyzed in the low-dimensional feature space. The selection of pre-trained models includes, but is not limited to, Transformer architecture models such as BERT and RoBERTa, or other neural network models pre-trained on large-scale text data.

[0073] Step 502: Based on the sample features, determine the degree of feature overlap of the training samples by calculating the overlapping region of feature values ​​of different categories of samples.

[0074] In some embodiments, in the low-dimensional feature space, two different categories are calculated for each feature dimension i. and The maximum value of this feature dimension and minimum value Then, the proportion of overlapping regions between the two categories on that feature dimension is calculated, and the final feature overlap degree is obtained by multiplying the proportions of overlapping regions across all feature dimensions. When there is significant overlap between different categories of samples in the feature space, the F2 value is large, indicating a complex and highly diverse sample distribution.

[0075] Step 503: Determine the class discrimination of the training samples by calculating the classification boundary clarity of the different class samples.

[0076] In some embodiments, a linear classifier is constructed in a low-dimensional feature space using linear programming to separate training samples of different classes. By solving the linear programming problem, the solution that minimizes the sum of distances from misclassified samples to the separating hyperplane is obtained, which is the class discrimination L1. A larger L1 value indicates that even the optimal linear classifier has difficulty completely separating samples of different classes, suggesting that the boundaries between classes are more blurred, the distribution of training samples is more complex, and the diversity is higher.

[0077] Step 504: Determine the diversity weight of the training samples based on the degree of feature overlap and the class discrimination.

[0078] In some embodiments, the feature overlap F2 and the class discriminancy L1 are added together to obtain the diversity contribution of a single sample in the mini-batch sample set D1, i.e. The overall diversity weight of mini-batch sample set D1 is obtained by summing the diversity contributions of all samples in the mini-batch sample set D1. The diversity weight reflects the level of diversity of the training samples in terms of both topic semantics and sentiment. The larger the weight value, the higher the sample diversity.

[0079] The above methods improve the diversity of samples during the training of the scoring model and avoid overfitting caused by sampling bias.

[0080] Figure 6 A flowchart illustrating an adversarial training method for a model provided in this disclosure is further illustrated, based on... Figure 1 The illustrated embodiment further explains step 104. Figure 6 This may include the following steps: Step 601: Obtain the generated content of the large language model during the interaction process.

[0081] In some embodiments, the content generated by the large language model in response to user questions is extracted by recording the user's historical dialogue with the large language model question-answering system. This generated content includes, but is not limited to, text answers, code snippets, solutions, and other forms of content. The acquisition of generated content can be achieved through a logging system, API call records, or a dedicated dialogue monitoring module, ensuring that all outputs of the large language model during the interaction process can be fully obtained.

[0082] Step 602: Input the generated content into the target scoring model for scoring, and obtain the scoring result.

[0083] In some embodiments, the generated content obtained in step 601 is used as input and sent to the target scoring model for processing. The target scoring model analyzes and evaluates the input content and outputs corresponding scoring results. The scoring results are usually numerical scores, and the score range can be a standardized range such as [0,1] or 0-100 points. The higher the score, the better the quality of the generated content and the more it meets the user's expectations and preferences. The scoring process may include a comprehensive evaluation of multiple dimensions such as relevance, accuracy, security, and fluency of the content.

[0084] Step 603: Adjust the parameters of the large language model based on the scoring results.

[0085] In some embodiments, based on the scoring results obtained in step 602, reinforcement learning algorithms are used to adjust the parameters of the large language model, including but not limited to using proximal policy optimization algorithms, using the scoring results as reward signals, and updating the parameters of the large language model through policy gradient methods. The parameter adjustment process of the large language model aims to enable the large language model to generate higher-quality content in the future, i.e., to generate content that can obtain higher scores. The scope of parameter adjustment may include some or all of the parameters of the large language model, depending on training requirements and computational resources.

[0086] Step 604: Repeat the steps of performing multi-dimensional risk assessment on training samples to adjusting the large language model based on the scoring results, until the scoring results reach a preset score threshold and / or the number of training iterations of the large language model reaches a preset number.

[0087] In some embodiments, a loop training mechanism is established to repeatedly execute the complete process of steps 101 to 104. The loop termination condition is set to at least one of the following: when the average score result of the content generated by the large language model reaches a preset score threshold, it indicates that the performance of the large language model has met the quality requirements; or when the number of training iterations of the large language model reaches a preset number, it ensures that the training process will not continue indefinitely. The specific values ​​of the preset score threshold and the preset number of iterations can be set according to the actual application scenario, and this disclosure embodiment does not limit them.

[0088] The above methods can establish a complete closed loop for optimizing large language models. Based on the feedback from the scoring model, the parameters of the large language model are continuously adjusted to achieve continuous improvement in the performance of the large language model. At the same time, clear termination conditions ensure effective control of the training process and rational use of resources.

[0089] Figure 7 A flowchart illustrating an adversarial training method for a model provided in this disclosure is further illustrated, based on... Figure 5 The illustrated embodiment further explains step 501. Figure 7 This may include the following steps: Step 701: Input the training samples into the pre-trained text semantic classification model and sentiment classification model.

[0090] In some embodiments, the collected training samples, namely the question-and-answer pairs between users and the large language model and their corresponding evaluations, are input into a pre-trained text semantic classification model and a sentiment classification model. The text semantic classification model is used to understand the topic content and semantic information of the question-and-answer pairs, and the sentiment classification model is used to analyze the emotional tendencies and moods contained in the text.

[0091] Step 702: Extract the output feature representations of the hidden layers of the text semantic classification model and the sentiment classification model.

[0092] In some embodiments, after the text semantic classification model and the sentiment classification model have processed the input training samples, the output vector of the last hidden layer of the text semantic classification model is extracted as the semantic feature representation, and the output vector of the last hidden layer of the sentiment classification model is extracted as the sentiment feature representation.

[0093] Step 703: Determine the output feature representation as the sample feature of the training sample in the low-dimensional feature space.

[0094] In some embodiments, the semantic feature representation extracted from the text semantic classification model and the sentiment feature representation extracted from the sentiment classification model in step 702 are concatenated or fused to form the final sample feature f(x) of the training sample in the low-dimensional feature space. The feature fusion method includes, but is not limited to, vector concatenation, weighted summation, or attention mechanism fusion.

[0095] The above method can effectively map high-dimensional original text training samples to a meaningful low-dimensional feature space, providing a reliable feature basis for measuring sample diversity.

[0096] Figure 8 This is a schematic diagram of the overall process of an adversarial training method for a model provided in an embodiment of the present disclosure, which includes the above-described method.

[0097] Corresponding to the adversarial training method for the model described above, this invention also proposes an adversarial training device for the model. Since the device embodiments of this invention correspond to the method embodiments described above, details not disclosed in the device embodiments can be referred to in the method embodiments described above, and will not be repeated here.

[0098] Figure 9 This is a schematic diagram of the structure of an adversarial training device for a model provided in an embodiment of the present disclosure, as shown below. Figure 9 As shown, it includes: Evaluation unit 81 is used to perform multi-dimensional risk assessment on training samples to obtain risk levels of the training samples in different dimensions; wherein, the multi-dimensional risk level includes at least one of model parameter gradient, similarity with historical samples, and similarity with the knowledge base, and the training samples are evaluation and question-answer pairs in the interaction process of the large language model. The first calculation unit 82 is used to update the weights of the training samples according to the risk levels of the different dimensions to obtain the target training sample. Training unit 83 is used to train the scoring model based on the target training samples to obtain the target scoring model; The adjustment unit 84 is used to score the content generated by the large language model during the interaction process based on the target scoring model, obtain the scoring result, and adjust the large language model based on the scoring result until the preset termination condition of the large language model training is met.

[0099] The adversarial training device for the model described in this embodiment can obtain risk levels in different dimensions by performing multi-dimensional risk assessments on evaluation and question-answer pairs (training samples) during the interaction process of a large language model. Based on the risk levels in different dimensions, the weights of the training samples are updated to obtain target training samples. A scoring model is trained based on the target training samples to obtain a target scoring model. The generated content of the large language model during the interaction process is scored based on the target scoring model, and the large language model is adjusted according to the scoring results until a preset termination condition is met. This improves the robustness of the scoring model and enhances the overall performance of the large language model after adjustment based on the scoring results of the scoring model during the interaction process.

[0100] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 10 As shown, the different dimensions of risk include at least one of the following: model parameter gradient risk, similarity risk with historical samples, and similarity risk with the knowledge base. The evaluation unit 81 includes: The first calculation module 811 is used to calculate the risk degree of the model parameter gradient corresponding to the training sample based on the model parameter gradient and the loss function. The second calculation module 812 is used to calculate the similarity risk degree with the historical samples based on the similarity between the training samples and each sample in the most recent preset number of historical training samples. The third calculation module 813 is used to calculate the similarity risk degree with the knowledge base based on the similarity between the training sample and each reference sample in the preset knowledge base.

[0101] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 10 As shown, the first computing unit 82 includes: The fourth calculation module 821 is used to perform combined calculations on the risk levels of the different dimensions to obtain the comprehensive risk level of the training sample; The fifth calculation module 822 is used to calculate the risk weight of the training sample based on the comprehensive risk level; wherein the risk weight is negatively correlated with the processed total risk level; The update module 823 is used to update the training samples according to the risk weights to obtain the target training samples.

[0102] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 10 As shown, the device further includes: The second calculation unit 85 is used to calculate the feature overlap and category discrimination of the training samples after the evaluation unit 81 performs multi-dimensional risk assessment on the training samples and obtains the risk level of the training samples in different dimensions, so as to obtain the diversity weight of the training samples.

[0103] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 10 As shown, the second computing unit 85 includes: The mapping module 851 is used to map the training samples to a low-dimensional feature space composed of pre-trained models to obtain sample features. The first determining module 852 is used to determine the degree of feature overlap of the training samples by calculating the overlapping region of feature values ​​of different categories of samples based on the sample features. The second determining module 853 is used to determine the class discrimination of the training samples by calculating the classification boundary clarity of the different class samples; The third determining module 854 is used to determine the diversity weight of the training samples based on the feature overlap degree and the category discrimination degree.

[0104] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 10 As shown, the update module 823 includes: The calculation submodule 8231 is used to calculate the risk weight and the diversity weight to obtain the updated weight of the training sample; The update submodule 8232 is used to update the training sample based on the update weight to obtain the target training sample.

[0105] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 10 As shown, the adjustment unit 84 includes: The acquisition module 841 is used to acquire the generated content of the large language model during the interaction process; The scoring module 842 is used to input the generated content into the target scoring model for scoring, and obtain the scoring result; Adjustment module 843 is used to adjust the parameters of the large language model based on the scoring results; The execution module 844 is used to repeatedly execute the steps of performing multi-dimensional risk assessment on the training samples to adjusting the large language model based on the scoring results, until the scoring results reach a preset score threshold, and / or the number of training iterations of the large language model reaches a preset number.

[0106] Furthermore, in one possible implementation of the embodiments of this disclosure, such as Figure 10 As shown, the mapping module 851 includes: The input submodule 8511 is used to input the training samples into the pre-trained text semantic classification model and sentiment classification model; Extraction submodule 8512 is used to extract the output feature representations of the hidden layers of the text semantic classification model and the sentiment classification model; The determination submodule 8513 is used to determine the output feature representation as the sample feature of the training sample in the low-dimensional feature space.

[0107] It should be noted that the foregoing explanation of the method embodiments also applies to the apparatus of the embodiments of this disclosure, and the principle is the same. Therefore, the embodiments of this disclosure are not limited thereto.

[0108] According to embodiments of this disclosure, this disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0109] Figure 11 A schematic block diagram of an example electronic device 900 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0110] like Figure 11As shown, the electronic device 900 includes a computing unit 901, which can perform various appropriate actions and processes based on a computer program stored in ROM (Read-Only Memory) 902 or loaded from storage unit 908 into RAM (Random Access Memory) 903. The RAM 903 can also store various programs and data required for the operation of the electronic device 900. The computing unit 901, ROM 902, and RAM 903 are interconnected via bus 904. An I / O (Input / Output) interface 905 is also connected to bus 904.

[0111] Multiple components in electronic device 900 are connected to I / O interface 905, including: input unit 906, such as keyboard, mouse, etc.; output unit 907, such as various types of displays, speakers, etc.; storage unit 908, such as disk, optical disk, etc.; and communication unit 909, such as network card, modem, wireless transceiver, etc. Communication unit 909 allows electronic device 900 to exchange information / data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0112] The computing unit 901 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the computing unit 901 include, but are not limited to, CPUs (Central Processing Units), GPUs (Graphics Processing Units), various special-purpose AI (Artificial Intelligence) computing chips, various computing units running machine learning model algorithms, DSPs (Digital Signal Processors), and any suitable processor, controller, microcontroller, etc. The computing unit 901 performs the various methods and processes described above, such as adversarial training methods for models. For example, in some embodiments, the adversarial training method for models can be implemented as a computer software program tangibly contained in a machine-readable medium, such as storage unit 908. In some embodiments, part or all of the computer program can be loaded and / or installed on the electronic device 900 via ROM 902 and / or communication unit 909. When the computer program is loaded into RAM 903 and executed by the computing unit 901, one or more steps of the methods described above can be performed. Alternatively, in other embodiments, the computing unit 901 may be configured to perform the adversarial training method of the aforementioned model by any other suitable means (e.g., by means of firmware).

[0113] Various implementations of the systems and techniques described above herein can be implemented in digital electronic circuit systems, integrated circuit systems, FPGAs (Field Programmable Gate Arrays), ASICs (Application-Specific Integrated Circuits), ASSPs (Application-Specific Standard Products), SOCs (System-on-Chips), CPLDs (Complex Programmable Logic Devices), computer hardware, firmware, software, and / or combinations thereof. These various implementations may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0114] The program code used to implement the methods of this disclosure may be written in any combination of one or more programming languages. This program code may be provided to a processor or controller of a general-purpose computer, special-purpose computer, or other programmable data processing apparatus, such that when executed by the processor or controller, the program code causes the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The program code may be executed entirely on a machine, partially on a machine, as a standalone software package partially on a machine and partially on a remote machine, or entirely on a remote machine or server.

[0115] In the context of this disclosure, a machine-readable medium can be a tangible medium that may contain or store a program for use by or in conjunction with an instruction execution system, apparatus, or device. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. More specific examples of machine-readable storage media include electrical connections based on one or more wires, portable computer disks, hard disks, RAM, ROM, EPROM (Electrically Programmable Read-Only Memory) or flash memory, optical fiber, CD-ROM (Compact Disc Read-Only Memory), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0116] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device for displaying information to the user (e.g., a CRT (Cathode-Ray Tube) or LCD (Liquid Crystal Display) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the computer. Other types of devices can also be used to provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0117] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or middleware components (e.g., application servers), or frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication of any form or medium (e.g., communication networks). Examples of communication networks include LANs (Local Area Networks), WANs (Wide Area Networks), the Internet, and blockchain networks.

[0118] Computer systems can include clients and servers. Clients and servers are generally geographically separated and typically interact via communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. A server can be a cloud server, also known as a cloud computing server or cloud host, a hosting product within the cloud computing service system that addresses the shortcomings of traditional physical hosts and VPS (Virtual Private Server) services, such as high management difficulty and weak business scalability. Servers can also be servers for distributed systems or servers incorporating blockchain technology.

[0119] It's important to note that artificial intelligence (AI) is the study of enabling computers to simulate certain human thought processes and intelligent behaviors (such as learning, reasoning, thinking, and planning). It encompasses both hardware and software technologies. AI hardware technologies generally include sensors, dedicated AI chips, cloud computing, distributed storage, and big data processing. AI software technologies primarily include computer vision, speech recognition, natural language processing, machine learning / deep learning, big data processing, and knowledge graph technologies.

[0120] It should be understood that the various forms of processes shown above can be used to rearrange, add, or delete steps. For example, the steps described in this disclosure can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution disclosed in this disclosure can be achieved, and this is not limited herein.

[0121] The specific embodiments described above do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure should be included within the scope of protection of this disclosure.

Claims

1. An adversarial training method for a model, characterized in that, include: A multi-dimensional risk assessment is performed on the training samples to obtain the risk level of the training samples in different dimensions; wherein, the multi-dimensional risk level includes at least one of model parameter gradient, similarity with historical samples, and similarity with the knowledge base, and the training samples are evaluation and question-answer pairs in the interaction process of the large language model; The weights of the training samples are updated and calculated based on the risk levels of the different dimensions to obtain the target training samples; The scoring model is trained based on the target training samples to obtain the target scoring model; The large language model generates content during the interaction process based on the target scoring model to obtain a scoring result. The large language model is then adjusted based on the scoring result until the preset termination condition of the large language model training is met.

2. The method according to claim 1, characterized in that, The different dimensions of risk include at least one of the following: model parameter gradient risk, similarity risk with historical samples, and similarity risk with the knowledge base. The multi-dimensional risk assessment of the training samples, to obtain the risk levels of the training samples in different dimensions, includes: Calculate the gradient risk of the model parameters corresponding to the training sample based on the gradient of the model parameters and the loss function; The similarity risk with historical samples is calculated based on the similarity between the training samples and each sample in the most recent preset number of historical training samples. The similarity risk level with the knowledge base is calculated based on the similarity between the training sample and each reference sample in the preset knowledge base.

3. The method according to claim 1, characterized in that, The step of updating the weights of the training samples based on the risk levels of the different dimensions to obtain the target training samples includes: The risk levels of the different dimensions are combined to obtain the comprehensive risk level of the training sample; The risk weight of the training sample is calculated based on the comprehensive risk level; wherein the risk weight is negatively correlated with the processed total risk level. The training samples are updated according to the risk weights to obtain the target training samples.

4. The method according to claim 3, characterized in that, After performing multi-dimensional risk assessment on the training samples to obtain the risk levels of the training samples in different dimensions, the method further includes: The feature overlap and class discrimination of the training samples are calculated to obtain the diversity weight of the training samples.

5. The method according to claim 4, characterized in that, The calculation of the feature overlap and class discrimination of the training samples to obtain the diversity weight of the training samples includes: The training samples are mapped to a low-dimensional feature space composed of pre-trained models to obtain sample features; Based on the sample features, the degree of feature overlap of the training samples is determined by calculating the overlapping region of feature values ​​of different categories of samples; The class discrimination of the training samples is determined by calculating the clarity of the classification boundaries of the samples of different categories; The diversity weights of the training samples are determined based on the degree of feature overlap and the class discrimination.

6. The method according to claim 4, characterized in that, The step of updating the training samples according to the risk weights to obtain the target training samples includes: The updated weights of the training samples are obtained by calculating the risk weights and the diversity weights. The training samples are updated based on the updated weights to obtain the target training samples.

7. The method according to claim 1, characterized in that, The process of scoring the content generated by the large language model during the interaction process based on the target scoring model, obtaining a scoring result, and adjusting the large language model based on the scoring result until the preset termination condition of the large language model training is met includes: Obtain the generated content of the large language model during the interaction process; The generated content is input into the target scoring model for scoring, and the scoring result is obtained; The parameters of the large language model are adjusted based on the scoring results; Repeat the steps of performing multi-dimensional risk assessment on training samples to adjusting the large language model based on the scoring results, until the scoring results reach a preset score threshold and / or the number of training iterations of the large language model reaches a preset number.

8. The method according to claim 5, characterized in that, The step of mapping the training samples to a low-dimensional feature space composed of pre-trained models to obtain sample features includes: The training samples are input into a pre-trained text semantic classification model and a sentiment classification model; Extract the output feature representations of the hidden layers of the text semantic classification model and the sentiment classification model; The output feature representation is determined as the sample feature of the training sample in the low-dimensional feature space.

9. An adversarial training device for a model, characterized in that, include: An evaluation unit is used to perform multi-dimensional risk assessment on training samples to obtain risk levels of the training samples in different dimensions; wherein, the multi-dimensional risk level includes at least one of model parameter gradient, similarity with historical samples, and similarity with the knowledge base, and the training samples are evaluation and question-answer pairs in the interaction process of the large language model. The first calculation unit is used to update the weights of the training samples according to the risk levels of the different dimensions to obtain the target training samples. The training unit is used to train the scoring model based on the target training samples to obtain the target scoring model; The adjustment unit is used to score the content generated by the large language model during the interaction process based on the target scoring model, obtain the scoring result, and adjust the large language model based on the scoring result until the preset termination condition of the large language model training is met.

10. An electronic device, characterized in that, include: At least one processor; as well as A memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor to enable the at least one processor to perform the method of any one of claims 1-8.

11. A non-transitory computer-readable storage medium storing computer instructions, characterized in that, The computer instructions are used to cause the computer to perform the method according to any one of claims 1-8.

12. A computer program product, characterized in that, Includes a computer program that, when executed by a processor, implements the method according to any one of claims 1-8.

Images