Sample processing method, device, electronic device and storage medium

Abstract

The present disclosure provides a sample processing method, apparatus, electronic device, and storage medium, relating to the field of data processing technology, particularly to the fields of artificial intelligence, large language models, and deep learning. A specific implementation scheme comprises: performing multiple batches of training and testing on a large language model to obtain a first data set for the large language model; training a simulator of the large language model based on a first sample pair and a first model evaluation index to obtain a target simulator; processing a second training sample and a second test sample based on the target simulator to obtain a prediction model evaluation index for a second sample pair, the second sample pair including the second training sample and the second test sample; and obtaining a target training sample from the plurality of second training samples based on the prediction model evaluation index.

Description

Technical Field

[0001] The present disclosure relates to the field of data processing technology, specifically to the fields of artificial intelligence, large language models and deep learning, and in particular to a sample processing method, device, electronic device and storage medium. Background Art

[0002] Currently, methods for estimating the impact of training data on test data can be based on gradients, by calculating the dot product of the loss gradients of the test data and the training data, and estimating the impact of training data on test data weighted by the learning rate. Alternatively, methods based on sampling are used, where a large amount of data and the corresponding loss function are first collected during the training process, an index is established for each data point, and then a linear model is learned to estimate the impact. However, current impact estimation methods cannot be generalized to new data points and have certain limitations in evaluating training data. Summary of the Invention

[0003] The present disclosure provides a sample processing method, device, electronic device, and storage medium.

[0004] According to one aspect of the present disclosure, a sample processing method is provided, the method comprising:

[0005] Performing multiple batches of training and testing on the large language model to obtain a first data set of the large language model, the first data set including reference model evaluation indicators of a plurality of first sample pairs, each of the first sample pairs including a first training sample and a first test sample;

[0006] Training a simulator of the large language model based on the first sample pairs and the reference model evaluation indicator to obtain a target simulator;

[0007] Processing a second training sample and a second test sample based on the target simulator to obtain a prediction model evaluation index of a second sample pair, where the second sample pair includes a second training sample and a second test sample;

[0008] According to the prediction model evaluation indicator, a target training sample is obtained from the plurality of second training samples.

[0009] According to another aspect of the present disclosure, there is provided a sample processing device, comprising:

[0010] a first acquisition module, configured to perform multi-batch training and testing on a large language model to obtain a first data set of the large language model, wherein the first data set includes reference model evaluation indicators of a plurality of first sample pairs, each of the first sample pairs including a first training sample and a first test sample;

[0011] A second acquisition module is configured to train a simulator of the large language model based on the first sample pairs and the reference model evaluation index to obtain a target simulator;

[0012] a third acquisition module, configured to process a second training sample and a second test sample based on the target simulator to obtain a prediction model evaluation index of a second sample pair, where the second sample pair includes a second training sample and a second test sample;

[0013] A sample processing module is used to obtain a target training sample from a plurality of second training samples according to the prediction model evaluation index.

[0014] According to a third aspect of the present disclosure, there is provided an electronic device, including:

[0015] at least one processor; and

[0016] a memory communicatively coupled to the at least one processor; wherein

[0017] The memory stores instructions that can be executed by the at least one processor. The instructions are executed by the at least one processor to enable the at least one processor to perform the method according to the first aspect.

[0018] According to a fourth aspect of the present disclosure, a non-transitory computer-readable storage medium storing computer instructions is provided, wherein the computer instructions are used to cause the computer to execute the method described in the first aspect.

[0019] According to a fifth aspect of the present disclosure, a computer program product is provided, comprising a computer program, which implements the steps of the method according to the first aspect when executed by a processor.

[0020] It should be understood that the contents described in this section are not intended to identify the key or important features of the embodiments of the present disclosure, nor are they intended to limit the scope of the present disclosure. Other features of the present disclosure will become readily understood through the following description. BRIEF DESCRIPTION OF THE DRAWINGS

[0021] The accompanying drawings are provided to facilitate a better understanding of the present invention and do not constitute a limitation of the present disclosure.

[0022] Figure 1 is a schematic diagram of a sample processing method provided by an embodiment of the present disclosure;

[0023] Figure 2 This is a logical schematic diagram of obtaining a first data set provided by an embodiment of the present disclosure;

[0024] Figure 3 is a schematic diagram of another sample processing method provided by an embodiment of the present disclosure;

[0025] Figure 4 This is a logic diagram for obtaining prediction model evaluation indicators provided by an embodiment of the present disclosure;

[0026] Figure 5 is a schematic diagram of another sample processing method provided by an embodiment of the present disclosure;

[0027] Figure 6 is a schematic diagram of another sample processing method provided by an embodiment of the present disclosure;

[0028] Figure 7 is a structural schematic diagram of a sample processing device provided by an embodiment of the present disclosure;

[0029] Figure 8 is a schematic block diagram of an electronic device that can be used to implement an embodiment of the present disclosure. DETAILED DESCRIPTION

[0030] The following description of exemplary embodiments of the present disclosure is made in conjunction with the accompanying drawings, including various details of the embodiments of the present disclosure to facilitate understanding. These details should be considered as merely exemplary. Therefore, those skilled in the art will recognize that various changes and modifications may be made to the embodiments described herein without departing from the scope and spirit of the present disclosure. Similarly, for the sake of clarity and conciseness, descriptions of well-known functions and structures are omitted in the following description.

[0031] Data processing is a technical process of analyzing and processing data (both numerical and non-numerical), including the collection, storage, retrieval, processing, transformation and transmission of raw data. Its basic purpose is to extract and derive valuable and meaningful data for certain specific people from large amounts of disorganized and difficult-to-understand data.

[0032] Artificial Intelligence (AI) is a new technical science that studies and develops theories, methods, technologies and application systems for simulating, extending and expanding human intelligence. It can simulate the information process of human consciousness and thinking. Its main goal is to enable humans to perform complex tasks that usually require human intelligence.

[0033] A Large Language Model (LLM) is an artificial intelligence model designed to understand and generate language. It can perform a wide range of tasks, including text summarization, translation, and sentiment analysis. It is characterized by its large scale, containing billions or even trillions of parameters, to learn complex patterns in language.

[0034] Deep learning is a new research direction in the field of machine learning. It is used to learn the inherent laws and representation levels of sample data. The ultimate goal is to enable machines to have analytical learning capabilities like humans and be able to recognize data such as text, images, and sounds.

[0035] Figure 1 FIG is a schematic diagram of a sample processing method provided by an embodiment of the present disclosure. Figure 1 As shown, the method includes the following steps:

[0036] S101: Perform multiple batches of training and testing on the large language model to obtain a first data set of the large language model.

[0037] Optionally, training samples and test samples can be obtained to train and test the large language model in multiple batches. Different batches can affect the training speed and memory usage of the model. Training and testing the large language model based on appropriate batches can improve training speed and accuracy. When training and testing the large language model in multiple batches, the training samples and test samples in each batch can correspond to each other. Through multiple batches of training and testing, a first data set of the large language model is obtained. In some implementations, the training samples and test samples can be obtained from a database, or from a training dataset and a test dataset.

[0038] In some implementations, the first data set includes reference model evaluation indicators for multiple first sample pairs, each first sample pair including a first training sample and a first test sample.

[0039] In some implementations, the large language model can be tested and trained based on the first sample pair, that is, the large language model is trained in multiple batches using the first training sample in the first sample pair, and the large language model is tested in multiple batches using the first test sample in the first sample pair. During the multi-batch training and testing of the large language model, a first data set of the large language model, that is, the reference model evaluation index of the first sample pair, is obtained.

[0040] Optionally, the reference model evaluation indicators can be the test set loss function, the recall-oriented understudy for gisting evaluation (ROUGE), the bilingual evaluation understudy (BLEU), and the accuracy (Acc); among them, the test set loss is the difference between the output result of inputting the first test sample into the large language model and the actual result; ROUGE is mainly an indicator for evaluating the quality of text summarization (or other natural language processing tasks) based on the recall rate; BLEU focuses on measuring the accuracy and precise matching of the translation, and Acc is used to represent the accuracy of the large language model, that is, the proportion of correct predictions made by the large language model.

[0041] It can be understood that after the large language model is trained based on the first training sample, the first test sample is input into the large language model to obtain the corresponding output result. At this time, based on the comparison between the output result and the actual result carried in the first test sample, the reference model evaluation index of the large language model can be determined, that is, the loss, ROUGE, BLEU and Acc indicators of the large language model under each first test sample can be obtained, such as Figure 2 As shown, training samples D are selected to perform multi-batch training on the large language model LLM, and test samples z′ are input into the large language model after each batch of training to obtain reference model evaluation indicators, and the reference model evaluation indicators of all first sample pairs are collected to obtain the first data set.

[0042] S102: Based on the first sample pair and the reference model evaluation index, a simulator of the large language model is trained to obtain a target simulator.

[0043] In some implementations, a simulator of a large language model can be used to achieve the same training effect as the large language model, that is, the output capability of the large language model is realized through the simulator of the large language model to reduce the cost of data analysis.

[0044] It can be understood that the reference model evaluation index is obtained based on the first sample pair and the large language model. Therefore, the simulator of the large language model can be trained based on the first sample pair and the reference model evaluation index, so that the simulator can learn the ability of the large language model to process the first sample pair and obtain results close to the reference model evaluation index, thereby realizing the training of the simulator and obtaining the target simulator.

[0045] Optionally, the first sample pairs can be input into the simulator of the large language model to obtain the prediction model evaluation index of each first sample pair output by the simulator. The closer the prediction model evaluation index is to the reference model evaluation index, the closer the simulator's ability to process the first sample pairs is to the large language model. Therefore, the simulator can be adjusted and trained based on the prediction model evaluation index output by the simulator to obtain a trained simulator that is closest to the ability of the large language model as the target simulator.

[0046] S103: Process the second training sample and the second test sample based on the target simulator to obtain a prediction model evaluation index of the second sample pair.

[0047] It can be understood that the second sample pair includes a second training sample and a second test sample, and the second sample pair can be obtained from the training sample set and the test sample set.

[0048] Inputting the second sample pair into the target simulator can obtain the prediction model evaluation indicators corresponding to the second sample pair. It can be understood that the target simulator can achieve the same processing capabilities as the large language model, so the prediction model evaluation indicators of the second sample pair can be loss, ROUGE, BLEU and Acc.

[0049] S104: Obtain a target training sample from the plurality of second training samples according to the prediction model evaluation index.

[0050] The prediction model evaluation indicators may include indicators such as loss, ROUGE, BLEU and Acc. The smaller the loss indicator, the better, and the larger the ROUGE, BLEU and Acc indicators, the better. Therefore, the quality of the corresponding second training sample can be determined based on the prediction model evaluation indicators. The quality of the training sample will have a greater impact on the performance of the model to be trained. Therefore, the better the quality of the training sample, the better the foundation for the training of the subsequent model. Therefore, based on the prediction model evaluation indicators of the second training sample, a target training sample with better quality can be obtained from multiple second training samples.

[0051] Optionally, a threshold range can be preset, and the second training samples corresponding to the prediction model evaluation indicators that meet the preset threshold range can be screened to obtain the target training samples; or all prediction model evaluation indicators can be arranged in descending order, and the second training samples with better prediction model evaluation indicators can be used as the target training samples.

[0052] In this embodiment, by performing multiple batches of training and testing on the large language model, a reference model evaluation index of each first sample pair output by the large language model is obtained, and the first sample pairs and the reference model evaluation index are used to train a simulator of the large language model to obtain a target simulator. The target simulator has the same processing capability as the large language model and reduces resource consumption. The second training sample and the second test sample are processed based on the target simulator to obtain the corresponding prediction model evaluation index. According to the prediction model evaluation index, target training samples with better quality can be screened out, the impact of training data on test data can be obtained more accurately, and high-quality target training samples can be screened out, with better processing effect.

[0053] Figure 3 This is a schematic diagram of another sample processing method provided by an embodiment of the present disclosure. Figure 3 As shown, the method includes the following steps:

[0054] S301 , performing multiple batches of training and testing on the large language model to obtain a first data set of the large language model.

[0055] In some implementations, a training sample set and a test sample set can be obtained, and the large language model can be trained in multiple batches based on the first training sample in the training sample set; the large language model after each batch of training is tested based on the first test sample in the test sample set to obtain a first data set of the large language model.

[0056] That is to say, the large language model is trained using the first training samples, and the first training samples of each batch correspond to the first test samples. After the large language model is trained using the first training samples of a batch, the corresponding first test samples are input into the large language model trained in the batch for testing, and the reference model evaluation index of the first sample pair of the current batch is obtained, and the first sample pair includes the first training sample and the first test sample; after performing multiple batches of training and testing on the large language model, the reference model evaluation index of all the first sample pairs is obtained, that is, the first data set of the large language model, and reliable reference model evaluation index is obtained through the corresponding training samples and test samples, and the influence of each training sample is accurately obtained.

[0057] In the embodiment of the present disclosure, the implementation method of step S301 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0058] S302: Input multiple first sample pairs into a simulator for batch processing to obtain prediction model evaluation indicators of the first sample pairs.

[0059] In some implementations, the simulator can be obtained based on a deep learning model, for example, by obtaining an encoded representation of the input sample through feature encoding with a training model, for example, feature encoding can be performed through a pre-trained language representation model (Bidirectional Encoder Representation from Transformers, BERT), that is, feature encoding is performed on the training samples and test samples of the input simulator, that is, the first training sample and the first test sample in the current first sample pair are encoded, the first training sample is encoded to obtain a first encoded representation, and the first test sample is encoded to obtain a second encoded representation.

[0060] Furthermore, the simulator can learn any neural network to predict the prediction model evaluation index of each first sample pair. Optionally, the learned neural network can be a commonly used network such as a convolutional neural network (CNN), a long short-term memory network (LSTM), and a recurrent neural network (RNN).

[0061] In some implementations, the model parameters of the simulator can also be obtained based on the encoded representations of the first training sample and the first test sample in the first sample pair, that is, the first model parameters and the second model parameters of the simulator are obtained based on the first encoded representation and the second encoded representation. The first model parameters and the second model parameters are both parameters in the simulator training process.

[0062] Optionally, a first vector representation of the Nth order of the first training sample and a second vector representation of the Nth order of the first test sample can be obtained based on the first coded representation and the second coded representation, where the Nth order is a natural number greater than or equal to 1; in some implementations, vector representations of different orders can be obtained through N multilayer perceptrons (MLPs), or other network models can be used for processing to obtain vector representations of different orders; the multilayer perceptron MLP can be understood as a fully connected feedforward neural network model, which at least includes an input layer, a hidden layer, and an output layer, and the first coded representation and the second coded representation will output the corresponding first vector representation and second vector representation after inputting each order MLP.

[0063] Furthermore, the vector dot product is performed on the first vector representation and the second vector representation of each order to obtain the first model parameter of each order of the simulator. That is, the vector dot product is performed on the first vector representation and the second vector representation of each order MLP output to obtain the first model parameter of each order of the simulator.

[0064] In some implementations, the first model parameters of the i-th order for each first sample pair from the same batch may be averaged to obtain a first average value, and the first average value may be determined as the first model parameter of the i-th order of the simulator. That is, for the i-th order, the first model parameters of each first sample pair from the same batch at that order may be obtained and averaged, and the first average value may be used as the first model parameter of the i-th order of the simulator. This first model parameter is more accurate and reliable.

[0065] Optionally, a first-order third vector representation of the first training sample and a first-order fourth vector representation of the first test sample can be obtained based on the first and second encoded representations; and a vector dot product can be performed on the third and fourth vector representations to obtain the second model parameters. In other words, a first-order MLP is performed based on the first and second encoded representations to obtain the corresponding third and fourth vector representations, and the result of the dot product of the third and fourth vector representations is used as the second model parameters.

[0066] In some implementations, the second model parameters of each first sample pair in the same batch can be averaged to obtain a second average value, and the second average value can be determined as the second model parameter of the simulator. That is, for each first sample pair in the same batch, the corresponding second model parameter is obtained based on its first-order third vector representation and fourth vector representation, and all the second model parameters in the same batch are averaged to obtain a second average value. The second average value is the second model parameter of the simulator, and the obtained second model parameter is more accurate and reliable.

[0067] Furthermore, after determining the first model parameters and the second model parameters, a prediction model evaluation index for a first sample pair predicted by the simulator is determined, where the predicted first sample pair and the current first sample pair are continuous in training order; for example, if the current first sample pair is labeled n, then the predicted first sample pairs are a plurality of consecutive first sample pairs from the n-1th pair onward. Further, based on the first model parameters and the second model parameters, and the prediction model evaluation index for the predicted first sample pair, a prediction model evaluation index for the current first sample pair is obtained.

[0068] Optionally, the prediction model evaluation index of the current first sample pair can be obtained by the following formula:

[0069]

[0070] in, Represents the prediction model evaluation index of the current first sample pair; Represents the prediction model evaluation index of the first sample pair that has been predicted; α j (c t ) represents the first model parameter of each order of the simulator; β(c t ) represents the second model parameter of the simulator.

[0071] In some implementations, if the prediction model evaluation indicator obeys the N-th order Markov assumption, then the prediction model evaluation indicator for predicting the current first sample pair only depends on the prediction model evaluation indicator of the previous N steps.

[0072] For example, Figure 4 As shown, the first training sample c t and the first training sample z ′ The simulator is input and encoded by the encoder to obtain the corresponding encoded representations r1 and r2. The encoded representations are passed through a multi-order MLP to obtain the first vector representation h1 and the second vector representation h2 of the simulator at each order, as well as the third vector representation h3 and the fourth vector representation h4, and then the first model parameter α of the simulator at each order and the second model parameter β of the simulator are obtained. Then, based on the first model parameter, the second model parameter and the prediction model evaluation index of the predicted first sample pair, the prediction model evaluation index of the current first sample pair is obtained.

[0073] S303: Adjust the simulator according to the prediction model evaluation index and the reference model evaluation index of the first sample pair, and continue training the adjusted simulator until the training end condition is met to obtain a target simulator.

[0074] It is understandable that the reference model evaluation index is the evaluation index output by the large language model. The ultimate goal of training the simulator is to enable the simulator to achieve the corresponding functions of the large language model. Therefore, the simulator can be adjusted based on the prediction model evaluation index of the first sample pair output by the simulator and the reference model evaluation index. In other words, using the reference model evaluation index as a benchmark, when the prediction model evaluation index of the first sample pair is infinitely close to the reference model evaluation index of the first sample pair, it means that the simulator can basically achieve the functions of the large language model.

[0075] The simulator is adjusted according to the difference between the prediction model evaluation index of the first sample pair and the reference model evaluation index, for example, the first model parameters and the second model parameters of the simulator are adjusted, and the adjusted simulator is continued to be trained until the training end condition is met, that is, when the prediction model evaluation index of the first sample pair is infinitely close to or equal to the reference model evaluation index of the first sample pair, the simulator training is completed, and a target simulator with better training effect is obtained.

[0076] Optionally, a stochastic gradient descent (SGD) algorithm may be used for simulator training until the model converges and meets the training end condition.

[0077] In some implementations, the target simulator can be used to simulate multiple batches of newly added third training samples to obtain prediction model evaluation indicators for different batches of the third training samples; and based on the prediction model evaluation indicators for different batches of the third training samples, the indicator change trend of the third training samples can be determined. In other words, the newly added third training sample data can be simulated based on the target simulator to obtain prediction model evaluation indicators for different batches of the third training samples, thereby determining the indicator change trend and convergence of the third training samples during the training process, that is, the change in the prediction model evaluation indicators of the third training samples. Furthermore, the impact of the third training samples can be determined based on the indicator change trend of the third training samples. Furthermore, the target simulator can predict the impact of new sample data and has good generalization capabilities.

[0078] S304: Process the second training sample and the second test sample based on the target simulator to obtain a prediction model evaluation index of the second sample pair.

[0079] In the embodiment of the present disclosure, the implementation method of step S304 can be implemented by using any of the methods in the various embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0080] S305 , obtaining a target training sample from a plurality of second training samples according to the prediction model evaluation index.

[0081] In the embodiment of the present disclosure, the implementation method of step S305 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0082] In this embodiment, the first coding representation and the second coding representation of the first sample pair are obtained, and the corresponding first vector representation and the second vector representation are determined according to the first coding representation and the second coding representation, respectively. Then, the first model parameters and the second model parameters of the simulator are determined according to the vector representation. The simulator can be flexibly adjusted based on the first model parameters and the second model parameters, and the prediction model evaluation indicators of the current first sample pair can be predicted according to the first model parameters, the second model parameters and the predicted prediction model evaluation indicators of the first sample pair, thereby obtaining the prediction model evaluation indicators of the first sample pair. The simulator is adjusted based on the reference model evaluation indicators obtained from the large language model as a benchmark, that is, the first model parameters or the second model parameters of the simulator are adjusted, and the adjusted simulator is trained to obtain a target simulator, thereby improving the training effect of the target simulator, so that the target simulator can predict more accurate model evaluation indicators, and the prediction effect of the target simulator is better.

[0083] Figure 5 This is a schematic diagram of another sample processing method provided by an embodiment of the present disclosure. Figure 5 As shown, the method includes the following steps:

[0084] S501 , performing multiple batches of training and testing on the large language model to obtain a first data set of the large language model.

[0085] In the embodiment of the present disclosure, the implementation method of step S501 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0086] S502 : Based on the first sample pair and the reference model evaluation index, a simulator of the large language model is trained to obtain a target simulator.

[0087] In the embodiment of the present disclosure, the implementation method of step S502 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0088] S503: Input the second training sample and the second test sample into the target simulator, and encode the second training sample and the second test sample respectively to obtain respective coded representations.

[0089] In some implementations, the second training sample and the second test sample can be encoded based on an encoder in the target simulator to obtain a third encoded representation of the second training sample and a fourth encoded representation of the second test sample. It is understood that the encoder in the target simulator can be based on a pre-trained BERT model for encoding representation.

[0090] S504 : Obtain a prediction model evaluation index of the second sample pair according to the third encoded representation of the second training sample and the fourth encoded representation of the second test sample.

[0091] In some implementations, the third model parameters and fourth model parameters of the simulator can be obtained based on the third encoded representation and the fourth encoded representation; for example, the vector representation of the third encoded representation and the fourth encoded representation is obtained through a multi-order MLP, and the third model parameters and fourth model parameters of the simulator are obtained based on the vector representation, where the third model parameter can be the third model parameter of the simulator at each order.

[0092] For example, the third coding representation and the vector representation of the fourth coding representation output of each order MLP are obtained, and the vector representation of the third coding representation and the fourth coding representation at each order are point multiplied to obtain the third model parameters of the simulator at that order; the vector representation of the third coding representation and the fourth coding representation at one order are point multiplied to obtain the fourth model parameters.

[0093] In some implementations, for the second sample pairs from the same batch, the third model parameters of the second sample pairs from the same batch can be averaged and used as the third model parameters of the simulator at each order, or the fourth model parameters of the second sample pairs from the same batch can be averaged and used as the fourth model parameters of the simulator.

[0094] Furthermore, the prediction model evaluation index of the second sample pair predicted by the target simulator is determined, and the predicted second sample pair and the current second sample pair are continuous in the training sequence; based on the third model parameters and the fourth model parameters, and the prediction model evaluation index of the predicted second sample pair, the prediction model evaluation index of the current second sample pair is obtained.

[0095] Optionally, the prediction model evaluation index of the current second sample pair can be obtained by the following formula:

[0096]

[0097] in, Represents the prediction model evaluation index of the current second sample pair; A represents the prediction model evaluation index of the predicted second sample pair; j (c t ) represents the third model parameter of each order of the target simulator; B(c t ) represents the fourth model parameter of the target simulator.

[0098] S505 , obtaining a target training sample from a plurality of second training samples according to the prediction model evaluation index.

[0099] In some implementations, multiple second sample pairs may be ranked by quality based on the prediction model evaluation index of the second sample pairs, and target training samples with higher quality may be selected from the multiple second training samples based on the quality ranking result.

[0100] It is understood that the prediction model evaluation index can be used to reflect the effectiveness of the training sample. For example, the smaller the loss, the better the training sample's performance; the larger the ROUGE, BLEU, and Acc indicators, the better the training sample's performance. Therefore, when the prediction model evaluation index is better, it means that the corresponding second training sample is better. Therefore, the prediction model evaluation index can be used to sort the samples in descending order to obtain a quality ranking result. Based on the quality ranking result, a target training sample with higher quality can be selected from the multiple second training samples. The target training sample corresponds to a better prediction model evaluation index.

[0101] It can be understood that among the prediction model evaluation indicators, the smaller the loss indicator, the better, and the larger the ROUGE, BLEU, and Acc indicators, the better. The second sample pairs are ranked by quality based on the actual effect of the prediction model evaluation indicators.

[0102] In some implementations, before quality sorting of the second sample pairs, abnormal second sample pairs can be eliminated from multiple second sample pairs based on the prediction model evaluation indicators of the second sample pairs, where the abnormal second sample pairs include second training samples whose prediction model evaluation indicators do not meet the standards, such as samples whose prediction model evaluation indicators are negative; for the remaining second sample pairs in the multiple second sample pairs, quality sorting of the remaining second sample pairs is performed according to the prediction model evaluation indicators of the remaining second sample pairs, that is, the prediction model evaluation indicators of the remaining second sample pairs are arranged in descending order to obtain quality sorting results, and the second training samples that are at the top of the quality sorting results are selected to screen out the target training samples in the remaining second sample pairs, and obtain target training samples with better quality and accuracy.

[0103] In this embodiment, after obtaining a target simulator with better prediction effect, the prediction model evaluation index of the second sample pair is obtained based on the target simulator, and the third model parameters and the fourth model parameters are respectively obtained through the third encoded representation and the fourth encoded representation of the second sample pair. The prediction model evaluation index of the current second sample pair is obtained based on the third model parameters and the fourth model parameters, as well as the predicted prediction model evaluation index of the second sample pair. The obtained prediction model evaluation index is more accurate. Based on the more accurate prediction model evaluation index, the training samples of the second sample pair are screened to obtain target training samples with better quality.

[0104] Figure 6 FIG is a schematic diagram of another sample processing method provided by an embodiment of the present disclosure. Figure 6 As shown, the method includes the following steps:

[0105] S601: Perform multiple batches of training and testing on the large language model to obtain a first data set of the large language model.

[0106] In the embodiment of the present disclosure, the implementation method of step S601 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0107] S602: Input multiple first sample pairs into a simulator for batch processing to obtain prediction model evaluation indicators of the first sample pairs.

[0108] In the embodiment of the present disclosure, the implementation method of step S602 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0109] S603: Adjust the simulator according to the prediction model evaluation index and the reference model evaluation index of the first sample pair, and continue training the adjusted simulator until the training end condition is met to obtain a target simulator.

[0110] In the embodiment of the present disclosure, the implementation method of step S603 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0111] S604: Input the second training sample and the second test sample into the target simulator, and encode the second training sample and the second test sample respectively to obtain respective coded representations.

[0112] In the embodiment of the present disclosure, the implementation method of step S604 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0113] S605 , obtaining a prediction model evaluation index of the second sample pair according to the third encoded representation of the second training sample and the fourth encoded representation of the second test sample.

[0114] In the embodiment of the present disclosure, the implementation method of step S605 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0115] S606: Obtain a target training sample from the plurality of second training samples according to the prediction model evaluation index.

[0116] In the embodiment of the present disclosure, the implementation method of step S606 can be implemented by using any of the methods in the embodiments of the present disclosure, which is not limited here and will not be described in detail.

[0117] In this embodiment, multiple batches of training and testing are performed on the large language model to obtain reference model evaluation indicators of multiple first sample pairs, and the first sample pairs are input into the simulator of the large language model to obtain prediction model evaluation indicators of the first sample pairs. The simulator is adjusted with the reference model evaluation indicators as a benchmark, so that the prediction model evaluation indicators of the first sample pairs are infinitely close to the reference model evaluation indicators, so as to achieve the purpose of simulator training and obtain a trained target simulator. Then, the second training samples and the second test samples are processed according to the target simulator to obtain corresponding encoding representations, and the prediction model evaluation indicators of the second sample pairs are obtained based on the encoding representations. The prediction model evaluation indicators can reflect the quality of the second sample pairs. Therefore, the second sample pairs are screened based on the prediction model evaluation indicators to obtain target training samples with better quality, thereby improving the effect and performance of model training using training samples. New sample data can also be processed based on the target simulator, and the generalization ability is better.

[0118] Figure 7 This is a schematic diagram of the structure of a sample processing device provided by an embodiment of the present disclosure. Figure 7 As shown, the sample processing device 700 includes:

[0119] A first acquisition module 701 is configured to perform multi-batch training and testing on the large language model to obtain a first data set of the large language model, where the first data set includes reference model evaluation indicators of multiple first sample pairs, each first sample pair including a first training sample and a first test sample;

[0120] A second acquisition module 702 is configured to train a simulator of the large language model based on the first sample pair and the reference model evaluation index to obtain a target simulator;

[0121] A third acquisition module 703 is configured to process the second training sample and the second test sample based on the target simulator to obtain a prediction model evaluation index of a second sample pair, where the second sample pair includes the second training sample and the second test sample;

[0122] The sample processing module 704 is configured to obtain a target training sample from the plurality of second training samples according to the prediction model evaluation index.

[0123] In some implementations, the first acquisition module 701 includes:

[0124] Performing multi-batch training on the large language model based on the first training sample in the training sample set;

[0125] The large language model trained in each batch is tested based on the first test sample in the test sample set to obtain a first data set of the large language model.

[0126] In some implementations, the second obtaining module 702 includes:

[0127] Inputting the plurality of first sample pairs into the simulator for batch processing to obtain prediction model evaluation indicators of the first sample pairs;

[0128] The simulator is adjusted according to the prediction model evaluation index and the reference model evaluation index of the first sample pair, and the adjusted simulator is continuously trained until the training end condition is met to obtain a target simulator.

[0129] In some implementations, the second obtaining module 702 includes:

[0130] Encoding the current first training sample to obtain a first encoded representation;

[0131] Encoding the current first test sample to obtain a second encoded representation;

[0132] Obtaining first model parameters and second model parameters of the simulator according to the first coded representation and the second coded representation;

[0133] Determine a prediction model evaluation metric for a first sample pair predicted by the simulator, where the predicted first sample pair is continuous with the current first sample pair in a training sequence;

[0134] The prediction model evaluation index of the current first sample pair is obtained according to the first model parameter, the second model parameter, and the prediction model evaluation index of the predicted first sample pair.

[0135] In some implementations, the second obtaining module 702 includes:

[0136] Obtaining, according to the first coded representation and the second coded representation, a first vector representation of the first training sample at order N and a second vector representation of the first test sample at order N, where order N is a natural number greater than or equal to 1;

[0137] Perform vector dot product on the first vector representation and the second vector representation of each order to obtain the first model parameter of each order of the simulator.

[0138] In some implementations, the second obtaining module 702 includes:

[0139] Obtaining a first-order third vector representation of the first training sample and a first-order fourth vector representation of the first test sample according to the first coded representation and the second coded representation;

[0140] Perform vector dot product on the third vector representation and the fourth vector representation to obtain second model parameters.

[0141] In some implementations, the apparatus 700 further includes:

[0142] The first model parameter of the i-th order of each first sample in the same batch is averaged to obtain a first average value, and the first average value is determined as the first model parameter of the i-th order of the simulator.

[0143] In some implementations, the apparatus 700 further includes:

[0144] The second model parameters of each first sample pair in the same batch are averaged to obtain a second average value, and the second average value is determined as the second model parameter of the simulator.

[0145] In some implementations, the third obtaining module 703 includes:

[0146] Inputting the second training sample and the second test sample into the target simulator, encoding the second training sample and the second test sample respectively to obtain respective encoded representations;

[0147] A prediction model evaluation index of the second sample pair is obtained according to the third encoded representation of the second training sample and the fourth encoded representation of the second test sample.

[0148] In some implementations, the third obtaining module 703 includes:

[0149] Obtaining third model parameters and fourth model parameters of the simulator according to the third coded representation and the fourth coded representation;

[0150] Determining a prediction model evaluation metric for a second sample pair predicted by the target simulator, where the predicted second sample pair is continuous with the current second sample pair in a training sequence;

[0151] The prediction model evaluation index of the current second sample pair is obtained according to the third model parameter and the fourth model parameter and the prediction model evaluation index of the predicted second sample pair.

[0152] In some implementations, the sample processing module 704 includes:

[0153] The plurality of second sample pairs are ranked by quality according to the prediction model evaluation index of the second sample pairs, and a target training sample with higher quality is selected from the plurality of second training samples according to the quality ranking result.

[0154] In some implementations, the sample processing module 704 includes:

[0155] Eliminating abnormal second sample pairs from the plurality of second sample pairs according to the prediction model evaluation index of the second sample pairs, wherein the abnormal second sample pairs include second training samples whose prediction model evaluation index does not meet the standard;

[0156] For the remaining second sample pairs among the plurality of second sample pairs, the remaining second sample pairs are ranked by quality according to the prediction model evaluation indicators of the remaining second sample pairs.

[0157] In some implementations, the apparatus 700 further includes:

[0158] Perform multiple batches of simulations on the newly added third training samples through the target simulator to obtain prediction model evaluation indicators for different batches of the third training samples;

[0159] According to the prediction model evaluation indicators of different batches of the third training samples, the indicator change trend of the third training samples is determined.

[0160] In this embodiment, by performing multiple batches of training and testing on the large language model, a reference model evaluation index of each first sample pair output by the large language model is obtained, and the first sample pairs and the reference model evaluation index are used to train a simulator of the large language model to obtain a target simulator. The target simulator has the same processing capability as the large language model and reduces resource consumption. The second training sample and the second test sample are processed based on the target simulator to obtain the corresponding prediction model evaluation index. According to the prediction model evaluation index, target training samples with better quality can be screened out, the impact of training data on test data can be obtained more accurately, and high-quality target training samples can be screened out, with better processing effect.

[0161] In the technical solutions disclosed herein, the acquisition, storage, and application of user personal information involved comply with the provisions of relevant laws and regulations and do not violate public order and good morals.

[0162] According to an embodiment of the present disclosure, the present disclosure also provides an electronic device, a readable storage medium, and a computer program product.

[0163] Figure 8 A schematic block diagram of an example electronic device 800 that can be used to implement embodiments of the present disclosure is shown. The electronic device is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workstations, personal digital assistants, servers, blade servers, mainframe computers, and other suitable computers. The electronic device can also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices, and other similar computing devices. The components shown herein, their connections and relationships, and their functions are provided as examples only and are not intended to limit the implementation of the present disclosure described and / or claimed herein.

[0164] like Figure 8 As shown, the device 800 includes a computing unit 801, which can perform various appropriate actions and processes according to a computer program stored in a read-only memory (ROM) 802 or a computer program loaded from a storage unit 808 into a random access memory (RAM) 803. Various programs and data required for the operation of the device 800 can also be stored in the RAM 803. The computing unit 801, the ROM 802, and the RAM 803 are connected to each other via a bus 804. An input / output (I / O) interface 805 is also connected to the bus 804.

[0165] Various components in device 800 are connected to I / O interface 805, including an input unit 806, such as a keyboard, mouse, etc.; an output unit 807, such as various types of displays, speakers, etc.; a storage unit 808, such as a magnetic disk, optical disk, etc.; and a communication unit 809, such as a network card, modem, wireless communication transceiver, etc. The communication unit 809 allows device 800 to exchange information / data with other devices via a computer network such as the Internet and / or various telecommunication networks.

[0166] The computing unit 801 can be a variety of general and / or special processing components with processing and computing capabilities. Some examples of the computing unit 801 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various dedicated artificial intelligence (AI) computing chips, various computing units that run machine learning model algorithms, digital signal processors (DSPs), and any appropriate processors, controllers, microcontrollers, etc. The computing unit 801 performs the various methods and processes described above, such as the sample processing method. For example, in some embodiments, the sample processing method can be implemented as a computer software program that is tangibly contained in a machine-readable medium, such as a storage unit 808. In some embodiments, part or all of the computer program can be loaded and / or installed on the device 800 via the ROM 802 and / or the communication unit 809. When the computer program is loaded into the RAM 803 and executed by the computing unit 801, one or more steps of the sample processing method described above can be performed. Alternatively, in other embodiments, the computing unit 801 can be configured to perform the sample processing method by any other appropriate means (e.g., by means of firmware).

[0167] Various embodiments of the systems and techniques described herein can be implemented in digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application specific integrated circuits (ASICs), application specific standard products (ASSPs), system-on-chip systems (SOCs), programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments can include being implemented in one or more computer programs that are executable and / or interpreted on a programmable system that includes at least one programmable processor, which can be a special purpose or general purpose programmable processor that can receive data and instructions from a storage system, at least one input device, and at least one output device, and transmit data and instructions to the storage system, the at least one input device, and the at least one output device.

[0168] The program code for implementing the method of the present disclosure can be written in any combination of one or more programming languages. These program codes can be provided to a processor or controller of a general-purpose computer, a special-purpose computer, or other programmable data processing device so that when the program code is executed by the processor or controller, the functions / operations specified in the flow chart and / or block diagram are implemented. The program code can be executed entirely on the machine, partially on the machine, as a stand-alone software package, partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0169] In the context of the present disclosure, a machine-readable medium can be a tangible medium that can contain or store a program for use by or in conjunction with an instruction execution system, device or equipment. A machine-readable medium can be a machine-readable signal medium or a machine-readable storage medium. A machine-readable medium can include, but is not limited to, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or equipment, or any suitable combination of the foregoing. A more specific example of a machine-readable storage medium can include an electrical connection based on one or more lines, a portable computer disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the foregoing.

[0170] To provide interaction with a user, the systems and techniques described herein can be implemented on a computer having: a display device (e.g., a CRT (cathode ray tube) or LCD (liquid crystal display) monitor) for displaying information to the user; and a keyboard and pointing device (e.g., a mouse or trackball) through which the user can provide input to the computer. Other types of devices can also be used to provide interaction with the user; for example, the feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including acoustic input, voice input, or tactile input).

[0171] The systems and techniques described herein can be implemented in a computing system that includes back-end components (e.g., as a data server), or a computing system that includes middleware components (e.g., an application server), or a computing system that includes front-end components (e.g., a user computer having a graphical user interface or a web browser through which a user can interact with implementations of the systems and techniques described herein), or a computing system that includes any combination of such back-end components, middleware components, or front-end components. The components of the system can be interconnected by any form or medium of digital data communication (e.g., a communication network). Examples of communication networks include a local area network (LAN), a wide area network (WAN), and the Internet.

[0172] A computer system may include a client and a server. The client and server are generally remote from each other and typically interact through a communication network. The client-server relationship arises through computer programs running on the respective computers and having a client-server relationship with each other. The server may be a cloud server, a server in a distributed system, or a server integrated with a blockchain.

[0173] It should be understood that the various forms of the processes shown above can be used to reorder, add, or delete steps. For example, the steps described in this disclosure can be performed in parallel, sequentially, or in a different order, as long as the desired results of the technical solutions disclosed in this disclosure can be achieved. This is not limited herein.

[0174] The above specific embodiments do not constitute a limitation on the scope of protection of this disclosure. Those skilled in the art will appreciate that various modifications, combinations, sub-combinations, and substitutions may be made based on design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this disclosure shall be included within the scope of protection of this disclosure.

Claims

1. A sample processing method, wherein: The method comprises: Performing multiple batches of training and testing on a large language model to obtain a first data set of the large language model, wherein the sample data of the large language model includes text, and the first data set includes reference model evaluation indicators of a plurality of first sample pairs, each of the first sample pairs including a first training sample and a first test sample; Training a simulator of the large language model based on the first sample pairs and the reference model evaluation indicator to obtain a target simulator; Processing a second training sample and a second test sample based on the target simulator to obtain a prediction model evaluation index of a second sample pair, where the second sample pair includes a second training sample and a second test sample; According to the prediction model evaluation indicator, a target training sample is obtained from the plurality of second training samples.

2. The method according to claim 1, wherein The performing of multiple batches of training and testing on the large language model to obtain a first data set of the large language model includes: Performing multi-batch training on the large language model based on the first training samples in the training sample set; The large language model trained in each batch is tested based on the first test sample in the test sample set to obtain a first data set of the large language model.

3. The method according to claim 1, wherein The step of training the simulator of the large language model based on the first sample pair and the reference model evaluation indicator to obtain a target simulator includes: Inputting a plurality of the first sample pairs into the simulator for batch processing to obtain prediction model evaluation indicators of the first sample pairs; The simulator is adjusted according to the prediction model evaluation index of the first sample pair and the reference model evaluation index, and the adjusted simulator is continuously trained until a training end condition is met to obtain the target simulator.

4. The method according to claim 3, wherein: The step of inputting the plurality of first sample pairs into the simulator for batch processing to obtain prediction model evaluation indicators of the first sample pairs includes: Encoding the first current training sample to obtain a first coded representation; Encoding the first test sample to obtain a second encoded representation; Obtaining first model parameters and second model parameters of the simulator according to the first encoded representation and the second encoded representation; Determining a prediction model evaluation index of a first sample pair predicted by the simulator, wherein the predicted first sample pair is continuous with the current first sample pair in training sequence; The prediction model evaluation index of the current first sample pair is obtained according to the first model parameter and the second model parameter, and the prediction model evaluation index of the predicted first sample pair.

5. The method according to claim 4, wherein The method for obtaining the first model parameter of the simulator includes: Obtaining, according to the first coded representation and the second coded representation, a first vector representation of the first training sample of order N and a second vector representation of the first test sample of order N, where the order N is a natural number greater than or equal to 1; Perform vector dot product on the first vector representation and the second vector representation of each order to obtain the first model parameter of each order of the simulator.

6. The method according to claim 4, wherein: The method for obtaining the second model parameter of the simulator includes: Obtaining a first-order third vector representation of the first training sample and a first-order fourth vector representation of the first test sample according to the first encoded representation and the second encoded representation; Perform vector dot product on the third vector representation and the fourth vector representation to obtain the second model parameters.

7. The method according to any one of claims 4 to 6, wherein The method further comprises: The first model parameters of the i-th order of each of the first samples in the same batch are averaged to obtain a first average value, and the first average value is determined as the first model parameter of the i-th order of the simulator.

8. The method according to any one of claims 4 to 6, wherein The method further comprises: The second model parameters of each of the first sample pairs in the same batch are averaged to obtain a second average value, and the second average value is determined as the second model parameter of the simulator.

9. The method according to any one of claims 1 to 6, wherein The processing of the second training sample and the second test sample based on the target simulator to obtain a prediction model evaluation index of the second sample pair includes: Inputting the second training sample and the second test sample into the target simulator, and encoding the second training sample and the second test sample to obtain respective encoded representations; Obtain a prediction model evaluation index of the second sample pair according to the third encoded representation of the second training sample and the fourth encoded representation of the second test sample.

10. The method according to claim 9, wherein: The obtaining, according to the third encoded representation of the second training sample and the fourth encoded representation of the second test sample, a prediction model evaluation index of the second sample pair, includes: Obtaining third model parameters and fourth model parameters of the simulator according to the third coded representation and the fourth coded representation; Determining a prediction model evaluation index of a second sample pair predicted by the target simulator, wherein the predicted second sample pair is continuous with the current second sample pair in a training sequence; The prediction model evaluation index of the current second sample pair is obtained according to the third model parameter and the fourth model parameter, and the prediction model evaluation index of the predicted second sample pair.

11. The method according to any one of claims 1 to 6, wherein: The step of obtaining a target training sample from a plurality of second training samples according to the prediction model evaluation indicator includes: According to the prediction model evaluation index of the second sample pairs, the plurality of second sample pairs are sorted by quality, and the target training sample with higher quality is selected from the plurality of second training samples according to the quality sorting result.

12. The method according to claim 11, wherein The step of ranking the plurality of second sample pairs by quality according to the prediction model evaluation index of the second sample pairs includes: Eliminate abnormal second sample pairs from a plurality of second sample pairs according to the prediction model evaluation index of the second sample pairs, wherein the abnormal second sample pairs include second training samples whose prediction model evaluation index does not meet the standard; For the remaining second sample pairs among the plurality of second sample pairs, the remaining second sample pairs are sorted by quality according to the prediction model evaluation indicators of the remaining second sample pairs.

13. The method according to any one of claims 1 to 6, wherein: The method further comprises: Performing multiple batches of simulations on the newly added third training samples through the target simulator to obtain prediction model evaluation indicators for different batches of the third training samples; According to the prediction model evaluation indicators of different batches of the third training samples, the indicator change trend of the third training samples is determined.

14. A sample processing device, wherein: The device comprises: a first acquisition module, configured to perform multi-batch training and testing on a large language model to obtain a first data set of the large language model, wherein the sample data of the large language model includes text, and the first data set includes reference model evaluation indicators of a plurality of first sample pairs, each of the first sample pairs including a first training sample and a first test sample; A second acquisition module is configured to train a simulator of the large language model based on the first sample pairs and the reference model evaluation index to obtain a target simulator; a third acquisition module, configured to process a second training sample and a second test sample based on the target simulator to obtain a prediction model evaluation index of a second sample pair, where the second sample pair includes a second training sample and a second test sample; A sample processing module is used to obtain a target training sample from multiple second training samples based on the prediction model evaluation index.

15. The device according to claim 14, wherein The first acquisition module includes: Performing multi-batch training on the large language model based on the first training samples in the training sample set; The large language model trained in each batch is tested based on the first test sample in the test sample set to obtain a first data set of the large language model.

16. The device according to claim 14, wherein The second acquisition module includes: Inputting a plurality of the first sample pairs into the simulator for batch processing to obtain prediction model evaluation indicators of the first sample pairs; The simulator is adjusted according to the prediction model evaluation index of the first sample pair and the reference model evaluation index, and the adjusted simulator is continuously trained until a training end condition is met to obtain the target simulator.

17. The device according to claim 16, wherein The second acquisition module includes: Encoding the first current training sample to obtain a first coded representation; Encoding the first test sample to obtain a second encoded representation; Obtaining first model parameters and second model parameters of the simulator according to the first encoded representation and the second encoded representation; Determining a prediction model evaluation index of a first sample pair predicted by the simulator, wherein the predicted first sample pair is continuous with the current first sample pair in training sequence; The prediction model evaluation index of the current first sample pair is obtained according to the first model parameter and the second model parameter, and the prediction model evaluation index of the predicted first sample pair.

18. The device according to claim 17, wherein The second acquisition module includes: Obtaining, according to the first coded representation and the second coded representation, a first vector representation of the first training sample at order N and a second vector representation of the first test sample at order N, where the order N is a natural number greater than or equal to 1; Perform vector dot product on the first vector representation and the second vector representation of each order to obtain the first model parameter of each order of the simulator.

19. The device according to claim 17, wherein The second acquisition module includes: Obtaining a first-order third vector representation of the first training sample and a first-order fourth vector representation of the first test sample according to the first encoded representation and the second encoded representation; Perform vector dot product on the third vector representation and the fourth vector representation to obtain the second model parameters.

20. The device according to any one of claims 17 to 19, wherein The device further comprises: The first model parameters of the i-th order of each of the first samples in the same batch are averaged to obtain a first average value, and the first average value is determined as the first model parameter of the i-th order of the simulator.

21. The device according to any one of claims 17 to 19, wherein: The device further comprises: The second model parameters of each of the first sample pairs in the same batch are averaged to obtain a second average value, and the second average value is determined as the second model parameter of the simulator.

22. The device according to any one of claims 14 to 19, wherein: The third acquisition module includes: Inputting the second training sample and the second test sample into the target simulator, and encoding the second training sample and the second test sample to obtain respective encoded representations; Obtain a prediction model evaluation index of the second sample pair according to the third encoded representation of the second training sample and the fourth encoded representation of the second test sample.

23. The device according to claim 22, wherein The third acquisition module includes: Obtaining third model parameters and fourth model parameters of the simulator according to the third coded representation and the fourth coded representation; Determining a prediction model evaluation index of a second sample pair predicted by the target simulator, wherein the predicted second sample pair is continuous with the current second sample pair in a training sequence; The prediction model evaluation index of the current second sample pair is obtained according to the third model parameter and the fourth model parameter, and the prediction model evaluation index of the predicted second sample pair.

24. The device according to any one of claims 14 to 19, wherein: The sample processing module includes: According to the prediction model evaluation index of the second sample pairs, the plurality of second sample pairs are sorted by quality, and the target training sample with higher quality is selected from the plurality of second training samples according to the quality sorting result.

25. The apparatus according to claim 24, wherein The sample processing module includes: Eliminate abnormal second sample pairs from a plurality of second sample pairs according to the prediction model evaluation index of the second sample pairs, wherein the abnormal second sample pairs include second training samples whose prediction model evaluation index does not meet the standard; For the remaining second sample pairs among the plurality of second sample pairs, the remaining second sample pairs are sorted by quality according to the prediction model evaluation indicators of the remaining second sample pairs.

26. The device according to any one of claims 14 to 19, wherein The device further comprises: Performing multiple batches of simulations on the newly added third training samples through the target simulator to obtain prediction model evaluation indicators for different batches of the third training samples; According to the prediction model evaluation indicators of different batches of the third training samples, the indicator change trend of the third training samples is determined.

27. An electronic device comprising: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 13.

28. A non-transitory computer-readable storage medium storing computer instructions, wherein: The computer instructions are used to cause the computer to execute the method according to any one of claims 1-13.

29. A computer program product comprising a computer program, which, when executed by a processor, implements the steps of the method according to any one of claims 1 to 13.

Images