A training data acquisition method, system, device, storage medium and program
The method optimizes large model training data acquisition by using smaller models to evaluate preprocessing methods, reducing information loss and resource use while improving efficiency.
Patent Information
- Application Number
- CN202311859600.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-12-30
- Publication Date
- 2024-12-03
- Estimated Expiration
- 2043-12-30
AI Technical Summary
Large model training requires extensive data processing that can damage useful information and is computationally intensive, leading to inefficiencies and high resource consumption.
A method involving multiple candidate processing methods to preprocess data for large models using smaller machine learning models, evaluating these methods to determine an optimal approach, and applying it to acquire training data efficiently.
Reduces information loss and resource consumption by identifying the best preprocessing techniques for large models, enhancing data processing efficiency and quality.
Smart Images

Figure CN117910603B_ABST
Abstract
Description
Technical Field
[0001] The present specification relates to the field of artificial intelligence, and in particular to a method, system, device, storage medium and program for acquiring training data, as well as a distributed system for acquiring training data. Background Art
[0002] The amount of data required for large model training is very large. Before training a large model, it is usually necessary to process the original data to obtain training data. In the process of processing the original data to obtain training data, information in the original data that is useful for large model training may be damaged. For example, different types of data may be suitable for different processing methods. When the original data is processed by an inappropriate processing method, the generated training samples will be unfavorable for large model training. For example, information useful for large model training is damaged, which affects the accuracy or quality of the large model. In addition, due to the large amount of data required for large model training, the processing of original data into training data requires a large amount of computing power, for example, more than one thousand cores, and actually about four or five thousand cores, in order to process the original data into the required training data as quickly as possible, which leads to low data processing efficiency and high resource consumption.
[0003] Therefore, the present invention proposes a training data acquisition method, which can determine the optimal processing method for different types of data, so as to process the original data into training data that is easy for large models to learn, reduce the loss of useful information in the data, and reduce resource consumption. Summary of the invention
[0004] One or more embodiments of the present specification provide a method for obtaining training data. The method for obtaining training data includes: obtaining an initial data set, the initial data set including data of the same type; processing the data in the initial data set using multiple candidate processing methods to obtain multiple data sets; training a first machine learning model based on the multiple data sets to obtain a trained first machine learning model; and evaluating the multiple candidate processing methods based on the trained first machine learning model to determine a target processing method, the target processing method being used to process pre-trained data to obtain training data for a second machine learning model, wherein the size of the first machine learning model is smaller than the size of the second machine learning model.
[0005] One or more embodiments of the present specification provide a distributed system for acquiring training data, the distributed system comprising a computing cluster, the computing cluster comprising a plurality of computing devices; a manager, configured to acquire raw data, classify the raw data based on data types to obtain a plurality of initial data sets; and send the plurality of initial data sets to the plurality of computing devices, respectively, any computing device among the plurality of computing devices being configured to execute a training data acquisition method based on the received initial data sets.
[0006] One or more embodiments of the present specification provide a training data acquisition system, including an acquisition module, a processing module, a training module and an evaluation module; the acquisition module is used to acquire an initial data set, and the initial data set includes data of the same type; the processing module is used to process the data in the initial data set using multiple candidate processing methods to obtain multiple data sets, and each data set in the multiple data sets corresponds to one of the multiple candidate processing methods; the training module is used to train a first machine learning model based on each data set to obtain a trained first machine learning model; and the evaluation module is used to evaluate the multiple candidate processing methods using the trained first machine model to determine a target processing method, and the target processing method is used to process pre-trained data to obtain training data for a second machine learning model, wherein the size of the first machine learning model is smaller than the size of the second machine learning model.
[0007] One or more embodiments of the present specification provide a training data acquisition device, including a processor, wherein the processor is configured to execute a training data acquisition method.
[0008] One or more embodiments of the present specification provide a computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes a training data acquisition method.
[0009] One or more embodiments of the present specification provide a computer program product, including a computer program or a computer executable instruction, wherein when the computer program or the computer executable instruction is executed by a processor, a training data acquisition method is implemented. BRIEF DESCRIPTION OF THE DRAWINGS
[0010] This specification will be further described in the form of exemplary embodiments, which will be described in detail by way of the accompanying drawings. These embodiments are not restrictive, and in these embodiments, the same number represents the same structure.
[0011] Figure 1 is an exemplary flow chart of a method for acquiring training data according to some embodiments of this specification.
[0012] Figure 2 is an exemplary flow chart of a method for acquiring training data according to some embodiments of this specification.
[0013] Figure 3 This is an exemplary flow chart of a method for acquiring training data according to some embodiments of this specification.
[0014] Figure 4 It is a schematic diagram of a training data acquisition method according to some embodiments of this specification.
[0015] Figure 5 is an exemplary structural diagram of a training data acquisition system according to some embodiments of this specification.
[0016] Figure 6 is an exemplary structural diagram of a distributed processing system for acquiring training data according to some embodiments of this specification. DETAILED DESCRIPTION
[0017] In order to more clearly illustrate the technical solutions of the embodiments of this specification, the following is a brief introduction to the drawings required for the description of the embodiments. Obviously, the drawings described below are only some examples or embodiments of this specification. For ordinary technicians in this field, this specification can also be applied to other similar scenarios based on these drawings without creative work. Unless it is obvious from the language environment or otherwise explained, the same reference numerals in the figures represent the same structure or operation.
[0018] It should be understood that the "system", "device", "unit" and / or "module" used herein are a method for distinguishing different components, elements, parts, portions or assemblies at different levels. However, if other words can achieve the same purpose, the words can be replaced by other expressions.
[0019] As shown in this specification and claims, unless the context clearly indicates an exception, the words "a", "an", "an" and / or "the" do not refer to the singular and may also include the plural. Generally speaking, the terms "comprise" and "include" only indicate the inclusion of the steps and elements that have been clearly identified, and these steps and elements do not constitute an exclusive list. The method or device may also include other steps or elements.
[0020] Flowcharts are used in this specification to illustrate the operations performed by the system according to the embodiments of this specification. It should be understood that the preceding or following operations are not necessarily performed precisely in order. Instead, the steps may be processed in reverse order or simultaneously. At the same time, other operations may also be added to these processes, or one or more operations may be removed from these processes.
[0021] Figure 1 is an exemplary flow chart of a training data acquisition method according to some embodiments of this specification. In some embodiments, process 100 may be executed by a processing device. In some embodiments, process 100 may be executed by a processing device. Figure 5 The training method shown obtains the system execution or Figure 6 The distributed system shown is executed.
[0022] Step 110: Acquire an initial data set, where the initial data set includes data of the same type.
[0023] The initial data set refers to the data that has been divided into different types.
[0024] In some embodiments, raw data (also referred to as raw data packets) may be obtained. The raw data may be preliminarily classified to obtain a plurality of first data sets of different types, and any one of the plurality of first data sets of different types may be designated as an initial data set. For example, the raw data may be classified into a numerical data set, a categorical data set, a binary data set, a temporal data set, a text data set, an image data set, an audio data set, a video data set, etc.
[0025] In some embodiments, the first data set may be further subclassified to obtain multiple different types of second data sets. Any one of the multiple different types of second data sets may be designated as the initial data set. For example, the text data set may be classified into a web page data set, a document data set, a forum data set, a topic data set, etc.
[0026] In some embodiments, the second data set may be further classified to obtain a plurality of third data sets of different types, and one of the plurality of third data sets of different types may be designated as the initial data set.
[0027] For example, the second data set can be further classified in detail according to the data format to obtain multiple third data sets of different types. For example, the web page data in the web page data set can be divided into hypertext markup language (HTML), cascading style sheets (CSS), scripting language (JavaScript), scripting language object representation (JSON), extensible markup language (XML), pictures, videos, audio and other multimedia files.
[0028] For another example, the second data set may be further classified in detail according to the data content to obtain a plurality of third data sets of different types. For example, the document data set may be classified into a scientific and technological document data set, a literary and fiction document data set, an educational document data set, a financial document data set, a life guide document data set, etc. For another example, the document data set may be classified into a PDF document, a word document, a text document, etc. according to the data storage format.
[0029] For another example, the second data set may be further classified according to the data attribution location to obtain multiple third data sets of different types, such as dividing the forum data set into the Asia and Pacific Forum data set, the European Forum data set, the North American Forum data set, the South American Forum data set, etc.
[0030] For another example, the second data set can be further classified according to the data representation form to obtain multiple different types of third data sets, such as dividing the question type data set into a judgment type data set, a single-choice type data set, a multiple-choice type data set, a text answer type data set, etc.
[0031] Step 120, using multiple candidate processing methods to process data in the initial data set to obtain multiple data sets. Using multiple candidate processing methods to process data in the same initial data set to obtain multiple data sets. Each of the multiple data sets corresponds to a candidate processing method, and each data set is obtained by processing the initial data set with the corresponding candidate processing method.
[0032] Among them, data processing methods include one or more combinations of data cleaning, data splicing, data conversion, feature selection, data reorganization, data labeling, data enhancement, etc.
[0033] Data cleaning refers to removing erroneous, incomplete or inconsistent parts in the data to improve the quality and accuracy of the data. Data cleaning can include missing value processing, outlier processing, data deduplication, format unification, data integration, data dimension reduction, etc. In some embodiments, the initial data set can be a web page data set, which includes multiple web page data, and the web page data can be cleaned. There are a large number of hyperlinks (URLs) in the web page data, some of which are useful data, and some are useless data (such as hyperlinks of advertisements, etc.). In addition, there is generally a lot of redundant information on the web page, such as the header and footer of the web page will have repeated hyperlink data, and the valid data information is only the title and text in the web page (for example, the news title and text), so the web page data can be deduplicated to extract the useful hyperlink data information, and remove the invalid, repeated and interfering data information.
[0034] Data splicing may include row splicing, column splicing, merging, etc. In some embodiments, the initial data set may include a PDF document data set, and PDF documents with the same subject content may be merged. In some embodiments, the processing method of data splicing includes the data splicing tools used, such as the concat and merge functions in the Pandas library and the JOIN operation in SQL. Using different tools may correspond to different data processing methods.
[0035] Data conversion may include data type conversion, data format conversion, data normalization, data discretization, data transformation (e.g., logarithmic transformation, exponential transformation, etc.), etc. In some embodiments, the initial data set may include a PDF document data set, and the PDF document may be converted into a text document using a data format conversion processing method. In some embodiments, the data processing method may include a conversion technology corresponding to the data conversion. For example, the data processing method may include using a model (e.g., a trained machine learning model) or a tool (e.g., Adobe Acrobat Pro, PDFMiner, etc.) to convert a PDF into a text document.
[0036] Step 130: Based on multiple data sets, train the first machine learning model to obtain a trained first machine learning model.
[0037] As described herein, the first machine learning model is a machine learning model whose parameter amount is less than or equal to a first threshold. The first threshold can be any value less than or equal to 100 megabytes, for example, 90 megabytes, etc. For example, the parameter amount of the first machine learning model is 100,000, or 1 megabyte, or 10 megabytes, or 100 megabytes. The first machine learning model can also be called a small model.
[0038] In some embodiments, the first machine learning model may include a neural network model, such as a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), etc. In some embodiments, the first machine learning model may be an unsupervised learning model. For example, the first machine learning model may include an autoencoder model, a generative latent semantic analysis (LSA) model, a hidden Markov model (HMM), an integrated clustering model, a variational autoencoder (VAE) model, etc.
[0039] In some embodiments, multiple data sets can be respectively input into the same first machine learning model to train the first machine learning model. For example, the training data in multiple data sets can be sequentially input into the first machine learning model to obtain the trained first machine learning model. Furthermore, after the training data in one of the data sets are input into the first machine learning model, the training data in the next data set can be continuously input for training until the training data in all data sets are used to train the first machine learning model. For another example, the training data in each data set can be divided into multiple data. Then, one piece of training data in each data set is sequentially input into the first machine learning model to train the first machine learning model. Then, another piece of training data in each data set is sequentially input until the first machine learning model converges, and the training of the first machine learning model is completed.
[0040] In some embodiments, the first machine learning model may include multiple sub-models. The multiple sub-models may be independent machine learning models or integrated into the first machine learning model. The multiple sub-models may be the same machine learning model. The same machine learning model refers to the same model structure and size. The multiple sub-models may have their own input layers. The training data in the multiple data sets may be input into the multiple sub-models for training. Each sub-model corresponds to a data set, and the sub-model is trained based on the corresponding data set to obtain a trained sub-model. For example, the multiple data sets include data set S1, data set S2, ..., data set Sn, and the sub-model includes sub-model M1, sub-model M2..., sub-model Mn. Data set S1 is used to train sub-model M1 to obtain trained sub-model M1, data set S2 is used to train sub-model M2 to obtain trained sub-model M2, and so on. Data set Sn is used to train sub-model Mn to obtain trained sub-model Mn. Based on this, different sub-models may correspond to a data set and also correspond to a data processing method, that is, each sub-model may be trained by a data set obtained by processing the initial data set by one of the multiple candidate processing methods.
[0041] In some embodiments, the data in each data set can be divided into training data and test data (or verification data), the training data is used to train the first machine learning model, and the test data or verification data can be used to test or verify the trained first machine learning model. For more descriptions on testing the trained first machine learning model, please refer to the subsequent descriptions.
[0042] In some embodiments, the first machine learning model may be trained using a model training method. Typical model training methods may include a gradient descent algorithm, a Bayesian optimization algorithm, an adaptive learning rate method, and the like.
[0043] Step 140, evaluate multiple candidate processing methods based on the trained first machine learning model to determine a target processing method. The target processing method is used to process the pre-trained data to obtain training data for the second machine learning model, wherein the size of the first machine learning model is smaller than the size of the second machine learning model. In some embodiments, the size of the model can be defined by the number of parameters of the model. The smaller size of the first machine learning model than the second machine learning model can also be referred to as the smaller number of parameters of the first machine learning model than the second machine learning model. In some embodiments, the size of the model can also be defined by the required computing power of the model. The greater the required computing power, the larger the model size. The type of the pre-trained data is the same as the data type in the initial data set. For example, if the initial data set includes documents in PDF format, the pre-trained data can be documents in PDF format.
[0044] As described herein, the second machine learning model is a machine learning model having a parameter amount greater than a second threshold. The second threshold may be any value greater than 100 megabytes, for example, 1000 megabytes, etc. For example, the parameter amount of the second machine learning model is 1000 megabytes. The second machine learning model may also be referred to as a large model.
[0045] In some embodiments, the second machine learning model may include a neural network model, such as a deep neural network (DNN), a convolutional neural network (CNN), a recurrent neural network (RNN), etc. In some embodiments, the second machine learning model may be an unsupervised learning model. For example, the second machine learning model may include an autoencoder model, a generative latent semantic analysis (LSA) model, a hidden Markov model (HMM), an integrated clustering model, a variational autoencoder (VAE) model, etc.
[0046] In some embodiments, evaluating multiple candidate processing methods based on the trained first machine learning model to determine the target processing method includes obtaining the performance evaluation results generated in the process of training the first machine learning model based on each data set in the multiple data sets. For example, each data set can be input into the first machine learning model, and the first machine learning model can generate a prediction result, and the prediction result is statistically determined to determine the performance evaluation result corresponding to each data set. The final target processing method is determined by comparing the performance evaluation results corresponding to each data set. For example, the test data in multiple data sets are respectively input into the first machine learning model for training to generate multiple sets of output data (i.e., multiple sets of prediction result sets). For example, multiple data sets may include data sets S1, data sets S2, ..., data sets Sn, and the test data in data sets S1, data sets S2, ..., data sets Sn are sequentially input into the trained first machine learning model to generate prediction result sets D1, prediction result sets D2, ..., prediction result sets Dn. The prediction results in prediction result sets D1, prediction result sets D2, ..., prediction result sets Dn can be statistically calculated to determine the values V1, V2, ..., Vn of the evaluation indicators (e.g., cross entropy). Filter out values that meet preset conditions (e.g., maximum value, minimum value, less than a preset threshold, greater than a preset threshold, within a preset interval), etc. from the values V1, V2, ..., Vn of the evaluation index (e.g., cross entropy). Designate the candidate processing method corresponding to the value of the evaluation index that meets the preset condition as the target processing method. The performance evaluation result of the first neural network model under the training of the data set obtained by the target processing method meets the preset condition.
[0047] In some embodiments, evaluating multiple candidate processing methods based on the trained first machine learning model to determine a target processing method includes obtaining test data for each of multiple data sets, processing the test data in each data set using the trained first machine learning model to obtain a prediction result, determining a performance evaluation result (e.g., a performance parameter value) of the first machine learning model under data set training obtained by processing an initial data set by each of the multiple candidate processing methods based on the prediction result, and determining a target processing method from the multiple candidate processing methods based on the performance evaluation result. The performance evaluation result of the trained first machine learning model under the test data obtained by the target processing method satisfies a preset condition. For more descriptions on how to determine the target processing method, please refer to Figure 2 Detailed description.
[0048] In some embodiments, the first machine learning model includes multiple sub-models of different sizes. Based on each data set in the multiple data sets, training the first machine learning model to obtain the trained first machine learning model includes using the same training data in each data set to train multiple sub-models respectively to obtain multiple trained sub-models. Using the trained first machine model to evaluate multiple candidate processing methods to determine the target processing method includes obtaining the performance evaluation results of multiple sub-models under the training of the data set obtained by each candidate processing method in the multiple candidate processing methods, each candidate processing method corresponds to a performance evaluation result, predicting the performance evaluation results of the second machine learning model under the training data obtained by each candidate processing method based on the performance evaluation results of the multiple sub-models, determining the target processing method based on the performance evaluation results of the second machine learning model under the training data obtained by each candidate processing method, and the performance evaluation results of the second machine learning model under the training of the data set obtained by the target processing method meet the preset conditions.
[0049] Predicting the performance evaluation results of the second machine learning model under the training data obtained by each candidate processing method based on the performance evaluation results of the multiple sub-models includes fitting the performance evaluation results of the multiple sub-models under the data set training obtained by each candidate processing method to generate a correspondence between the model parameters and the performance parameters under the data set training obtained by each candidate processing method; and determining the performance evaluation results of the second machine learning model under the training data obtained by each candidate processing method based on the correspondence between the model parameters and the performance parameters under the data set training obtained by each candidate processing method.
[0050] Determining the performance evaluation result of the second machine learning model under training with the training data obtained by each candidate processing method based on the correspondence between the model parameters and the performance parameters includes obtaining the model parameters of the second machine learning model, the model parameters including at least one of the structural parameters and training parameters of the second machine learning model; determining the estimated performance parameters of the second machine learning model under training with the training data obtained by each candidate processing method based on the correspondence between the model parameters and the performance parameters and the model parameters of the second machine learning model.
[0051] In some embodiments, there are multiple initial data sets, and the data types in the multiple initial data sets are different. The processing device can perform steps 110-140 on each initial data set to determine the target processing method (also called the optimal processing method) for each type of data. In some embodiments, different types of data processing can be different. In some embodiments, different types of data target processing methods can be the same.
[0052] In some embodiments of the present invention, different processing methods are used to process the same type of initial data set to obtain multiple data sets, a small model is trained based on the multiple data sets to obtain a trained small model, and different processing methods are evaluated based on the performance evaluation results of the small model to select a data processing method suitable for large model training. Based on this, the optimal data processing method can be selected to reduce the damage rate of useful information in the process of obtaining large model training data, and the training data most suitable for large model training can be obtained, so that the large model can obtain better performance with fewer training steps. At the same time, by evaluating the candidate processing method through the small model, compared with directly using the large model to evaluate the candidate processing method, it can reduce resource consumption and improve data processing efficiency.
[0053] Figure 2 is an exemplary flow chart of obtaining training data according to some embodiments of this specification. Figure 2 As shown, the process 200 includes the following steps. In some embodiments, the process 200 may be Figure 5 The training data acquisition system shown (eg, evaluation module 540) performs or Figure 6 Some components in the distributed processing system 600 (for example, computing device 610-1, or computing device 610-2, ..., or computing device 610-N) are executed.
[0054] Step 210: Acquire test data from each of the multiple data sets.
[0055] As described in this article, the test data can be used to evaluate the performance of the trained model and can also be called the test set.
[0056] The data types in each data set are the same. Each data set is obtained by processing the initial data set by one of multiple candidate processing methods. For example, the multiple data sets may include data set S1, data set S2, ..., data set Sn, and the candidate processing methods may include candidate processing methods P1, candidate processing methods P2, ..., candidate processing methods Pn, which are respectively used to process the same initial data set to obtain data set S1, data set S2, ..., data set Sn. For more descriptions of the initial data set and the candidate processing methods, see Figure 1 Steps 110 and 120 in the above. For example, the data types in the initial data set are the same, and the initial data set can be obtained by classifying the original data (for example, the original data packet). For another example, the candidate processing methods include data cleaning, data conversion, data concatenation, data conversion, feature selection, data reorganization, data labeling, data enhancement, etc.
[0057] In some embodiments, each data set may be divided into training data (i.e., training set) and test data (test set). For example, before the first machine learning model is trained, part of the data in each data set may be used as test data, and another part of the data may be used as training data. The training data may be used for training the first machine learning model. The test data may be stored in a storage device. The training data is different from the test data.
[0058] In some embodiments, a certain number or proportion of data can be randomly selected from each data set to form a test set. For example, 100 pieces of data can be randomly selected from 10,000 pieces of data as the test set of the data set. The test set is independently stored in a specific storage space and does not participate in any model training, thereby ensuring the independence of the test data set. It also ensures that the first machine learning model has no training memory and training traces of the test data set, thereby improving the accuracy of the verification results.
[0059] In some embodiments, the test set can be backed up to prevent the model training from contaminating the test set. For example, a specific storage space is opened on the server with only read data permission to ensure that the test set will not be contaminated by data and cause interference errors in the model performance evaluation results.
[0060] Step 220, using the trained first machine learning model to process the test data in each data set to obtain a prediction result. The prediction result includes output data generated after the trained first machine learning model processes the test data in each data set. The output data generated after the test set in the same data set is processed by the trained first machine learning model can constitute a prediction result set corresponding to each data set (or test set).
[0061] Using the trained first machine learning model to process the test data in each data set means using each test data as the input of the trained first machine learning model to obtain the output data of the trained first machine model. The test data in each data set can be input into the trained first machine learning model to generate a set of output data (i.e., a set of prediction results). The test data in multiple data sets can be respectively input into the trained first machine learning model to generate multiple sets of output data (i.e., multiple sets of prediction results). For example, multiple data sets may include data set S1, data set S2, ..., data set Sn, and the test data in data set S1, data set S2, ..., data set Sn can be sequentially input into the trained first machine learning model to generate prediction result set D1, prediction result set D2, ..., prediction result set Dn.
[0062] Step 230, based on the prediction results, determines the performance evaluation results of the trained first machine learning model under the training of the data set obtained under each candidate processing method.
[0063] In some embodiments, the performance of the trained first machine learning model can be evaluated based on the prediction results to obtain a performance evaluation result of the trained first machine learning model, and the performance evaluation result may include the values of one or more performance parameters or evaluation indicators.
[0064] Performance evaluation indicators or performance parameters may include accuracy, precision, recall, F1 score, AUC (Area Under the Curve), mean square error (MSE), R2 score, cross-entropy, etc.
[0065] For example, the prediction results in each prediction result set may be statistically calculated to determine the value of the performance parameter of the trained first machine learning model corresponding to the prediction result set. Furthermore, the prediction results in the prediction result set D1, the prediction result set D2, ..., and the prediction result set Dn may be statistically calculated to determine the value V1, V2, ..., Vn of at least one evaluation indicator (e.g., cross entropy).
[0066] Step 240 , based on the performance evaluation result, determine a target processing method from a plurality of candidate processing methods.
[0067] Based on the above description, each data set is obtained by processing the initial data set by one of the candidate processing methods among multiple candidate processing methods, and it can be understood that each data set corresponds to a candidate processing method. Each prediction result set is obtained by processing the test data in a data set by the trained first machine learning model, and it can be understood that each prediction result set corresponds to a data set, that is, it also corresponds to a candidate processing method. Each value of the performance evaluation index is obtained by counting the prediction results in a prediction result set, so it can be understood that the value of each evaluation index can correspond to a prediction result set, that is, it also corresponds to a data set and a candidate processing method. The value of the performance evaluation index can be used to measure the performance of the first machine learning model under the training of the data set obtained by different candidate processing methods. For example, the evaluation index value V1 represents the performance of the first machine learning model under the training of the data set S1 obtained by processing the initial data set by the candidate processing method P1; the evaluation index value V2 represents the performance of the first machine learning model under the training of the data set S2 obtained by processing the initial data set by the candidate processing method P2; and so on, the evaluation index value Vn represents the performance of the first machine learning model under the training of the data set Sn obtained by processing the initial data set by the candidate processing method Pn.
[0068] Therefore, the values of performance evaluation indicators corresponding to multiple candidate processing methods can be compared to determine the values of all performance evaluation indicators that meet the preset conditions, and the candidate processing method corresponding to the performance evaluation indicator value that meets the preset conditions is determined as the target processing method, thereby representing that the performance parameters (i.e., evaluation indicators) of the first machine learning model under the evaluation of the test data obtained by the target processing method meet the preset conditions. The preset condition can be that the value of the performance evaluation indicator is the maximum value or the minimum value of all performance evaluation indicator values, or is greater than a preset threshold, or is less than a preset threshold, or is within a preset range. For example, when the performance evaluation indicator is cross entropy, the candidate processing method corresponding to the maximum value of all cross entropy values is the target processing method.
[0069] In this embodiment, the degree of adaptability of each candidate processing method to the processing of the initial data set is obtained by comparing the performance parameters to determine the optimal processing method for this type of data. For example, a sentence is input into the machine learning model, and the next sentence is outputted by associating or inferring it according to the context and corpus. By comparing the cross entropy values of the output data of the machine learning model, the candidate processing method corresponding to the test data with the lowest cross entropy value is determined as the optimal processing method for the current test set.
[0070] In some embodiments, steps 210-240 may be performed for each type of data set to determine a target processing method (also referred to as an optimal processing method) for each type of data. In some embodiments, the target processing methods for different types of data may be different. In some embodiments, the target processing methods for different types of data may be the same.
[0071] In some embodiments, the first machine learning model trained in step 220 may include multiple trained sub-models. The sub-models have the same size. Each trained sub-model is obtained by training the sub-model with training data in one of the multiple data sets. Step 230 may include inputting the test data in the multiple data sets into the corresponding trained sub-models to obtain the prediction results of each trained sub-model. Then perform step 240. For example, the test data in data sets S1, data sets S2, ..., and data sets Sn may be respectively input into the trained sub-model M1, the trained sub-model M2, ..., and the trained sub-model Mn to obtain the prediction result set D1, the prediction result set D2, ..., and the prediction result set Dn. The prediction result set D1, the prediction result set D2, ..., and the prediction result set Dn correspond to the data sets S1, the data sets S2, ..., and the data sets Sn, respectively. The data sets S1, the data sets S2, ..., and the data sets Sn correspond to the candidate processing methods P1, the candidate processing methods P2, ..., and the candidate processing methods Pn, respectively. The performance evaluation index (e.g., cross entropy) values V1, V2, ..., and Vn of the trained sub-model can be determined based on the prediction result set D1, the prediction result set D2, ..., and the prediction result set Dn, respectively. The performance evaluation index (e.g., cross entropy) values V1, V2, ..., and Vn can be compared to determine the optimal performance evaluation index, and the target prediction result set corresponding to the optimal performance evaluation index can be determined. The corresponding target data set is determined based on the target prediction result set corresponding to the optimal performance parameter, and the candidate processing method corresponding to the target data set is the target processing method.
[0072] By processing this type of data in the optimal processing method, the training data for the large model is determined. The training data determined based on this method is more suitable for the training of the large model, and the processing of the training data is not likely to cause information loss in the training data, thereby improving the quality of the training data for the large model. Since large models consume more computing power and resources, a small model with a scale of only about 1 / 10 or even 1% of the large model is used to evaluate the quality of the data, which can reduce resource consumption.
[0073] Figure 3 is an exemplary flow chart of obtaining training data according to some embodiments of this specification. Figure 3 As shown, the process 300 includes the following steps. In some embodiments, the process 300 may be Figure 3 The training data acquisition system shown (eg, evaluation module 540) performs or Figure 6 Some components in the distributed processing system 600 (for example, computing device 610-1, or computing device 610-2, ..., or computing device 610-N) are executed.
[0074] Step 310: Use the same training data in the same data set to train multiple sub-models of different sizes to obtain multiple trained sub-models. The training data may be at least part of the data in the data set. The data set may be obtained by processing the initial data set in one of multiple candidate methods. The initial data set includes data of the same type. For more information about the initial data set and the candidate processing methods, see Figure 1 Steps 110 and 120 in the above. For example, the data types in the initial data set are the same, and the initial data set can be obtained by classifying the original data (for example, the original data packet). For another example, the candidate processing methods include one or more combinations of data cleaning, data conversion, data concatenation, data conversion, feature selection, data reorganization, data labeling, data enhancement, etc.
[0075] The multiple sub-models may be small models, that is, the multiple sub-models of different sizes may be machine learning models with a size less than 100 megabytes (i.e., the number of parameters is less than 100 megabytes). For example, the multiple sub-models of different sizes may be machine learning models with sizes of 100K, 1 megabyte, 10 megabytes, 100 megabytes, etc. For another example, the multiple sub-models of different sizes may be machine learning models with sizes of 100K, 0.5 megabytes, 1.5 megabytes, 15 megabytes, etc.
[0076] The same training data is input into multiple sub-models of different sizes for training to obtain multiple trained sub-models. In some embodiments, sub-models of different sizes may have different training steps. The number of training steps can be determined based on the size of the sub-model. For example, the smaller the sub-model size, the larger the number of training steps. In some embodiments, sub-models of different sizes may have the same number of training steps. Model training can be performed with a fixed number of training steps or a variable number of training steps for sub-models of different sizes. The number of training steps refers to the number of times the model updates parameters during the training process. The process in which a sub-model learns from the training data each time and updates the model parameters according to the loss function is called one-step training. In deep learning, optimization algorithms such as stochastic gradient descent (SGD) can be used to update parameters. A small batch of training data (mini-batch) is used each time to calculate the gradient and update the parameters. Such a parameter update is a one-step training.
[0077] Step 320, respectively obtain the performance evaluation results of the trained sub-models of different sizes.
[0078] In some embodiments, for any sub-model, during the training process of the sub-model, at each step of training, the sub-model generates a prediction result based on a batch of input training data, and based on the prediction result, the sub-model in the training step is evaluated to generate performance evaluation data (e.g., the value of a performance evaluation index), and the sub-model parameters are updated based on the performance evaluation data. For example, the sub-model can be evaluated based on the prediction result generated by a batch of input training data using a cross entropy loss function, and a cross entropy value (i.e., a statistical value) is generated. For multiple training steps, the sub-model can generate multiple prediction results and multiple performance evaluation data (e.g., multiple values of performance evaluation indicators, such as cross entropy values). Multiple performance evaluation data are related to the number of training steps and the size of the sub-model. Multiple performance evaluation data can be linearly fitted to obtain the relationship between the number of training steps, model size, and performance evaluation index (e.g., cross entropy). In some embodiments, the relationship between the number of training steps, model size, and performance evaluation index (e.g., cross entropy) corresponding to any sub-model can be understood as the performance evaluation result of the sub-model (e.g., a relationship curve between performance parameters-model size-training steps). In some embodiments, the number of training steps, model size, and performance evaluation data (eg, cross entropy) corresponding to any sub-model may be collectively referred to as the performance evaluation result of the sub-model.
[0079] In some embodiments, for any sub-model, the performance of the trained sub-model can be evaluated using the test data in the data set used to train the sub-model to obtain a performance evaluation result. For example, for any trained sub-model, the test data is input into the trained sub-model to generate a prediction result (i.e., a prediction result set), and the performance of the trained sub-model is evaluated based on each prediction result in the prediction result set to generate performance evaluation data. The performance evaluation data, model size, and the number of training steps of the model constitute the performance evaluation result. For example, the prediction results in the prediction result set can be statistically analyzed to generate performance evaluation data (e.g., cross entropy value). For another example, the model performance can be evaluated based on the prediction results using indicators such as mean square error (MSE) and mean absolute error (MAE).
[0080] Step 330, based on the performance evaluation results of the multiple sub-models, predict the performance evaluation results of the second machine learning model under the training data obtained by each candidate processing method.
[0081] The performance evaluation results of the second machine learning model under the training data obtained by each candidate processing method are predicted based on the performance evaluation results of multiple sub-models, and the performance evaluation results of the multiple sub-models under the training of the data set obtained by each candidate processing method are fitted to generate the corresponding relationship between the model parameters and the performance parameters under the training of the data set obtained by each candidate processing method; and the performance evaluation results of the second machine learning model under the training of the training data obtained by each candidate processing method are determined based on the corresponding relationship between the model parameters and the performance parameters.
[0082] Model parameters may include model structure parameters (e.g., the total number of model parameters) and / or training parameters (e.g., the number of training steps). Performance parameters (performance evaluation indicators) may include statistics such as cross entropy, mean square error (MSE), mean absolute error (MAE), etc.
[0083] The correspondence between the model parameters and the performance parameters can be expressed as a functional relationship between the model parameters and the performance parameters. In the functional relationship, the model parameters are independent variables and the performance parameters are dependent variables. For example, the correspondence between the model parameters and the performance parameters can be a functional relationship between the model size, the number of training steps, and the cross entropy. The model size and the number of training steps are the independent variables of the functional relationship, and the cross entropy is the dependent variable of the functional relationship.
[0084] Based on step 320, multiple performance evaluation results of multiple sub-models of different sizes under the same training data training can be obtained. Each size of the sub-model can correspond to a performance evaluation result. For example, the performance evaluation result can be expressed as the value of the performance evaluation index of the sub-model of a specific size under different training steps. The corresponding relationship between the size of the model, the number of training steps and the performance parameters can be obtained by fitting the performance of sub-models of different sizes under different training steps. For another example, the performance evaluation result can be expressed as the relationship between the number of training steps, model size and performance evaluation index (e.g., cross entropy) corresponding to any sub-model (e.g., the relationship curve between performance parameters-model size-training steps). The corresponding relationship between the size of the model, the number of training steps and the performance parameters can be obtained by fitting the relationship between the number of training steps, model size and performance evaluation index (e.g., cross entropy) of sub-models of different sizes (e.g., the relationship curve between performance parameters-model size-training steps); typical fitting algorithms may include least squares method, gradient descent method, etc.
[0085] Determining the performance evaluation result of the second machine learning model under training with the training data obtained by each candidate processing method based on the correspondence between the model parameters and the performance parameters may include obtaining the model parameters of the second machine learning model, the model parameters including at least one of the structural parameters and the training parameters of the model; based on the correspondence between the model parameters and the performance parameters and the model parameters of the second machine learning model, determining the estimated performance parameters of the second machine learning model under training with the training data obtained by each candidate processing method, that is, the performance evaluation result. For example, the correspondence between the model parameters and the performance parameters is a functional relationship between model size-training steps-performance parameters, and the size and training steps of the second machine learning model can be input into the functional relationship between model size-training steps-performance parameters, so as to determine the estimated performance parameters of the second machine learning model under training with the training data obtained by each candidate processing method.
[0086] Step 340, determining a target processing method based on the performance evaluation results of the second machine learning model under the training data obtained by each candidate processing method. The performance evaluation results of the second machine learning model under the training of the data set obtained by the target processing method meet the preset conditions.
[0087] The target processing method is used to process the pre-trained data to obtain training data for the second machine learning model. For more description of the second machine learning model, please refer to Figure 1 Detailed instructions in.
[0088] The multiple data sets are obtained by processing the same initial data set by multiple candidate processing methods. Each data set is obtained by processing the initial data set by one of the multiple candidate processing methods. Steps 310-340 are performed on each of the multiple data sets to obtain the corresponding relationship between the model parameters and the performance parameters corresponding to each data set.
[0089] The size and / or preset number of training steps of the second machine learning model can be acquired in advance. The preset number of training steps can be determined based on actual needs, for example, can be determined based on computing resources. A smaller number of steps can be set for fewer computing resources. Based on the size and / or preset number of training steps of the second machine learning model and the correspondence between multiple model parameters and performance parameters, the performance parameters of the second machine learning model at the size and preset number of training steps are determined. For example, for the correspondence between each model parameter and the performance parameter, a performance parameter of the second machine learning model can be determined based on the size and / or preset number of training steps of the second machine learning model. The performance parameters of multiple second machine learning models are compared to determine the optimal performance parameters. The candidate processing method for the data set corresponding to the optimal performance parameter is determined as the target processing method.
[0090] In some embodiments, for each type of initial data set among the multiple types of initial data sets, different candidate processing methods may be used to process the initial data set of the type to obtain multiple data sets of the type, and each data set may be processed separately through steps 310-340 to obtain a target processing method for the type of data. Figure 4 , after processing the initial data set using the candidate processing method Pi, a data set Si can be obtained, and the data set Si is respectively input into the sub-model M1, sub-model M2, sub-model M3, ..., sub-model Mn to train the sub-models M1, sub-model M2, sub-model M3, ..., sub-model Mn. The training of each sub-model includes a certain number of training steps. The number of training steps of multiple sub-models can be the same or different. In each step of training of each sub-model, sub-model M1, sub-model M2, sub-model M3, ..., sub-model Mn will generate prediction results 1, prediction results 2, prediction results 3, ..., prediction results N respectively. The performance evaluation results 1, performance evaluation results 2, performance evaluation results 3, ..., performance evaluation results N corresponding to each sub-model can be generated based on the prediction results 1, prediction results 2, prediction results 3, ..., prediction results N generated in each step of training in all training steps. For example, the prediction result 1 generated by each training step of sub-model 1 can be used to determine the performance parameters of sub-model 1 after each training step, and then the training steps, size, and performance parameters of sub-model 1 after all training steps of sub-model 1 are fitted to generate performance evaluation result 1 of sub-model 1. The performance evaluation results of other sub-models can also be obtained. Furthermore, performance evaluation result 1, performance evaluation result 2, performance evaluation result 3, ..., performance evaluation result N can be fitted to finally generate a fitting relationship fi between model parameters and performance parameters. The fitting relationship fi between model parameters and performance parameters represents the corresponding relationship between the performance parameters of the model under the training of the data set obtained by the candidate processing method Pi and the model parameters.
[0091] Similarly, the initial data set can be processed using the candidate processing methods P1, P2, ..., Pn to obtain data sets S1, S2, ..., Sn. Then, multiple sub-models are trained based on the data sets S1, S2, ..., Sn to obtain the corresponding relationships f1, f2, ..., fn between the performance parameters of the model under the training of the data sets obtained by processing the candidate processing methods P1, P2, ..., Pn and the model parameters. The performance parameters of the second machine learning model can be calculated by the size and training steps of the second machine learning model using the corresponding relationships f1, f2, ..., fn between the performance parameters and the model parameters, so as to obtain the performance parameters of the second machine learning model under the training of the data sets obtained by processing the candidate processing methods P1, P2, ..., Pn. The optimal performance parameters are determined by comparing the performance parameters of the second machine learning model under the training of the data sets obtained by processing the candidate processing methods P1, P2, ..., Pn, so as to determine the candidate processing method (i.e., the target processing method) corresponding to the optimal performance parameters.
[0092] In this embodiment, the performance evaluation results of the data sets obtained by training the large model under different data processing methods are predicted by the performance evaluation results of the data sets obtained by training the small models of different sizes under different data processing methods, so as to determine the data processing method suitable for the large model training. Since the large model consumes more computing power and resources, a small model with a scale of only about 1 / 10 or even 1% of the large model is used to evaluate the quality of the data, which can reduce resource consumption.
[0093] Figure 5 is a schematic diagram of a training data acquisition system according to some embodiments of this specification. Figure 5 As described, the training data acquisition system 500 may include an acquisition module 510, a processing module 520, a training module 530, and an evaluation module 540. Each module in the training data acquisition system 500, for example, the acquisition module, the processing module, the training module, and the evaluation module may be executed by a computing device or a processing device.
[0094] The acquisition module 510 is used to acquire an initial data set, which includes data of the same type. For more information about acquiring the initial data set, see Figure 1 Step 110 in .
[0095] The processing module 520 is used to process the data in the initial data set using multiple candidate processing methods to obtain multiple data sets, each of which corresponds to one of the multiple candidate processing methods. For more information about the acquisition of data sets and candidate processing methods, please refer to Figure 1 Step 120 in .
[0096] The training module 530 is used to train the first machine learning model based on multiple data sets to obtain a trained first machine learning model. For more information about the training of the first machine learning model, see Figure 1 Step 130 in .
[0097] The evaluation module 540 is used to evaluate the multiple candidate processing methods using the trained first machine model to determine a target processing method, wherein the target processing method is used to process the pre-trained data to obtain training data for the second machine learning model, wherein the size of the first machine learning model is smaller than the size of the second machine learning model. For more information on determining the target candidate method, please refer to Figure 1 Step 140 in .
[0098] One or more embodiments of the present specification provide a computer program product, including a computer program or a computer executable instruction. When the computer program or the computer executable instruction is executed by a processor, a training data acquisition method is implemented.
[0099] Figure 6 Schematic diagram of a distributed processing system according to some embodiments of this specification. Figure 6 As shown, the distributed processing system 600 may include a computing cluster 610, a network 620, a memory 630, a load balancer 650, and a manager 640. The distributed processing system 600 may be used in fields such as natural language processing, video processing, autonomous driving, healthcare, and financial risk control. The components in the distributed processing system 600 may be connected in one or more different ways. As an example only, one or more computing devices of the computing cluster 610 may be connected to the manager 640 and / or the memory 630 via the network 620. As another example, one or more computing devices of the computing cluster 610 may be directly connected to the memory 630, such as Figure 6 Memory 630 may also be connected to manager 640 via network 620. Network 620 may include any suitable network (e.g., a wide area network (WAN), a local area network (LAN), wired and / or wireless network access points, etc.) that may facilitate information and / or data exchange for distributed processing system 600. In some embodiments, network 620 may be the Internet, an intranet, or a combination thereof.
[0100] The computing cluster 610 may include at least two computing devices, such as computing device 610-1, computing device 610-2, ..., computing device 610-N, etc. In some embodiments, the at least two computing devices may be connected and / or communicate with each other via a wireless connection (e.g., network 620), a wired connection, or a combination thereof. For example, computing device 610-1 may receive data processed by computing device 610-2 via network 620. As another example, one of the at least two computing devices (e.g., computing device 610-1) may be directly connected to and / or communicate with the other computing device (e.g., computing device 610-2).
[0101] As used herein, a computing device may also be referred to as a computing node. Each of the at least two computing devices may include at least one processor and at least one storage device. In some embodiments, the computing device (e.g., computing device 610-1) may be any suitable computer, such as a laptop computer, a tablet computer, a desktop computer, etc. At least one processor may include a microcontroller, a microprocessor, a reduced instruction set computer (RISC), an application-specific integrated circuit (ASIC), an application-specific instruction set processor (ASIP), a central processing unit (CPU), a graphics processing unit (GPU), a physical processing unit (PPU), a microcontroller unit, a digital signal processor (DSP), a field programmable gate array (FPGA), an advanced RISC machine (ARM), a programmable logic device (PLD), etc. Any circuit or processor capable of executing one or more functional circuits or processors or any combination thereof. At least one storage device may store data, instructions, and / or any other information. In some embodiments, at least one storage device of the computing device may store data obtained from the load balancer 650 and / or the manager 640. In some embodiments, the storage device may store algorithms and / or instructions, and the at least one processor may execute or use the algorithms and / or instructions to execute the training data acquisition method described in the present application. For example, the processor may execute the training data acquisition method described in processes 100, 200, and 300. For further example, the processor may obtain an initial data set from the load balancer 650, process the data in the initial data set using multiple candidate processing methods to obtain multiple data sets; based on each data set in the multiple data sets, train a first machine learning model to obtain a trained first machine learning model; and evaluate the multiple candidate processing methods based on the trained first machine learning model to determine a target processing method.
[0102] In some embodiments, the processor may also process the received data set based on a determined target processing method to generate training data for training a large model.
[0103] The manager 640 can obtain the original data packet and divide the original data packet into multiple initial data sets based on the data type. The manager 640 can distribute the multiple initial data sets to the computing devices in the computing cluster 610 through the load balancer 650. For more information about data classification, see Figure 1 The computing device may determine the optimal processing method (i.e., target processing method) corresponding to the received initial data set based on the received initial data set. The computing device may also process the data set of this type based on the optimal processing method to obtain training data for training the large model.
[0104] The load balancer 650 allocates at least two computing devices according to a load balancing algorithm or strategy to achieve the acquisition of training data. Exemplary load balancing algorithms may include a round-robin algorithm, a weighted round-robin algorithm, an automatic backup algorithm, a minimum number of connections algorithm, a random algorithm, a weighted random algorithm, etc. In some embodiments, the load balancer 650 may allocate computing devices based on the resources of the computing devices and the amount of data in each initial data set to achieve the acquisition of model training data. In some embodiments, the load balancer 650 may be integrated with the manager 640.
[0105] Memory 630 may store data, instructions, and / or any other information. In some embodiments, memory 630 may store data obtained from load balancer 650 and / or manager 640. For example, memory 630 may store configuration information for each of at least two computing devices. In some embodiments, memory 630 may store data and / or instructions that manager 640 may execute or use to perform the exemplary methods / systems described herein.
[0106] The basic concepts have been described above. Obviously, for those skilled in the art, the above detailed disclosure is only for example and does not constitute a limitation of this specification. Although not explicitly stated here, those skilled in the art may make various modifications, improvements and corrections to this specification. Such modifications, improvements and corrections are suggested in this specification, so such modifications, improvements and corrections still belong to the spirit and scope of the exemplary embodiments of this specification.
[0107] In some embodiments, numbers describing the number of components and attributes are used. It should be understood that such numbers used in the description of the embodiments are modified by the modifiers "about", "approximately" or "substantially" in some examples. Unless otherwise specified, "about", "approximately" or "substantially" indicate that the numbers are allowed to vary by ±20%. Accordingly, in some embodiments, the numerical parameters used in the specification and claims are approximate values, which may change according to the required features of individual embodiments. In some embodiments, the numerical parameters should take into account the specified significant digits and adopt the general method of retaining the digits. Although the numerical domains and parameters used to confirm the breadth of the range in some embodiments of this specification are approximate values, in specific embodiments, the setting of such numerical values is as accurate as possible within the feasible range.
[0108] Finally, it should be understood that the embodiments described in this specification are only used to illustrate the principles of the embodiments of this specification. Other variations may also fall within the scope of this specification. Therefore, as an example and not a limitation, alternative configurations of the embodiments of this specification may be considered consistent with the teachings of this specification. Accordingly, the embodiments of this specification are not limited to the embodiments explicitly introduced and described in this specification.
Claims
1. A method for obtaining training data, comprising: Acquire an initial data set, wherein the initial data set includes data of the same type; The initial data set is one of the following data sets: a text data set, an image data set, an audio data set, and a video data set; Processing the data in the initial data set using a plurality of candidate processing methods to obtain a plurality of data sets; the candidate processing methods include one or more combinations of the following: data cleaning, data concatenation, data conversion, feature selection, data reorganization, data labeling, and data enhancement; Based on the multiple data sets, training the first machine learning model to obtain a trained first machine learning model further includes: using the same training data in each of the data sets to respectively train multiple sub-models of different sizes included in the first machine learning model to obtain multiple trained sub-models; and Based on the trained first machine learning model, the multiple candidate processing methods are evaluated to determine a target processing method, which further includes: determining a performance evaluation result of the trained first machine learning model under each data set training, and determining the target processing method from the multiple candidate processing methods based on the performance evaluation result; wherein the target processing method is used to process pre-trained data to obtain training data for a second machine learning model, and the size of the first machine learning model is smaller than the size of the second machine learning model; Determining the target processing method further includes: obtaining performance evaluation results of the multiple sub-models under the training of the data set obtained by each candidate processing method among the multiple candidate processing methods, each candidate processing method corresponding to a performance evaluation result; Predicting the performance evaluation results of the second machine learning model under the training data obtained by each candidate processing method based on the performance evaluation results of the multiple sub-models, further includes: fitting the performance evaluation results of the multiple sub-models under the training of the data set obtained by each candidate processing method to generate a correspondence between the model parameters and performance parameters under the training of the data set obtained by each candidate processing method; and determining the performance evaluation results of the second machine learning model under the training of the training data obtained by each candidate processing method based on the correspondence between the model parameters and the performance parameters; and The target processing method is determined based on the performance evaluation results of the second machine learning model under the training data obtained under each candidate processing method, wherein the performance evaluation results of the second machine learning model under the training of the data set obtained under the target processing method meet preset conditions.
2. The training data acquisition method according to claim 1, wherein: Evaluating the plurality of candidate processing modes based on the trained first machine model to determine a target processing mode includes: Acquire test data from each of the multiple data sets; Processing the test data in each of the data sets using the trained first machine learning model to obtain a prediction result; Determining, based on the prediction result, a performance evaluation result of the first machine learning model under the training of each data set obtained under each candidate processing manner among the multiple candidate processing manners; and Based on the performance evaluation result, the target processing method is determined from the multiple candidate processing methods, wherein the performance evaluation result of the trained first machine learning model under the test data obtained under the target processing method meets a preset condition.
3. The training data acquisition method according to claim 1, wherein: Determining the performance evaluation result of the second machine learning model under the training data obtained by processing each candidate processing mode based on the corresponding relationship between the model parameters and the performance parameters includes: Acquire model parameters of the second machine learning model, wherein the model parameters include at least one of a structural parameter and a training parameter of the model; Based on the correspondence between the model parameters and the performance parameters and the model parameters of the second machine learning model, determine the estimated performance parameters of the second machine learning model when trained with the training data obtained by each candidate processing method.
4. The training data acquisition method according to claim 1, wherein: Based on each of the multiple data sets, training the first machine learning model to obtain a trained first machine learning model includes: The training data in the multiple data sets are sequentially input into the first machine learning model to obtain the trained first machine learning model.
5. The training data acquisition method according to claim 1, wherein: The first machine learning model includes a plurality of sub-models of the same size, and based on each of the plurality of data sets, training the first machine learning model to obtain a trained first machine learning model includes: The training data in the multiple data sets are respectively input into the multiple sub-models to obtain the trained first machine learning model, and each data set corresponds to a sub-model.
6. The training data acquisition method according to claim 1, wherein: An initial data set is obtained, wherein the initial data set includes data of the same type including: Get the original data; Classifying the original data based on data types to obtain a plurality of the initial data sets; and The multiple initial data sets are respectively sent to multiple computing devices to obtain the initial data sets, and the multiple computing devices are nodes in a distributed system.
7. A distributed system for acquiring training data, comprising: A computing cluster, the computing cluster comprising a plurality of computing devices; A manager, configured to obtain raw data, and classify the raw data based on data types to obtain a plurality of the initial data sets; And the multiple initial data sets are sent to the multiple computing devices respectively, and any computing device among the multiple computing devices is used to execute any one of the training data acquisition methods as claimed in claims 1 to 6 based on the received initial data sets.
8. A training data acquisition system, comprising an acquisition module, a processing module, a training module and an evaluation module; The acquisition module is used to acquire an initial data set, and the initial data set includes data of the same type; The initial data set is one of the following data sets: a text data set, an image data set, an audio data set, and a video data set; The processing module is used to process the data in the initial data set using multiple candidate processing methods to obtain multiple data sets, each of the multiple data sets corresponds to a candidate processing method among the multiple candidate processing methods; the candidate processing methods include one or more combinations of the following: data cleaning, data splicing, data conversion, feature selection, data reorganization, data labeling, and data enhancement; The training module is used to train a first machine learning model based on the multiple data sets to obtain a trained first machine learning model, wherein: The first machine learning model includes a plurality of sub-models of different sizes, and the training module uses the same training data in each data set to train the plurality of sub-models respectively to obtain a plurality of trained sub-models; as well as The evaluation module is used to evaluate the multiple candidate processing methods using the trained first machine model to determine a target processing method, and the evaluation module is further used to: determine the performance evaluation results of the trained first machine learning model under the training of each data set, and determine the target processing method from the multiple candidate processing methods based on the performance evaluation results; wherein the target processing method is used to process pre-trained data to obtain training data for a second machine learning model, and the size of the first machine learning model is smaller than the size of the second machine learning model; The evaluation module is further used to: obtain performance evaluation results of the multiple sub-models under the training of the data set obtained by each candidate processing method among the multiple candidate processing methods, and each candidate processing method corresponds to a performance evaluation result; Predicting the performance evaluation results of the second machine learning model under the training data obtained by each candidate processing method based on the performance evaluation results of the multiple sub-models, further includes: fitting the performance evaluation results of the multiple sub-models under the training of the data set obtained by each candidate processing method to generate a correspondence between the model parameters and performance parameters under the training of the data set obtained by each candidate processing method; and determining the performance evaluation results of the second machine learning model under the training of the training data obtained by each candidate processing method based on the correspondence between the model parameters and the performance parameters; and The target processing method is determined based on the performance evaluation results of the second machine learning model under the training data obtained under each candidate processing method, wherein the performance evaluation results of the second machine learning model under the training of the data set obtained under the target processing method meet preset conditions.
9. A training data acquisition device, comprising a processor, wherein the processor is used to execute the training data acquisition method according to any one of claims 1 to 6.
10. A computer-readable storage medium, wherein the storage medium stores computer instructions. When a computer reads the computer instructions in the storage medium, the computer executes the training data acquisition method according to any one of claims 1 to 6.
11. A computer program product comprising a computer program or computer executable instructions, characterized in that: When the computer program or computer executable instruction is executed by a processor, the training data acquisition method according to any one of claims 1 to 6 is implemented.
