Model training method and device, equipment, storage medium and program product

Abstract

The invention discloses a model training method and device, equipment, a storage medium and a program product, and relates to the technical field of models. The method comprises the following steps: acquiring a first data set, wherein each data in the first data set carries manually labeled first label information; dividing the first data set into a test set and a training set, and training the initial first model by using the training set to obtain a second model; inputting the data in the test set into a second model to obtain second label information of the data in the test set output by the second model; according to the first label information and the second label information of the data in the test set, eliminating noise data in the test set to obtain a second data set; and performing iterative training on the second model based on the second data set to obtain a trained data check model which is used for evaluating the quality of the data. According to the embodiment of the invention, the model training efficiency can be improved.

Description

Technical Field

[0001] The present application belongs to the field of model technology, and in particular, relates to a model training method, device, equipment, storage medium and program product. Background Art

[0002] In the prior art, a data verification model is usually trained to complete the data verification. In the process of training the data verification model, the original data is often collected from the business system, and the original data is subjected to multi-layer classification sampling and manual labeling to obtain the labeled data. Subsequently, the labeled data is used to perform multi-fold cross-validation, build multiple classifiers, and perform model cross-validation to obtain a data verification model that meets the requirements.

[0003] Since the training method of this model requires the use of multiple classification models for cross-validation, the training process is long and the efficiency of model training is low. Summary of the invention

[0004] The embodiments of the present application provide a model training method, apparatus, device, storage medium and program product, which can solve the problem of low efficiency of existing model training.

[0005] In a first aspect, an embodiment of the present application provides a method for training a model, the method comprising:

[0006] Acquire a first data set, where each data in the first data set has manually annotated first label information;

[0007] Dividing the first data set into a test set and a training set, and using the training set to train the initial first model to obtain a second model;

[0008] Inputting the data in the test set into the second model to obtain second label information of the data in the test set output by the second model;

[0009] According to the first label information and the second label information of the data in the test set, noise data in the test set is eliminated to obtain a second data set;

[0010] The second model is iteratively trained based on the second data set to obtain a trained data verification model, where the data verification model is used to evaluate the quality of the data.

[0011] In some embodiments, the first tag information includes a plurality of first sub-tags and a first distribution probability of each first sub-tag, and the second tag information includes a plurality of second sub-tags and a second distribution probability of each second sub-tag;

[0012] Before removing noise data in the test set according to the first label information and the second label information of the data in the test set to obtain the second data set, the method further includes:

[0013] Determine a joint distribution probability of the data in the test set according to the first sub-label, the first distribution probability, the second sub-label, and the second distribution probability of the data in the test set;

[0014] The noise data in the test set is determined according to the joint distribution probability.

[0015] In some embodiments, determining the noise data in the test set according to the joint distribution probability includes:

[0016] For any second test data in the test set, determine a first distribution probability and a second distribution probability of a target sub-label of the second test data, wherein the target sub-label is the same first sub-label and second sub-label;

[0017] Determine the product of the first distribution probability and the second distribution probability as the joint distribution probability of the target sub-label;

[0018] When the joint distribution probability is less than a probability distribution threshold, the second test data is determined as the noise data.

[0019] In some embodiments, before removing noise data in the test set according to the first label information and the second label information of the data in the test set to obtain the second data set, the method further includes:

[0020] For any first test data in the test set, when the first label information of the first test data is different from the second label information, the first test data is determined to be the noise data.

[0021] In some embodiments, the iterative training of the second model based on the second data set to obtain a trained data verification model includes:

[0022] After the second model is trained i times, a performance evaluation is performed on the second model to obtain a performance score of the second model after the i times of training;

[0023] When the performance score is less than the score threshold, performing denoising again on the second data set;

[0024] The second model is trained again using the second data set after denoising until the performance score of the trained second model is greater than or equal to the score threshold, and the training of the second model is stopped to obtain a trained data verification model.

[0025] In some embodiments, before obtaining the first data set, wherein each data in the first data set has manually annotated first label information, the method further includes:

[0026] Get unlabeled datasets;

[0027] Displaying a label set and a label interval corresponding to the unlabeled data set, so that a user can label the data in the unlabeled data set based on the label set and the label interval;

[0028] In response to a user's input of a label for the unlabeled data set, first label information is added to data in the unlabeled data set to obtain the first data set.

[0029] In a second aspect, an embodiment of the present application provides a model training device, the device comprising:

[0030] An acquisition module, configured to acquire a first data set, wherein each data in the first data set has manually annotated first label information;

[0031] A splitting module, used for dividing the first data set into a test set and a training set, and using the training set to train the initial first model to obtain a second model;

[0032] A pre-training module, used for inputting the data in the test set into the second model to obtain second label information of the data in the test set output by the second model;

[0033] A denoising module, configured to remove noise data in the test set according to first label information and second label information of the data in the test set, so as to obtain a second data set;

[0034] A training module is used to iteratively train the second model based on the second data set to obtain a trained data verification model, and the data verification model is used to evaluate the quality of the data.

[0035] In a third aspect, an embodiment of the present application provides a model training device, the device comprising: a processor and a memory storing computer program instructions;

[0036] The processor implements the above model training method when executing computer program instructions.

[0037] In a fourth aspect, an embodiment of the present application provides a computer storage medium having computer program instructions stored thereon, and when the computer program instructions are executed by a processor, the training method of the model as described above is implemented.

[0038] In a fifth aspect, an embodiment of the present application provides a computer program product, which includes computer program instructions, and when the computer program instructions are executed by a processor, the training method of the model as above is implemented.

[0039] In the present application, by processing and eliminating noise data in steps, a cleaner data set can be obtained, the quality of the data set is improved, and thus the training effect of the model is improved. Moreover, after eliminating the noise data, the model is iteratively trained on a cleaner data set, reducing the model error caused by erroneous or abnormal data. Compared with the prior art, the present application avoids the complexity of building and verifying multiple classifiers by directly eliminating noise data and iteratively training a single model. This not only simplifies the model training process, but also reduces the consumption of computing resources and time cost, thereby significantly improving the speed of model training, thereby improving the efficiency of model training. BRIEF DESCRIPTION OF THE DRAWINGS

[0040] In order to more clearly illustrate the technical solutions of the embodiments of the present application, the drawings required for use in the embodiments of the present application will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0041] Figure 1 It is a flowchart of a training method for a model provided in one embodiment of the present application;

[0042] Figure 2 It is a structural schematic diagram of a training device for a model provided in one embodiment of the present application;

[0043] Figure 3 It is a schematic diagram of the hardware structure of a training device for a model provided in one embodiment of the present application. DETAILED DESCRIPTION

[0044] The features and exemplary embodiments of various aspects of the present application will be described in detail below. In order to make the purpose, technical solutions and advantages of the present application clearer, the present application will be further described in detail below in conjunction with the accompanying drawings and specific embodiments. It should be understood that the specific embodiments described herein are only intended to explain the present application, rather than to limit the present application. For those skilled in the art, the present application can be implemented without the need for some of these specific details. The following description of the embodiments is only to provide a better understanding of the present application by illustrating the examples of the present application.

[0045] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device. In the absence of further restrictions, the elements defined by the statement "include..." do not exclude the existence of other identical elements in the process, method, article or device including the elements.

[0046] It should be noted that the acquisition, storage, use, and processing of data in the technical solution of this application are in compliance with the relevant provisions of national laws and regulations.

[0047] It should be noted that in the embodiments of the present application, certain software, components, models and other existing solutions in the industry may be mentioned, and they should be regarded as exemplary. Their purpose is only to illustrate the feasibility of implementing the technical solution of the present application, but it does not mean that the applicant has or will necessarily use the solution.

[0048] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The embodiments will be described in detail below in conjunction with the accompanying drawings.

[0049] Specifically, in order to solve the problems of the prior art, the embodiments of the present application provide a model training method, device, equipment, storage medium and program product. The following first introduces the model training method provided in the embodiments of the present application.

[0050] Figure 1 A flow chart of a model training method provided by an embodiment of the present application is shown. The method comprises the following steps:

[0051] S110, obtaining a first data set, wherein each data in the first data set carries manually annotated first label information.

[0052] In this embodiment, the first data set is the original data set used to train and evaluate the model. Each data sample in the first data set has been manually annotated and has corresponding first label information, which refers to the category or attribute manually assigned to each data sample, such as the object category annotated for an image, such as "cat" or "dog".

[0053] Specifically, the first data set can be a data set obtained by collecting multiple raw heterogeneous data for users and manually annotating each raw heterogeneous data. Raw heterogeneous data refers to data from different sources and in different formats, which may include structured data (such as table data in a database), semi-structured data (such as JSON files), and unstructured data (such as text, images). The collection methods of raw heterogeneous data may include batch files, databases, business embedding points, streaming data, and other channels.

[0054] After collecting multiple raw heterogeneous data, the multiple raw heterogeneous data can be classified, and then the classified raw heterogeneous data can be stored in a preset database, which can be a HIVE library. Then, the entire data set in the HIVE library can be divided according to certain conditions or dimensions (such as time, region, user group, etc.), that is, full slicing, to obtain full slicing data. Then, a portion of representative sample data can be selected from the full slicing data as an unlabeled data set for manual labeling.

[0055] Specifically, as an optional embodiment, before obtaining the first data set, each data in the first data set has manually annotated first label information, the method further includes:

[0056] Get unlabeled datasets;

[0057] Displaying a label set and a label interval corresponding to the unlabeled data set, so that a user can label the data in the unlabeled data set based on the label set and the label interval;

[0058] In response to a user's input of a label for the unlabeled data set, first label information is added to data in the unlabeled data set to obtain the first data set.

[0059] In this embodiment, the tag set can be a plurality of tags pre-defined according to business needs and scenarios, and users can directly use these tags to label the data in the unlabeled data set. For example, in a credit scoring system, the tag set may include three tags: "high risk", "medium risk" and "low risk". The labeling interval is the specific range or condition defined for the label of the data, which is used to guide the user's labeling process. For example, in the above credit scoring system, the labeling interval of the "high risk" tag can be defined as a credit score between 0 and 300, the "medium risk" tag can be defined as between 301 and 600, and the "low risk" tag can be defined as between 601 and 850.

[0060] Unlabeled data sets can be stored in the database of the device. When a user calls an unlabeled data set from the database, it means that the user is about to label the unlabeled data set. Then, the device can display the label set and label interval corresponding to the unlabeled data set, so that the user can label each data according to the label set and label interval corresponding to the unlabeled data set, that is, add the first label information to each data in the unlabeled data set to obtain the first data set.

[0061] In the above method, by displaying the predefined label set and labeling interval, it helps users to complete the labeling task more efficiently and consistently when labeling unlabeled data, reducing human errors and inconsistencies in subjective judgment. At the same time, such a process ensures the standardization of data labels, improves the quality of the data set and the accuracy of subsequent analysis.

[0062] S120, dividing the first data set into a test set and a training set, and using the training set to train the initial first model to obtain a second model.

[0063] In this embodiment, after obtaining the first data set, the first data set can be divided into two sets of data samples, a test set and a training set. The training set is used to train the model. The model adjusts its own parameters by learning the data features in the training set so that it can recognize or classify new data. The test set is completely independent of the training set. It is used to evaluate the performance of the model and help determine the performance of the model on unseen data.

[0064] Specifically, the training set can be used to train the initial first model to obtain the second model. Specifically, the initial first model is an untrained model. By training with the data and labels of the training set, the model will adjust internal parameters and finally obtain a preliminarily trained second model.

[0065] S130, inputting the data in the test set into the second model to obtain second label information of the data in the test set output by the second model.

[0066] In this embodiment, after the initial first model is preliminarily trained to obtain the second model, the data of the test set can be input into the trained second model, and the second model will predict the data and output the second label information of each sample.

[0067] S140: According to the first label information and the second label information of the data in the test set, remove the noise data in the test set to obtain a second data set.

[0068] In this embodiment, noise data refers to data that may contain labeling errors, outliers, or other interference factors. Since the first label information manually labeled for each data in the test set is known, and the second label information of each data output by the model is obtained, the first label information and the second label information of each data can be compared, and the data with a large difference between the first label information and the second label information can be determined as noise data. After obtaining the noise data, they can be removed from the test set to obtain a cleaner second data set, in which the sample data is more consistent and reliable.

[0069] As an optional embodiment, the first tag information includes a plurality of first sub-tags and a first distribution probability of each first sub-tag, and the second tag information includes a plurality of second sub-tags and a second distribution probability of each second sub-tag;

[0070] Before removing noise data in the test set according to the first label information and the second label information of the data in the test set to obtain the second data set, the method further includes:

[0071] Determine a joint distribution probability of the data in the test set according to the first sub-label, the first distribution probability, the second sub-label, and the second distribution probability of the data in the test set;

[0072] The noise data in the test set is determined according to the joint distribution probability.

[0073] In this embodiment, the first label information refers to manually annotated data label information, which includes multiple first sub-labels and their corresponding first distribution probabilities. The first distribution probability indicates the possibility of each first sub-label. Each first sub-label represents a category to which the data may belong. For example, a sample data in the test set may have a 70% probability of being labeled as "category A" and a 30% probability of being labeled as "category B". Then "category A" and "category B" are both first sub-labels, the first distribution probability of "category A" is 70%, and the first distribution probability of "category B" is 30%. The second label information refers to the data label information predicted by the model, which also includes multiple sub-labels and their corresponding distribution probabilities. This distribution probability is generated by the prediction results of the model on the test set.

[0074] The joint distribution probability refers to the probability of the combination of the first sub-label and the second sub-label of the data in the test set. For example, if the first label information of a sample indicates that it has a 70% probability of belonging to "category A", and the second label information indicates that it has a 60% probability of being predicted as "category A", then the joint distribution probability of the label "category A" is the product of these two probabilities. The joint distribution probability calculates the probability of each sub-label appearing simultaneously under manual annotation and model prediction.

[0075] Based on the calculated joint distribution probability, identify the noise data in the test set. Noise data usually refers to data points with a low joint distribution probability, that is, samples with large differences between manual annotations and model predictions. These samples may be mislabeled, inconsistent, or otherwise abnormal, and are therefore considered noise data. After identifying the noise data, remove it from the test set to obtain a cleaner and more consistent second dataset. This new dataset should be more representative of the actual situation and reduce the errors caused by noise data.

[0076] In the above embodiment, by calculating the joint distribution probability of the first label and the second label, noise data in the test set can be effectively identified and eliminated, thereby improving the quality of the data set.

[0077] As an optional embodiment, determining the noise data in the test set according to the joint distribution probability includes:

[0078] For any second test data in the test set, determining a first distribution probability and a second distribution probability of a target sublabel of the second test data;

[0079] Determine the product of the first distribution probability and the second distribution probability as the joint distribution probability of the target sub-label;

[0080] When the joint distribution probability is less than a probability distribution threshold, the second test data is determined as the noise data.

[0081] In this embodiment, for any second test data in the test set, the target sub-label of the sample is first determined, that is, the same label as the model and the manual label. Then, the first distribution probability of the manual label and the second distribution probability predicted by the model corresponding to the target sub-label can be found, and then the first distribution probability and the second distribution probability are multiplied to obtain the joint distribution probability of the target sub-label. This probability represents the possibility that the sample meets both the manual labeling and the model prediction. The calculated joint distribution probability is compared with the preset probability distribution threshold. If the joint distribution probability is lower than this threshold, it means that the difference between the manual labeling and the model prediction of the sample is large, and there may be anomalies. Therefore, the sample is marked as noise data.

[0082] For example, the second test data has a target sub-label of "category A", the first distribution probability of category A is 70%, and the second distribution probability is 60%. The joint distribution probability is 0.7×0.6=0.42. If the probability distribution threshold is set to 50%, the joint distribution probability of 42% is less than 50%, and the sample will be marked as noise data.

[0083] Through the above method, we can effectively identify those noisy data that are inconsistent with manual annotations and model predictions and may be problematic, thereby improving the overall quality of the data set.

[0084] As an optional embodiment, before removing noise data in the test set according to the first label information and the second label information of the data in the test set to obtain the second data set, the method further includes:

[0085] For any first test data in the test set, when the first label information of the first test data is different from the second label information, the first test data is determined to be the noise data.

[0086] In this embodiment, the noise data in the test set can be identified by comparing the manually annotated first label information of each data sample in the test set with the second label information predicted by the model. Specifically, if the manually annotated label of a first test data is inconsistent with the label predicted by the model, that is, the first label information and the second label information are different, then the first test data will be marked as noise data. Noise data may represent labeling errors or data anomalies, so it needs to be removed from the data set to improve the overall quality of the data set.

[0087] Through the above method, data samples with inconsistent labels and predictions can be effectively identified and eliminated, reducing the negative impact of noise data on model training, thereby improving the accuracy and generalization ability of the model.

[0088] S150, iteratively training the second model based on the second data set to obtain a trained data verification model, where the data verification model is used to evaluate data quality.

[0089] In this embodiment, iterative training refers to further training the second model using the second data set after removing the noise data. Through multiple iterative training, the model can better adapt to the data and improve its performance and accuracy. The final data verification model is a carefully trained model used to evaluate and detect the quality of new data, such as identifying anomalies or errors in the data to ensure the accuracy and consistency of the data.

[0090] Specifically, the data verification model can perform data quality verification and classification evaluation on the input data. Data quality verification refers to the use of the model to automatically check the integrity, accuracy, consistency and other aspects of the input data. The model will identify anomalies, incorrect annotations or missing values ​​in the data and mark these data as "problem data". Data classification evaluation refers to the use of the model to classify the input data and evaluate the accuracy of the classification results. The model classifies the data into predefined categories based on the characteristics of the data, such as classifying users into "high risk", "medium risk" and "low risk" categories.

[0091] Specifically, as an optional embodiment, the iterative training of the second model based on the second data set to obtain a trained data verification model includes:

[0092] After the second model is trained i times, a performance evaluation is performed on the second model to obtain a performance score of the second model after the i-times of training, where i is a positive integer;

[0093] When the performance score is less than the score threshold, performing denoising again on the second data set;

[0094] The second model is trained again using the second data set after denoising until the performance score of the trained second model is greater than or equal to the score threshold, and the training of the second model is stopped to obtain a trained data verification model.

[0095] In this embodiment, during the iterative training of the second model, the performance of the trained second model is evaluated after each training to obtain a performance score of the second model, and whether the training of the second model is completed is determined based on the performance score.

[0096] Specifically, taking the i-th training as an example, the second model can be trained i times using the second data set. After the training, the performance of the second model after the i-th training can be evaluated to obtain a performance score, wherein the performance evaluation can be based on indicators such as accuracy, precision, recall rate or F1 score to measure the performance of the second model on the test set.

[0097] If the performance score of the second model is lower than the preset score threshold, it means that the performance of the second model has not yet met expectations. At this time, the second data set can be denoised again according to the above denoising method to further improve the data quality and eliminate the noise data that may affect the model training. After that, the denoised second data set can be used to continue training the second model, and this process is repeated until the performance score of the second model reaches or exceeds the score threshold, indicating that the model is robust enough and training can be stopped to obtain the final trained data verification model.

[0098] In the above solution, the performance of the model can be gradually improved through the iterative process of repeated denoising and training, ensuring that the final model has good generalization ability on high-quality data. By dynamically adjusting data quality and model training, this method can effectively improve the accuracy and reliability of the model and adapt to different actual data scenarios.

[0099] Based on the model training method provided in the above embodiment, the present application also provides a specific implementation of the model training device. Please refer to the following embodiment.

[0100] In the present application, by processing and eliminating noise data in steps, a cleaner data set can be obtained, the quality of the data set is improved, and thus the training effect of the model is improved. Moreover, after eliminating the noise data, the model is iteratively trained on a cleaner data set, reducing the model error caused by erroneous or abnormal data. Compared with the prior art, the present application avoids the complexity of building and verifying multiple classifiers by directly eliminating noise data and iteratively training a single model. This not only simplifies the model training process, but also reduces the consumption of computing resources and time cost, thereby significantly improving the speed of model training, thereby improving the efficiency of model training.

[0101] See first Figure 2 The model training device 200 provided in the embodiment of the present application includes the following modules:

[0102] An acquisition module 201 is used to acquire a first data set, where each data in the first data set has manually annotated first label information;

[0103] A splitting module 202 is used to divide the first data set into a test set and a training set, and use the training set to train the initial first model to obtain a second model;

[0104] A pre-training module 203, configured to input the data in the test set into the second model to obtain second label information of the data in the test set output by the second model;

[0105] A denoising module 204, configured to remove noise data in the test set according to the first label information and the second label information of the data in the test set, so as to obtain a second data set;

[0106] The training module 205 is used to iteratively train the second model based on the second data set to obtain a trained data verification model, and the data verification model is used to evaluate the quality of the data.

[0107] The device can obtain a cleaner data set by processing and eliminating noise data in steps, thereby improving the quality of the data set and improving the training effect of the model. In addition, after eliminating the noise data, the model is iteratively trained on a cleaner data set, reducing model errors caused by erroneous or abnormal data. Compared with the prior art, the present application avoids the complexity of building and verifying multiple classifiers by directly eliminating noise data and iteratively training a single model. This not only simplifies the model training process, but also reduces the consumption of computing resources and time costs, thereby significantly improving the speed of model training, thereby improving the efficiency of model training.

[0108] As an implementation of the present application, the training device 200 of the above model may also include:

[0109] A first determination module, configured to determine a joint distribution probability of the data in the test set according to the first sub-label, the first distribution probability, the second sub-label, and the second distribution probability of the data in the test set;

[0110] The second determination module is used to determine the noise data in the test set according to the joint distribution probability.

[0111] As an implementation of the present application, the second determination module is specifically used for:

[0112] For any second test data in the test set, determine a first distribution probability and a second distribution probability of a target sub-label of the second test data, wherein the target sub-label is the same first sub-label and second sub-label;

[0113] Determine the product of the first distribution probability and the second distribution probability as the joint distribution probability of the target sub-label;

[0114] When the joint distribution probability is less than a probability distribution threshold, the second test data is determined as the noise data.

[0115] As an implementation of the present application, the training device 200 of the above model may also include:

[0116] The third determination module is configured to determine, for any first test data in the test set, the first test data as the noise data when the first label information of the first test data is different from the second label information.

[0117] As an implementation of the present application, the training module 205 may further include:

[0118] A scoring unit, configured to perform a performance evaluation on the second model after the second model is trained i times, so as to obtain a performance score of the second model after the i times of training;

[0119] a denoising unit, configured to perform denoising processing on the second data set again if the performance score is less than a score threshold;

[0120] The training unit is used to train the second model again using the second data set after denoising until the performance score of the trained second model is greater than or equal to the score threshold, stop training the second model, and obtain a trained data verification model.

[0121] As an implementation of the present application, the training device 200 of the above model may also include:

[0122] The acquisition module 201 is also used to acquire an unlabeled data set;

[0123] A display module, configured to display a label set and a label interval corresponding to the unlabeled data set, so that a user can label the data in the unlabeled data set based on the label set and the label interval;

[0124] The labeling module is used to add first label information to the data in the unlabeled data set in response to the user's labeling input for the unlabeled data set to obtain the first data set.

[0125] The model training device provided in the embodiment of the present invention can implement each step in the above-mentioned method embodiment, and will not be described again here to avoid repetition.

[0126] Figure 3 A schematic diagram of the hardware structure of a training device for a model provided in an embodiment of the present application is shown.

[0127] The model training device may include a processor 1001 and a memory 1002 storing computer program instructions.

[0128] Specifically, the processor 1001 may include a central processing unit (CPU), or an application specific integrated circuit (ASIC), or may be configured to implement one or more integrated circuits of the embodiments of the present application.

[0129] The memory 1002 may include a large capacity memory for data or instructions. By way of example and not limitation, the memory 1002 may include a hard disk drive (HDD), a floppy disk drive, a flash memory, an optical disk, a magneto-optical disk, a magnetic tape, or a universal serial bus (USB) drive or a combination of two or more of these. In appropriate cases, the memory 1002 may include a removable or non-removable (or fixed) medium. In appropriate cases, the memory 1002 may be inside or outside the integrated gateway disaster recovery device. In a specific embodiment, the memory 1002 is a non-volatile solid-state memory.

[0130] The memory may include read-only memory (ROM), random access memory (RAM), magnetic disk storage media devices, optical storage media devices, flash memory devices, electrical, optical or other physical / tangible memory storage devices. Thus, typically, the memory includes one or more tangible (non-transitory) computer-readable storage media (e.g., memory devices) encoded with software including computer-executable instructions, and when the software is executed (e.g., by one or more processors), it is operable to perform the operations described with reference to the method according to an aspect of the present disclosure.

[0131] The processor 1001 implements any one of the model training methods in the above embodiments by reading and executing computer program instructions stored in the memory 1002.

[0132] In one example, the model training device may further include a communication interface 1003 and a bus 1010. Figure 3 As shown, the processor 1001, the memory 1002, and the communication interface 1003 are connected via a bus 1010 and communicate with each other.

[0133] The communication interface 1003 is mainly used to implement communication between various modules, devices, units and / or equipment in the embodiments of the present application.

[0134] Bus 1010 includes hardware, software or both, and the parts of the training equipment of model are coupled to each other.For example, but not limitation, bus may include accelerated graphics port (AGP) or other graphics bus, enhanced industrial standard architecture (EISA) bus, front side bus (FSB), hypertransport (HT) interconnection, industrial standard architecture (ISA) bus, infinite bandwidth interconnection, low pin count (LPC) bus, memory bus, micro channel architecture (MCA) bus, peripheral component interconnection (PCI) bus, PCI-Express (PCI-X) bus, serial advanced technology attachment (SATA) bus, video electronics standard association local (VLB) bus or other suitable bus or two or more of these combinations. In appropriate cases, bus 1010 may include one or more buses. Although the present application embodiment describes and shows a specific bus, the application considers any suitable bus or interconnection.

[0135] The training device of the model can be based on the above-mentioned embodiment, so as to realize the training method and apparatus combined with the above-mentioned model.

[0136] In addition, in combination with the training method of the model in the above embodiment, the embodiment of the present application may provide a computer storage medium to implement. The computer storage medium stores computer program instructions; when the computer program instructions are executed by the processor, the training method of any one of the models in the above embodiment is implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here. Among them, the above-mentioned computer-readable storage medium may include a non-transitory computer-readable storage medium, such as a read-only memory (Read-Only Memory, referred to as ROM), a random access memory (Random Access Memory, referred to as RAM), a disk or an optical disk, etc., which is not limited here.

[0137] In addition, an embodiment of the present application further provides a computer program product, including computer program instructions, which can implement the steps and corresponding contents of the aforementioned method embodiment when the computer program instructions are executed by a processor.

[0138] It should be clear that the present application is not limited to the specific configuration and processing described above and shown in the figures. For the sake of simplicity, a detailed description of the known method is omitted here. In the above embodiments, several specific steps are described and shown as examples. However, the method process of the present application is not limited to the specific steps described and shown, and those skilled in the art can make various changes, modifications and additions, or change the order between the steps after understanding the spirit of the present application.

[0139] The functional blocks shown in the above block diagram can be implemented as hardware, software, firmware or a combination thereof. When implemented in hardware, it can be, for example, an electronic circuit, an application specific integrated circuit (ASIC), appropriate firmware, a plug-in, a function card, etc. When implemented in software, the elements of the present application are programs or code segments that are used to perform the required tasks. The program or code segment can be stored in a machine-readable medium, or transmitted on a transmission medium or a communication link by a data signal carried in a carrier wave. "Machine-readable medium" can include any medium capable of storing or transmitting information. Examples of machine-readable media include electronic circuits, semiconductor memory devices, ROM, flash memory, erasable ROM (EROM), floppy disks, CD-ROMs, optical disks, hard disks, optical fiber media, radio frequency (RF) links, etc. The code segment can be downloaded via a computer network such as the Internet, an intranet, etc.

[0140] It should also be noted that the exemplary embodiments mentioned in this application describe some methods or systems based on a series of steps or devices. However, this application is not limited to the order of the above steps, that is, the steps can be performed in the order mentioned in the embodiment, or in a different order from the embodiment, or several steps can be performed simultaneously.

[0141] Aspects of the present disclosure are described above with reference to the flowchart and / or block diagram of the method, device and computer program product according to the embodiment of the present disclosure. It should be understood that each box in the flowchart and / or block diagram and the combination of each box in the flowchart and / or block diagram can be implemented by computer program instructions. These computer program instructions can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device to produce a machine so that these instructions executed by the processor of the computer or other programmable data processing device enable the implementation of the function / action specified in one or more boxes of the flowchart and / or block diagram. Such a processor can be, but is not limited to, a general-purpose processor, a special-purpose processor, a special application processor, or a field programmable logic circuit. It can also be understood that each box in the block diagram and / or flowchart and the combination of boxes in the block diagram and / or flowchart can also be implemented by dedicated hardware that performs a specified function or action, or can be implemented by a combination of dedicated hardware and computer instructions.

[0142] The above are only specific implementation methods of the present application. Those skilled in the art can clearly understand that for the convenience and simplicity of description, the specific working processes of the systems, modules and units described above can refer to the corresponding processes in the aforementioned method embodiments, and will not be repeated here. It should be understood that the protection scope of the present application is not limited to this. Any technician familiar with the technical field can easily think of various equivalent modifications or replacements within the technical scope disclosed in this application, and these modifications or replacements should be included in the protection scope of this application.

Claims

1. A model training method, characterized in that: The method comprises: Acquire a first data set, where each data in the first data set has manually annotated first label information; Dividing the first data set into a test set and a training set, and using the training set to train the initial first model to obtain a second model; Inputting the data in the test set into the second model to obtain second label information of the data in the test set output by the second model; According to the first label information and the second label information of the data in the test set, noise data in the test set is eliminated to obtain a second data set; The second model is iteratively trained based on the second data set to obtain a trained data verification model, where the data verification model is used to evaluate the quality of the data.

2. The training method of the model according to claim 1, characterized in that: The first tag information includes a plurality of first sub-tags and a first distribution probability of each first sub-tag, and the second tag information includes a plurality of second sub-tags and a second distribution probability of each second sub-tag; Before removing noise data in the test set according to the first label information and the second label information of the data in the test set to obtain the second data set, the method further includes: Determine a joint distribution probability of the data in the test set according to the first sub-label, the first distribution probability, the second sub-label, and the second distribution probability of the data in the test set; The noise data in the test set is determined according to the joint distribution probability.

3. The training method of the model according to claim 2, characterized in that: The determining the noise data in the test set according to the joint distribution probability comprises: For any second test data in the test set, determine a first distribution probability and a second distribution probability of a target sub-label of the second test data, wherein the target sub-label is the same first sub-label and second sub-label; Determine the product of the first distribution probability and the second distribution probability as the joint distribution probability of the target sub-label; When the joint distribution probability is less than a probability distribution threshold, the second test data is determined as the noise data.

4. The training method of the model according to claim 1, characterized in that: Before removing noise data in the test set according to the first label information and the second label information of the data in the test set to obtain the second data set, the method further includes: For any first test data in the test set, when the first label information of the first test data is different from the second label information, the first test data is determined to be the noise data.

5. The training method of the model according to claim 1, characterized in that: The iterative training of the second model based on the second data set to obtain a trained data verification model includes: After the second model is trained i times, a performance evaluation is performed on the second model to obtain a performance score of the second model after the i times of training; When the performance score is less than the score threshold, performing denoising again on the second data set; The second model is trained again using the second data set after denoising until the performance score of the trained second model is greater than or equal to the score threshold, and the training of the second model is stopped to obtain a trained data verification model.

6. The training method of the model according to claim 1, characterized in that: Before obtaining the first data set, wherein each data in the first data set has first label information that is manually annotated, the method further includes: Get unlabeled datasets; Displaying a label set and a label interval corresponding to the unlabeled data set, so that a user can label the data in the unlabeled data set based on the label set and the label interval; In response to a user's input of a label for the unlabeled data set, first label information is added to data in the unlabeled data set to obtain the first data set.

7. A model training device, characterized in that: The device comprises: An acquisition module, configured to acquire a first data set, wherein each data in the first data set has manually annotated first label information; A splitting module, used for dividing the first data set into a test set and a training set, and using the training set to train the initial first model to obtain a second model; A pre-training module, used for inputting the data in the test set into the second model to obtain second label information of the data in the test set output by the second model; A denoising module, configured to remove noise data in the test set according to first label information and second label information of the data in the test set, so as to obtain a second data set; A training module is used to iteratively train the second model based on the second data set to obtain a trained data verification model, and the data verification model is used to evaluate the quality of the data.

8. A model training device, characterized in that: The training device of the model includes: a processor and a memory storing computer program instructions; When the processor executes the computer program instructions, it implements the training method of the model as described in any one of claims 1-6.

9. A computer storage medium, characterized in that The computer storage medium stores computer program instructions, which, when executed by a processor, implement the training method for the model as described in any one of claims 1 to 6.

10. A computer program product, characterized in that The computer program product comprises computer program instructions, which, when executed by a processor, implement the training method of the model described in any one of claims 1 to 6.

Images