Model data sample intelligent screening method and system for knowledge distillation

Abstract

The present application relates to the model training sample screening field, for solving the problem that the data cannot be effectively screened in the model training process, resulting in the training cost is improved and the training effect is reduced, specifically for the model data sample intelligent screening method and system for knowledge distillation; The present application carries out multistage screening to the basic data for training when training the model, so as to fully remove the data completely ineffective for training, false interference data and high repeatability homogeneous data, simultaneously, the large amount of data is processed by dichotomization, the homogeneous data can be efficiently selected by jumping, both the removal rate of homogeneous data is guaranteed and the workload when removing homogeneous data is reduced, the data for model training can play an optimizing role for model training, meanwhile, the negative interference of false data is reduced, also can avoid a large amount of homogeneous data to make repeated training, improve the data quality, reduce the training cost.

Description

Intelligent filtering method and system for model data samples for knowledge distillation Technical Field

[0001] This invention relates to the field of model training sample selection, specifically to a method and system for intelligent selection of model data samples for knowledge distillation. Background Technology

[0002] Model training is the process in machine learning and deep learning of adjusting model parameters to effectively fit data and improve predictive ability. Its core lies in optimizing the model using datasets and algorithms so that it can learn patterns from the data.

[0003] The foundation of model training is data, which is typically divided into training set, validation set, and test set. The training set is used to adjust parameters, the validation set is used to fine-tune hyperparameters, and the test set is used to evaluate the final performance. The choice of algorithm depends on the task type; for example, linear regression is used for regression problems, and convolutional neural networks are used for image recognition. The algorithm defines how the model learns patterns from the data. Among these factors, data quality, algorithm fit, and computational resources are the core of successful training. At the same time, the amount of data determines the training cost during model training, while data quality determines the training effect. Therefore, controlling the quality and quantity of data samples is the key to ensuring both training effectiveness and cost control during model training.

[0004] Currently, existing technologies typically use database access or web crawlers to acquire massive amounts of data for model training. However, the accuracy of the data obtained through this method cannot be guaranteed, and the sheer volume of data significantly increases training costs. Furthermore, the training results are usually determined using a validation set. This results in a process of training, validating, adjusting, optimizing, and retraining with massive amounts of data, which is costly and yields poor training results. Therefore, a precise data selection method is needed to reduce model training costs and improve training effectiveness.

[0005] To address the aforementioned technical problems, this application proposes a solution. Summary of the Invention

[0006] This invention employs a multi-level screening process for the basic training data during model training. This process effectively removes data that is completely ineffective for training, erroneous interference data, and highly repetitive homogeneous data. This ensures that the data used for model training can optimize the model, while reducing the negative interference of erroneous data. It also avoids repetitive training with a large amount of homogeneous data, improves data quality, and reduces training costs. This invention addresses the problem of ineffective data screening during model training, which leads to increased training costs and decreased training effectiveness. Therefore, it proposes an intelligent model data sample screening method and system for knowledge distillation.

[0007] The objective of this invention can be achieved through the following technical solutions:

[0008] The intelligent screening method for model data samples used in knowledge distillation includes the following steps:

[0009] Step 1: Access the full sample database and filter the samples according to preset conditions to remove completely invalid samples;

[0010] Step 2: Compare the remaining samples from Step 1 with the set interference feature library to obtain the first confidence assessment result of the samples;

[0011] Step 3: Obtain correlation features from the samples obtained in Step 2, and perform credibility aggregation based on the correlation features to obtain the secondary credibility evaluation results of the samples;

[0012] Step 4: Select samples from Step 3 according to the established criteria to obtain the sample application pool;

[0013] Step 5: Perform repeatability verification on the samples in the sample application pool, and skip samples with excessively high repetition rates through repeatability verification.

[0014] Step 6: Use the selected samples after the jump as the final application samples and record the final application samples;

[0015] Step 7: After the model training is completed, repeat the skip selection of samples in Step 5, while avoiding the final application samples recorded in Step 6.

[0016] A model data sample intelligent screening system for knowledge distillation includes a data sample initial screening module, which is used to screen completely invalid samples to obtain a secondary sample database.

[0017] The data interference cleaning module performs a credibility assessment on the secondary sample database, selects samples based on the credibility assessment, and obtains a primary result sample.

[0018] The correlation assessment module performs correlation assessment on a single result sample, corrects the reliability of the data sample through the correlation assessment, and obtains a sample application pool.

[0019] The sample application selection module verifies the sample repeatability of the sample application pool and performs a jump selection on the highly repeatable sample group according to the binary method to obtain the final application sample.

[0020] The coverage verification module records the final application samples and compares them in subsequent training to exclude samples with excessively high similarity.

[0021] In a preferred embodiment of the present invention, after accessing the full sample database, the data sample screening module obtains preset sample screening conditions, which include sample data type, sample feature distribution, and sample domain distribution. The module then performs screening based on these conditions. The sample data type is divided into document data, graphic data, and audio data. The sample feature distribution refers to the records in the sample itself that can describe key technologies. The sample domain distribution refers to the production domain in which the sample is located.

[0022] In a preferred embodiment of the present invention, the data sample screening module performs the following screening steps:

[0023] Comparison of sample screening criteria: If the data data type of the data sample meets the sample data type in the sample screening criteria, it will be retained; otherwise, it will be removed.

[0024] Comparison of sample domain distribution: If the sample domain of the data sample meets the sample domain in the sample screening criteria, it is retained; otherwise, it is removed.

[0025] Sample feature distribution comparison: If the sample feature distribution overlaps with at least one of the preset sample screening conditions, it will be retained; if there is no overlap, it will be recorded as completely invalid and removed.

[0026] In a preferred embodiment of the present invention, the data interference cleaning module can be connected to the interference feature library, which stores erroneous data features. The data interference cleaning module obtains the total number of data feature points in each data sample participating in the comparison, and compares the data feature points with the erroneous data features in the interference feature library to obtain the proportion of the number of overlapping data feature points in the total number of data feature points, which is recorded as the error overlap degree. The reciprocal of the error overlap degree is recorded as the confidence level.

[0027] The data interference cleaning module compares the credibility with the credibility threshold, and records the evaluation as passed or failed based on the comparison result. The data interference cleaning module records the data samples that pass the evaluation as a result sample.

[0028] In a preferred embodiment of the present invention, the correlation assessment module records data samples with similar data sources as a highly correlated sample set. The correlation assessment module obtains the credibility of each data sample in the highly correlated sample set and performs an arithmetic mean on the credibility of the highly correlated sample set to obtain the credibility mean. The correlation assessment module fits the credibility of the data samples in the highly correlated sample set with the credibility mean to obtain the corrected credibility.

[0029] The correlation assessment module compares the corrected credibility with the correction threshold, retains or removes data, and records all retained data samples as a sample application pool.

[0030] In a preferred embodiment of the present invention, the process of performing repeatability verification using the sample application selection module is as follows:

[0031] The sample pool is divided level by level, with each division resulting in two subsets. At each division, the similarity results of the two subsets from the same parent set are compared with a set threshold. If the similarity results are within the set range, the samples are considered similar and no further division is performed. At the same time, the two corresponding subsets are recorded as high repetition rate samples. If they are not within the set range, the samples are considered dissimilar and the division continues. Finally, when the i-th level sub-data pool is reached, all sub-data pools that were not recorded as high repetition rate samples are recorded as normal data.

[0032] When the sample application pool obtains high repetition rate samples, it records one sub-data pool within the high repetition rate samples as the application samples and the other sub-data pool as the skipped samples.

[0033] The sample application pool aggregates application samples and normal data from all high-repetition-rate samples to obtain the final application samples.

[0034] In a preferred embodiment of the present invention, the coverage verification module records the final application samples when the model uses them for training, obtaining historical application samples. Each time a new final application sample is obtained subsequently, the new final application sample is used to perform adversarial training with the recorded historical application samples. If the similarity of the adversarial training is within a set range, the data is deemed invalid and training is not performed, while a data duplication warning is generated. If the similarity exceeds the set range, the data is deemed valid and model training is performed normally.

[0035] Compared with the prior art, the beneficial effects of the present invention are:

[0036] 1. In this invention, when training the model, the basic data used for training is screened at multiple levels to fully remove data that is completely ineffective for training, erroneous interference data, and highly repetitive homogeneous data. This ensures that the data used for model training can not only optimize the model training, but also reduce the negative interference of erroneous data, and avoid repeated training with a large amount of homogeneous data, thereby reducing training costs. This ensures that the data input to the model for training is high-performance data, thus improving data quality.

[0037] 2. In this invention, when filtering data, simultaneous filtering is performed through multiple aspects such as data domain distribution, feature distribution, and condition distribution. This ensures the professionalism of the data domain, the usability of the data distribution, and the validity of the data, establishing a scientific and effective basic sample for subsequent multi-level precise filtering, and reducing the difficulty and workload of subsequent precise filtering.

[0038] 3. In this invention, when performing multi-level data screening, the feature comparison of erroneous data is used to ensure data quality. At the same time, the large amount of data is divided into two parts, which can efficiently select homogeneous data. This not only ensures the removal rate of homogeneous data but also reduces the workload of removing homogeneous data, thereby improving the overall operating efficiency of the system. Attached Figure Description

[0039] To facilitate understanding by those skilled in the art, the present invention will be further described below with reference to the accompanying drawings.

[0040] Figure 1 is a system block diagram of the present invention;

[0041] Figure 2 is a system flowchart of the present invention. Detailed Implementation

[0042] The technical solution of the present invention will be clearly and completely described below with reference to the embodiments. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without creative effort are within the scope of protection of the present invention.

[0043] Example 1:

[0044] Please refer to Figures 1 and 2 for the intelligent screening method of model data samples for knowledge distillation, which includes the following steps:

[0045] Step 1: Access the full sample database and filter the samples in the full sample database according to preset conditions to remove samples that are completely ineffective for model training.

[0046] Step 2: Select the remaining samples from Step 1 and compare them with the set interference feature library. Use the interference feature library to evaluate the credibility of the samples and obtain the first credibility evaluation result of the samples.

[0047] Step 3: Statistically analyze the first credibility assessment results of the samples in Step 2, and obtain the correlation characteristics between different samples. Based on the correlation characteristics, perform credibility clustering to obtain the second credibility assessment results of the samples.

[0048] Step 4: Based on the set standards and the results of the secondary credibility assessment of the samples, select samples with credibility higher than the set standards to obtain the sample application pool;

[0049] Step 5: Perform repeatability verification on the samples in the sample application pool, and skip samples with excessively high repetition rates through repeatability verification.

[0050] Step 6: Use the selected samples after the jump as the final application samples and record the final application samples;

[0051] Step 7: After the model training is completed, repeat the skip selection of samples in Step 5, while avoiding the final application samples recorded in Step 6.

[0052] Example 2:

[0053] Please refer to Figures 1 and 2 for the intelligent screening system of model data samples for knowledge distillation, which includes a data sample initial screening module, a data interference cleaning module, a correlation evaluation module, a sample application selection module, and a coverage verification module.

[0054] The data sample initial screening module is used to access the full sample database, which contains all samples. The samples are obtained through automatic network extraction, manual input, or access to the model database. After accessing the full sample database, the data sample initial screening module obtains preset sample screening conditions, including sample data type, sample feature distribution, and sample domain distribution. The sample data type is divided into document data, graphic data, and audio data. The sample feature distribution is the record of key technologies in the sample itself, and the sample domain distribution is the production field in which the sample is located.

[0055] The initial screening module compares the data samples with the screening criteria. If the data sample's data type matches the screening criteria, it is retained; otherwise, it is discarded.

[0056] The initial screening module compares the data samples with the sample domain distribution in the sample screening criteria. If the sample domain of the data sample matches the sample domain in the sample screening criteria, it is retained; otherwise, it is discarded.

[0057] If the sample feature distribution overlaps with at least one of the preset sample screening conditions, it will be retained; if the sample feature distribution does not overlap with the preset sample screening conditions, it will be recorded as completely invalid and the completely invalid samples will be removed.

[0058] The initial screening module records the proposed samples as samples that are completely ineffective for model training, and retains the remaining samples as a secondary sample database.

[0059] The data interference cleaning module can connect to an interference feature library, which stores erroneous data features. The module analyzes and matches data samples from the secondary sample database with the interference feature library. During this analysis and matching, it first determines the total number X of data feature points in the data samples being compared, then compares these data feature points with the erroneous data features in the interference feature library, recording the number of overlapping data feature points as N. Calculate the proportion of the total number of data feature points, and record it as the error overlap degree E. Then, calculate the reciprocal of the error overlap degree. The record represents the credibility, where j is a non-zero constant;

[0060] The data interference cleaning module obtains a preset confidence threshold and compares the confidence level with the confidence threshold. If the confidence level is greater than the confidence threshold, it is recorded as passing the evaluation. If the confidence level is not greater than the confidence threshold, it is recorded as failing the evaluation. The data interference cleaning module records the data samples that pass the evaluation as a result sample.

[0061] The correlation assessment module acquires a result sample and the data source of each data sample in the result sample. It then verifies the correlation of the data sources and records data samples with similar data sources as a high correlation sample set. The correlation assessment module acquires the credibility of each data sample in a high correlation sample set and calculates the arithmetic mean of the credibility in the high correlation sample set to obtain the credibility mean. The correlation assessment module then performs fitting processing on the credibility of the data samples in the high correlation sample set and the credibility mean to make the credibility of the data samples closer to the credibility mean, thus obtaining the corrected credibility.

[0062] The correlation assessment module compares the corrected confidence level with the correction threshold. If the corrected confidence level is greater than the set correction threshold, the data is retained. If the corrected confidence level is not greater than the set correction threshold, the data is not retained. Finally, all retained data samples are recorded as a sample application pool, where the correction threshold is greater than the confidence level threshold.

[0063] The correlation assessment module makes its judgment on the similarity of data sources for data samples based on the following criteria:

[0064] By obtaining the source information of the data samples, including UTM parameters, tracking ID, or API request header information, if the data sample has source information, the data sample with the same source information is recorded as having a similar source; if the data sample does not have source information, the data sample is recorded as having a dissimilar source.

[0065] The sample application selection module performs repeatability verification on the data samples in the sample application pool. The specific repeatability verification process is as follows:

[0066] The sample application pool is divided into two sub-data pools A1 and A2. One sub-data pool is used as the training set 0 and the other as the test set 1. The test set is distinguished by a binary classification model through adversarial training. The similarity AUC of the two sub-data pools is obtained and recorded. Then, each sub-data pool is divided into two secondary sub-data pools A1a, A1b, A2a, and A2b. The similarity AUC1 of the two secondary sub-data pools A1a and A1b, and the similarity AUC2 of A2a and A2b are compared. This process is repeated multiple times.

[0067] At each division, the obtained similarity result is compared with the set threshold. If the similarity result is within the set range, for example, the range can be set to 0.45 to 0.55, the samples are determined to be similar and no further division is performed. At the same time, the corresponding subsets of the two groups are recorded as high repetition rate samples. If the result is not within the set range, the samples are determined to be dissimilar and the division continues. Finally, when the division reaches the i-th level sub-data pool, the division stops and all sub-data pools that were not recorded as high repetition rate samples are recorded as normal data.

[0068] When the sample application pool obtains high repetition rate samples, it records one sub-data pool within the high repetition rate samples as the application sample and the other sub-data pool as the skipped sample.

[0069] The sample application pool aggregates the application samples and normal data from all high-repetition-rate samples to obtain the final application samples.

[0070] The coverage verification module records the final application samples when the model uses them for training, thus obtaining historical application samples. Each time a new final application sample is obtained, the new final application sample is used to perform adversarial training against the recorded historical application samples. If the similarity of the adversarial training is within a set range, the data is deemed invalid and training is not performed, while a data duplication warning is generated. If the similarity exceeds the set range, the data is deemed valid and model training proceeds normally. The set range can be set to 0.42 to 0.58.

[0071] Thresholds, preset values, or preset ranges are set for result comparison and analysis to determine good or bad. The magnitude of these values ​​is determined by a combination of large-scale model analysis of sample data and human experience, and can also be adjusted appropriately based on seasonal or common-sense influence conditions. Similarly, the weighting ratio coefficients and influence factors are set based on the magnitude of each parameter's influence on the results, and the specific values ​​are allocated to ultimately reflect the impact on the results. These settings are also determined by a combination of large-scale model analysis of sample data and human experience, and can also be adjusted appropriately based on seasonal or common-sense influence conditions.

[0072] The preferred embodiments of the present invention disclosed above are merely illustrative of the invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the invention to any specific implementation. Clearly, many modifications and variations can be made based on the content of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the invention, thereby enabling those skilled in the art to better understand and utilize the invention. The invention is limited only by the claims and their full scope and equivalents.

Claims

1. An intelligent screening system for model data samples used in knowledge distillation, characterized in that, The system includes a data sample initial screening module, which filters out completely invalid samples to obtain a secondary sample database; a data interference cleaning module, which evaluates the credibility of the secondary sample database and selects samples based on the credibility evaluation to obtain primary result samples; and a correlation evaluation module, which evaluates the correlation of the primary result samples and corrects the credibility of the data samples based on the correlation evaluation to obtain a sample application pool. The sample application selection module verifies the sample repeatability of the sample application pool and performs a jump selection on the highly repeatable sample group according to the binary method to obtain the final application sample. The coverage verification module records the final application samples and compares them in subsequent training to exclude samples with excessively high similarity. The data sample initial screening module performs the following steps: Sample screening condition comparison: If the data type of the data sample meets the sample data type in the sample screening conditions, it is retained; otherwise, it is discarded. The sample data types are divided into document data, image data, and audio data. Sample domain distribution comparison: If the sample domain of the data sample meets the sample domain in the sample screening conditions, it is retained; otherwise, it is discarded. Sample feature distribution comparison: If the sample feature distribution overlaps with at least one of the preset sample selection conditions, it is retained; if there is no overlap, it is recorded as completely invalid and discarded. The data interference cleaning module can be connected to the interference feature library, which stores erroneous data features. The data interference cleaning module obtains the total number of data feature points in each data sample participating in the comparison, and compares the data feature points with the erroneous data features in the interference feature library to obtain the proportion of overlapping data feature points in the total number of data feature points, which is recorded as the error overlap degree. The reciprocal of the error overlap degree is recorded as the confidence level. The interference cleaning module compares the confidence level with the confidence level threshold, and records the evaluation result as passed or failed. The data interference cleaning module records the data samples that pass the evaluation as a result sample. The correlation evaluation module records data samples with similar data sources as a high correlation sample set. The correlation evaluation module obtains the confidence level of each data sample in a high correlation sample set, and performs an arithmetic mean on the confidence levels in the high correlation sample set to obtain the confidence level mean. The correlation evaluation module fits the confidence level of the data samples in the high correlation sample set with the confidence level mean to obtain the corrected confidence level. The correlation assessment module compares the corrected credibility with the correction threshold, retains or removes the data, and records all retained data samples as a sample application pool. The process of repeatability verification by the sample application selection module is as follows: the sample application pool is divided into levels, and each division results in two subsets. At each division, the similarity results of the two subsets divided from the same parent set are compared with a set threshold. If the similarity results are within the set interval, the samples are determined to be similar and no further division is performed. At the same time, the two corresponding subsets are recorded as high repetition rate samples. If they are not within the set interval, the samples are determined to be dissimilar and the division continues. Finally, when the i-th level sub-data pool is reached, all sub-data pools that were not recorded as high repetition rate samples are recorded as normal data. When the sample application pool acquires high-repetition-rate samples, it records one sub-data pool within the high-repetition-rate samples as application samples and the other sub-data pool as skipped samples. The sample application pool aggregates the application samples and normal data from all high-repetition-rate samples to obtain the final application samples. When the model uses the final application samples for training, the coverage verification module records the final application samples to obtain historical application samples. Each time a new final application sample is acquired, the new final application sample is subjected to adversarial training against the recorded historical application samples. If the similarity of the adversarial training is within a set range, the data is deemed invalid and training is not performed, while a data duplication reminder is generated. If the similarity exceeds the set range, the data is deemed valid and model training is performed normally. The system also includes an intelligent filtering method for model data samples for knowledge distillation, comprising the following steps: Step 1: Access the full sample database and filter the samples according to preset conditions to remove completely invalid samples. Step 2: Compare the remaining samples from Step 1 with the established interference feature library to obtain the first confidence assessment result of the samples; Step 3: Obtain correlation features from the samples obtained in Step 2, and perform confidence aggregation based on the correlation features to obtain the second confidence assessment result of the samples; Step 4: Select samples from Step 3 according to the established criteria to obtain the sample application pool; Step 5: Perform repeatability verification on the samples in the sample application pool, and skip selection of samples with excessively high repetition rates through repeatability verification; Step 6: Use the samples after skip selection as the final application samples and record the final application samples; Step 7: After the model training is completed, repeat the skip selection of samples from Step 5, while avoiding the final application samples recorded in Step 6.

2. The intelligent screening system for model data samples for knowledge distillation according to claim 1, characterized in that, After accessing the full sample database, the data sample initial screening module obtains preset sample screening conditions, including sample data type, sample feature distribution, and sample domain distribution. The module then performs screening based on these conditions. The sample feature distribution refers to the records within the sample that can describe key technologies, and the sample domain distribution refers to the production domain in which the sample is located.

Images