Data classification method and device, electronic equipment, chip and medium

Abstract

The invention provides a data classification method and device, electronic equipment, a chip and a medium, and relates to the technical field of artificial intelligence. The data classification method comprises the following steps: acquiring original sample data from an original data set; reasoning the original sample data by using at least two large models to obtain corresponding reasoning results; respectively determining the inference result and a loss value of a sample label of the corresponding original sample data; and classifying the quality grade of the original sample data according to the loss value. Through the technical scheme provided by the invention, the problems that the quality of the training samples in the original data set is uneven and the reliability cannot be guaranteed are solved, and the reliability of the training data and the training efficiency of the model are improved.

Description

Technical Field

[0001] The present disclosure relates to the field of artificial intelligence, and in particular to a data classification method, device, electronic device, chip, and medium. Background Art

[0002] In the field of artificial intelligence, the quality of training data has become a key factor in determining model performance. In related technologies, manually labeled original data sets are used to train models. However, the training samples in the original data sets have uneven quality and cannot be guaranteed in terms of reliability. Summary of the Invention

[0003] The present disclosure aims to solve one of the technical problems in the related art at least to a certain extent.

[0004] The present disclosure provides a data classification method, device, electronic device, chip and medium to solve the problems of uneven quality and unreliable training samples in the original data set. By analyzing the quality level of training samples through the inference results of multiple large models on the training samples, high-quality training data with high reliability is obtained, the efficiency of model training is improved, and the performance of the model is enhanced.

[0005] A first embodiment of the present disclosure provides a data classification method, the method comprising:

[0006] Obtain original sample data from the original dataset;

[0007] Use at least two large models to infer the original sample data and obtain corresponding inference results;

[0008] Determine the loss value of the inference result and the sample label of the corresponding original sample data respectively;

[0009] The quality level of the original sample data is classified according to the loss value.

[0010] In one embodiment of the present disclosure, before using at least two large models to perform inference on the original sample data respectively, the method further includes:

[0011] Obtain the scene identifier of the original sample data;

[0012] If the scene identifier is a preset scene identifier, the original sample data is identified as special data, and a copy of the original sample data is created in the original data set according to the first ratio;

[0013] Among them, using at least two large models to infer the original sample data includes:

[0014] If the scene identifier is not a preset scene identifier, at least two large models are used to perform inference on the original sample data respectively.

[0015] In one embodiment of the present disclosure, the quality level of the original sample data is classified according to the loss value, including:

[0016] If the loss values ​​of the original sample data are all less than or equal to the loss threshold, the quality level of the original sample data is marked as medium quality;

[0017] If there is at least one loss value in the loss value of the original sample data that is greater than the loss threshold, the ratio of the number of large models that meet the abnormal condition to the total number of at least two large models is calculated, and the quality level of the original sample data is classified according to the ratio, where the abnormal condition is that the loss value is greater than the loss threshold.

[0018] In one embodiment of the present disclosure, after identifying the quality level of the original sample data as medium quality, the method further includes:

[0019] All samples in the original data set with medium quality are retained as medium quality data sets;

[0020] Deleting the second proportion of samples from the medium quality data set to obtain a first quality data set;

[0021] The first quality dataset is used to replace the medium quality dataset in the original dataset.

[0022] In one embodiment of the present disclosure, the quality level of the original sample data is classified according to the ratio value, including:

[0023] Compare the ratio value with the ratio threshold;

[0024] If the ratio value is less than or equal to the ratio threshold, the quality level of the original sample data is identified as high quality;

[0025] If the ratio value is greater than the ratio threshold, the quality level of the original sample data is identified as low quality, and the original sample data is deleted from the original data set.

[0026] In one embodiment of the present disclosure, after marking the quality level of the original sample data as high quality, the method further includes:

[0027] All samples in the original dataset with a high-quality quality level are retained as high-quality datasets;

[0028] Oversampling the high-quality dataset to generate samples of a third proportion in the high-quality dataset and combining the samples of the high-quality dataset to form a second-quality dataset;

[0029] The high-quality dataset in the original dataset is replaced with the second-quality dataset.

[0030] A second aspect of the present disclosure provides a data classification device, the device comprising:

[0031] The acquisition module is used to obtain the original sample data from the original data set;

[0032] The inference module is used to use at least two large models to infer the original sample data and obtain corresponding inference results;

[0033] A determination module is used to determine the loss value of the inference result and the sample label of the corresponding original sample data respectively;

[0034] The classification module is used to classify the quality level of the original sample data according to the loss value.

[0035] The third aspect embodiment of the present disclosure proposes an electronic device, comprising: at least one processor; and a memory communicatively connected to the at least one processor; wherein the memory stores instructions executable by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to execute any one of the methods in the first aspect embodiment of the present disclosure.

[0036] The fourth aspect embodiment of the present disclosure proposes a non-transitory computer-readable storage medium storing computer instructions, characterized in that the computer instructions are used to enable a computer to execute the method in the first aspect embodiment of the present disclosure.

[0037] The fifth aspect of the present disclosure provides a computer program product, characterized in that it includes a computer program, and when the computer program is executed by a processor, it implements any one of the methods in the first aspect of the present disclosure.

[0038] The sixth aspect embodiment of the present disclosure proposes a chip, comprising at least one processor and a communication interface; the communication interface is used to receive signals input into the chip or signals output from the chip, the processor communicates with the communication interface and implements any one of the methods in the first aspect embodiment of the present disclosure through logic circuits or executing code instructions.

[0039] The data classification method provided by the present disclosure has the following beneficial effects: original sample data is obtained from an original data set, providing an original data set for data classification; at least two large models are used to infer the original sample data respectively, and corresponding inference results are obtained, thereby obtaining the inference results of multiple large models for the original sample data, providing a reference basis for data classification; the loss values ​​of the inference results and the sample labels of the corresponding original sample data are respectively determined, thereby determining a direct reference for evaluating the quality level of the sample data; the quality level of the original sample data is classified according to the loss value, completing the division of the quality level of the original sample data and obtaining reliable model training data. The high-quality training data helps to improve the performance and training efficiency of the model.

[0040] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] The accompanying drawings herein are incorporated into and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the description are used to explain the principles of the present disclosure, and do not constitute an improper limitation of the present disclosure.

[0042] Figure 1 A flowchart of a data classification method according to an embodiment of the present disclosure;

[0043] Figure 2 This is a flow chart of re-determining an original data set according to an embodiment of the present disclosure;

[0044] Figure 3 This is a flow chart of classifying the quality level of original sample data according to loss value according to an embodiment of the present disclosure;

[0045] Figure 4 A flowchart of a quality data set in processing according to an embodiment of the present disclosure;

[0046] Figure 5 This is a flow chart of classifying the quality level of original sample data according to the ratio value according to an embodiment of the present disclosure;

[0047] Figure 6 A flowchart of processing a high-quality data set according to an embodiment of the present disclosure;

[0048] Figure 7 A flowchart of data screening according to an embodiment of the present disclosure;

[0049] Figure 8 This is a structural diagram of a data classification device according to an embodiment of the present disclosure;

[0050] Figure 9 is a block diagram of an electronic device for implementing the data classification method disclosed herein according to an exemplary embodiment;

[0051] Figure 10 It is a schematic structural diagram of a chip according to an embodiment of the present disclosure. DETAILED DESCRIPTION

[0052] The following describes in detail embodiments of the present disclosure, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout identify the same or similar components or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are intended to be used to explain the present disclosure, and should not be construed as limiting the present disclosure.

[0053] First, a brief introduction to the relevant terms in this disclosure is given:

[0054] A large model, also referred to as a base model in this disclosure, is a key concept in the field of artificial intelligence. Its core is its large number of parameters, often billions or even hundreds of billions, and its complex computational structure. The following are several key characteristics of a large model:

[0055] 1. Huge scale: Large models often contain more than one billion parameters, which enables them to process massive amounts of data and complete various complex tasks.

[0056] 2. Deep Neural Network Construction: These models are usually built using a deep neural network, which is a type of neural network containing multiple hidden layers that can learn deep features of the data.

[0057] 3. Wide range of application areas: Large models can be applied to multiple fields such as natural language processing, computer vision, and speech recognition, demonstrating strong versatility.

[0058] 4. Massive data training: The training process of large models requires a large amount of data, which helps the model learn richer and more accurate data representations.

[0059] Loss value: In this disclosure, it refers to the difference between the predicted value of the large model and the true value, which is calculated through the loss function. Commonly used loss functions include Mean Absolute Error (MAE), also known as L1 loss, which calculates the average of the absolute values ​​of the differences between the predicted value and the true value. It also includes Mean Squared Error (MSE), also known as L2 loss, which calculates the average of the squares of the differences between the predicted value and the true value. In addition to L1 loss and L2 loss, it also includes cross entropy loss, multi-label soft edge loss, cosine embedding loss, etc.

[0060] In the field of artificial intelligence, the quality of datasets is crucial to the performance and reliability of machine learning models. High-quality datasets should be accurate, contain correct labels, and include consistent data. Errors in the data can cause the model to learn incorrect patterns, affecting its predictive ability. The dataset should represent the overall distribution of the real-world problem. If the dataset is biased, the model will not be able to generalize correctly to new data. A diverse dataset encompasses multiple problem scenarios, helping to improve the model's generalization ability and enable it to handle a variety of real-world situations. The dataset should also be complete, covering all relevant features, which helps the model learn the true relationships in the data. Furthermore, the dataset should not contain contradictory information, which can cause confusion during model training. While big data is generally considered to help train better models, too much data can also lead to overfitting. Only the right dataset size can strike a balance between a model's generalization ability and computational efficiency.

[0061] In related technologies, after manually annotating training sample datasets, the training sample data is directly used to train the target task model, regardless of whether the task requires high or low precision. However, manually annotated datasets often have uneven quality and cannot guarantee reliability.

[0062] This disclosure aims to solve the problem that the training samples in the original data set are of uneven quality and reliability cannot be guaranteed, provide reliable and high-quality training data for training machine learning models, and improve the performance and training efficiency of the training models.

[0063] The method proposed in this disclosure is applicable to data classification tasks, particularly for adjusting training datasets, increasing the number of samples in high-quality datasets, reducing the number of samples in medium-quality datasets, and deleting samples in low-quality datasets. This is particularly true in scenarios where high-quality labeled data is required, such as in the medical field, where accurate data labeling is crucial as it directly impacts patient health and well-being. Using large models to assess the quality of data such as medical images and clinical records can help improve the accuracy and reliability of diagnostic models. In the field of data analysis, large models can be applied to business intelligence (BI) for data analysis, helping enterprise users process massive amounts of data and conduct highly complex decision analysis. In these scenarios, high-quality data is a prerequisite for accurate analytical results. In the field of artificial intelligence generated content (AIGC), the generation of content, including text, images, and videos, relies on high-quality training data. By evaluating and optimizing datasets using large models, the authenticity and consistency of generated content can be improved. Furthermore, with technological advancements, large models have expanded into fields such as visual recognition, autonomous driving, and decision support. In these fields, models need to make accurate judgments on complex visual information or decision-making problems, making high-quality training data essential. In scientific research, large models are also being used to solve scientific problems such as protein prediction and aerospace. In these cutting-edge fields, accurate data annotation is crucial for the model's discovery and prediction capabilities. The application scenarios and fields of this disclosure are not limited.

[0064] The data classification method provided by the present disclosure is described in detail below with reference to the accompanying drawings.

[0065] Figure 1 This is a flow chart of a data classification method according to an embodiment of the present disclosure. The method is executed on a server, cloud or mobile terminal. Figure 1 In the embodiment shown, the data classification method includes:

[0066] Step 101: Obtain original sample data from the original data set.

[0067] In this embodiment, the original dataset refers to a dataset consisting of manually labeled sample data, which contains a large amount of sample data. The original sample data is any sample data in the original dataset, consisting of data and labels. First, the original sample data is extracted one by one from the original dataset.

[0068] Step 102: Use at least two large models to infer the original sample data respectively to obtain corresponding inference results.

[0069] In this embodiment, at least two large models refer to multiple different large models, and the large models have different network structures. The inference result refers to the prediction result of the large model on the original sample data. Using multiple large models to infer the original sample data respectively, a corresponding multiple inference results can be obtained. For example, in the three large models A, B, and C, for the original sample data x, A(x) = y1 can be expressed as the prediction result y1 obtained after the original sample data x is processed by the large model A, that is, the inference result of model A for the original sample data x is y1. B(x) = y2 can be expressed as the prediction result y2 obtained after the original sample data x is processed by the large model B, that is, the inference result of model B for the original sample data x is y2. C(x) = y3 can be expressed as the prediction result y3 obtained after the original sample data x is processed by the large model C, that is, the inference result of model C for the original sample data x is y3.

[0070] Step 103: Determine the loss value of the inference result and the sample label of the corresponding original sample data respectively.

[0071] In this embodiment, the sample label of the original sample data refers to the sample label set for the original sample data in the manually annotated original data set. The label is artificially generated and is equivalent to the actual value of the original sample data. After obtaining the inference results of the above-mentioned multiple large models, the loss value between each inference result and the actual value is calculated respectively. For example, the sample label of the original sample data x is y, the loss value delta1 of the large model A = |y-y1|, the loss value delta2 of the large model B = |y-y2|, and the loss value delta3 of the large model C = |y-y3|. Delta1, delta2 and delta3 represent the inference errors of the large models A, B and C for the original sample data x, respectively. The larger the value, the greater the difference between the result of the large model inference and the actual result of the manual annotation. Otherwise, it means that the result of the large model inference is closer to the actual result of the manual annotation.

[0072] Step 104: classify the quality level of the original sample data according to the loss value.

[0073] In this embodiment, the quality level of the original sample data refers to the different target categories into which the original sample data is divided by this disclosure, representing different quality levels. These include high quality level, medium quality level, and low quality level. After determining the loss values ​​of different large models for the original sample data, the quality level of the original sample data is classified using the magnitude of these loss values.

[0074] In summary, according to the data classification method proposed in the present disclosure, original sample data is obtained from the original data set, providing an original data set for data classification; at least two large models are used to infer the original sample data respectively, and corresponding inference results are obtained, and the inference results of multiple large models for the original sample data are obtained, providing a reference basis for data classification; the loss values ​​of the sample labels of the inference results and the corresponding original sample data are determined respectively, and a direct reference value for the sample data quality level assessment is determined; the quality level of the original sample data is classified according to the loss value, completing the division of the quality level of the original sample data, and obtaining reliable model training data. High-quality training data helps to improve the performance and training efficiency of the model.

[0075] Figure 2 The present invention provides a flowchart for re-determining an original data set according to an embodiment of the present invention. Figure 2 Yes Figure 1 Further explanation is given before step 102, based on Figure 2 The embodiment shown includes the following steps:

[0076] Step 201: Obtain the scene identifier of the original sample data.

[0077] In this embodiment, the scene identifier of the original sample data refers to additional information of the original sample data in addition to the sample label. This additional marking is usually performed during manual annotation. For example, the original sample data is a road image with pedestrians, lanes, vehicles, and other content in the image. If the JSON or XML script file corresponding to the image records that the image was taken on a rainy day, then the scene identifier of the original sample data is rainy. This scene identifier can be obtained by reading the original dataset or the additional information of the original sample data.

[0078] Step 202: If the scene identifier is a preset scene identifier, the original sample data is identified as special data, and a copy of the original sample data is created in the original data set according to the first ratio, wherein at least two large models are used to infer the original sample data respectively, including: if the scene identifier is not a preset scene identifier, at least two large models are used to infer the original sample data respectively.

[0079] In this embodiment, the preset scene identifier refers to a scene tag preset for the original sample data to achieve the target task. Special data refers to the original sample data for achieving the target task. The original data set will contain the original sample data with the same preset scene identifier, which can be used as a collection of special data. The first ratio is a ratio value, which is used to represent the ratio of the increase in the original sample data. In the original data set, a copy is created for the collection of special data using the first ratio. For example, the target task is to identify lane lines in rainy scenes. In the original data set consisting of 10,000 collected vehicle driving images, 1,000 images have been identified as rainy days when the sample data are manually labeled. Then, the 1,000 vehicle driving images on rainy days are used as a collection of special data. If the first ratio is 50%, 500 copies of the images are created from the 1,000 images, and the vehicle driving images on rainy days are eventually expanded to 1,500 images.

[0080] Furthermore, the triggering condition for at least two large models to independently reason on the original sample data is that the scene identifier of the original sample data is not a preset scene identifier. In other words, the scene identifier of the original sample data does not contain the scene tagging of the original sample data required to achieve the target task. Therefore, at least two large models are required to independently reason on the original sample data in order to implement the technical solutions of the above-mentioned embodiments of the present disclosure.

[0081] In this embodiment, by obtaining the scene identifier, the original data set is redefined based on whether it contains the preset scene identifier. For samples with the preset scene identifier, copies are created to increase the number of samples in the special data set. This provides reliable training data for training the dedicated model corresponding to the special data.

[0082] Figure 3 This is a flow chart of an embodiment of the present disclosure for classifying the quality level of original sample data according to loss value. Figure 3 Yes Figure 2 Further explanation of step 203 is based on Figure 3 The embodiment shown includes the following steps:

[0083] Step 301: If the loss values ​​of the original sample data are all less than or equal to the loss threshold, the quality level of the original sample data is identified as medium quality.

[0084] In this embodiment, the loss threshold is a numerical value used to indicate a very large loss value. After obtaining the loss values ​​for the original sample data inferred by multiple large models, the loss values ​​corresponding to each of the multiple large models are compared with the loss threshold. If the loss values ​​corresponding to each of the multiple large models are all less than or equal to the loss threshold, it indicates that the original sample data was highly accurate during the manual annotation phase, and the quality level of the original sample data is then marked as medium.

[0085] Step 302: If at least one loss value among the loss values ​​of the original sample data is greater than the loss threshold, the ratio of the number of large models that meet the abnormal condition to the total number of at least two large models is calculated, and the quality level of the original sample data is classified according to the ratio, wherein the abnormal condition is that the loss value is greater than the loss threshold.

[0086] In this embodiment, the abnormal condition refers to the condition that the loss value of the large model for the original sample data is greater than the loss threshold. If at least one of the multiple loss values ​​of the large models for the original sample data is greater than the loss threshold, it indicates that the original sample data may contain inaccurately labeled low-quality data or high-quality data used to improve the model's generalization ability. The ratio of the number of large models meeting the abnormal condition to the total number of all large models used is calculated, and the quality level of the sample data is classified based on this ratio.

[0087] In this embodiment, the quality levels of the original sample data are classified according to the loss value, and sample data of different quality levels are obtained with higher reliability.

[0088] Figure 4 The present invention is a flowchart of a process for processing a quality data set according to an embodiment of the present invention. Figure 4 Yes Figure 3 The specific description after step 301 is based on Figure 4 The embodiment shown includes the following steps:

[0089] Step 401: retain all original sample data in the original data set with a quality level of medium quality as a medium quality data set.

[0090] In this embodiment, after the quality level of the original sample data is marked as medium quality, all samples in the original data set whose quality level is medium quality are retained as a medium quality data set.

[0091] Step 402: Delete a second proportion of samples from the medium quality data set to obtain a first quality data set.

[0092] In this embodiment, the first-quality dataset is a dataset obtained by deleting some samples from a medium-quality dataset, i.e., by undersampling the quality dataset. Undersampling refers to a technique for addressing data imbalance by reducing the number of samples in a dataset with multiple classes to balance the class distribution. The second ratio is a ratio value representing the percentage of samples removed from the medium-quality dataset. Undersampling the medium-quality dataset involves reducing the number of samples in the medium-quality dataset. For example, if the medium-quality dataset contains 4000 images and the second ratio is 75%, then undersampling the medium-quality dataset by the second ratio means deleting 75% of the samples from the 4000 images in the medium-quality dataset. Therefore, the number of image samples in the first-quality dataset is 4000 - 4000 * 75% = 1000. In other words, after undersampling, the medium-quality dataset contains only 1000 images. This completes the deletion of samples from the medium-quality dataset. Optionally, the number of deleted samples is determined randomly.

[0093] Step 403: Use the first quality data set to replace the medium quality data set in the original data set.

[0094] In this embodiment, all samples of the first quality data set are used to replace all samples of the medium quality data set in the original data set, thereby updating the original data set and obtaining a balanced original data set.

[0095] In this embodiment, after the original sample data is identified as medium quality, the generated medium-quality dataset is undersampled. This reduces the number of samples that typically account for a large proportion of the dataset, balances the sample distribution within the dataset, and avoids bias during model training. This reduces overfitting during model training, shortens model training time, and improves model training efficiency.

[0096] Figure 5 This is a flow chart of classifying the quality level of original sample data according to the ratio value according to an embodiment of the present disclosure. Figure 5 Yes Figure 3 The specific description of step 302 is based on Figure 5 The embodiment shown includes the following steps:

[0097] Step 501: compare the ratio value with the ratio threshold.

[0098] In this embodiment, the ratio represents the ratio of the number of large models whose inference deviates from the manually annotated actual value to the total number of large models used. For example, if large models A, B, C, D, E, F, G, and H meet the anomaly condition, only large models A and C will have a ratio of 2 / 8 = 25%. The ratio threshold is a percentage value that represents the magnitude of the ratio.

[0099] Step 502: If the ratio value is less than or equal to the ratio threshold, the quality level of the original sample data is identified as high quality.

[0100] In this embodiment, if the ratio value is less than or equal to the ratio threshold, it indicates that the majority of the large models participating in the inference give similar inference results. The manual labeling of the sample data is considered correct, but the sample data is uncommon and can be used as a high-quality sample to improve the model's generalization ability. Therefore, the quality level of the original sample data is considered high. For example, if the ratio threshold is 30%, then for the ratio value in step 501, 25% is less than 30%, and the quality level of the original sample data is identified as high.

[0101] Step 503: If the ratio value is greater than the ratio threshold, the quality level of the original sample data is identified as low quality, and the original sample data is deleted from the original data set.

[0102] In this embodiment, if the ratio value is greater than the ratio threshold, it means that the reasoning results given by most of the large models participating in the reasoning are deviated from the manually labeled labels. It can be considered that the original sample data is very likely to be incorrectly labeled in the manual labeling stage, and the quality level of the original sample data is identified as low quality, and then the original sample data is deleted from the original data set, thereby reducing the interference of the incorrectly labeled unreliable low-quality data on the model training.

[0103] In this embodiment, the quality level of the original sample data is determined as high or low quality through an overall analysis of the inference results of the large model involved in the inference. This further narrows the scope of quality grading and helps to screen out more reliable original sample data.

[0104] Figure 6 The present invention provides a flowchart for processing a high-quality data set according to an embodiment of the present invention. Figure 8 Yes Figure 5 The specific description after step 502 is based on Figure 6 The embodiment shown includes the following steps:

[0105] Step 601: retain all original sample data in the original data set with a quality level of high quality as a high-quality data set.

[0106] In this embodiment, the high-quality data set is a data set consisting of a large number of high-quality samples. After the quality level of the original sample data is marked as high quality, all samples with high quality level in the original data set are retained as the high-quality data set.

[0107] In step 602 , the high-quality data set is oversampled to generate samples of a third proportion in the high-quality data set and the samples in the high-quality data set are combined to form a second-quality data set.

[0108] In this embodiment, the second quality dataset refers to a dataset consisting of all samples in the high-quality dataset and copies of some samples generated from the high-quality dataset. This refers to a dataset obtained by oversampling the high-quality dataset. Oversampling refers to a technique that addresses data imbalance by increasing the number of samples from the minority class in a dataset to balance the class distribution. The third ratio is a ratio value representing the proportion of samples added to the high-quality dataset. Oversampling the high-quality dataset, in other words, increasing the number of samples in the high-quality dataset. For example, if the high-quality dataset contains 1000 images and the third ratio is 80%, then oversampling the high-quality dataset with the third ratio means generating 80% of the samples in the 1000 images in the high-quality dataset from the 1000 images in the high-quality dataset. In other words, the number of image samples in the second quality dataset is 1000 + 1000 * 80% = 1800. In other words, the image samples in the high-quality dataset after oversampling now include 1800 images. This completes the expansion of the samples in the high-quality dataset. Optionally, the created samples are determined randomly.

[0109] Step 603: Use the second quality data set to replace the high quality data set in the original data set.

[0110] In this embodiment, all samples of the second quality data set are used to replace all samples of the high quality data set in the original data set, thereby updating the original data set and balancing the sample distribution of the original data set.

[0111] In this embodiment, a high-quality data set is determined from the original data set, and data oversampling is performed on the high-quality data set. Copies of the high-quality samples are created, the number of samples in the high-quality data set is increased, and more reliable sample data is obtained, which helps the model training to achieve better performance.

[0112] Figure 7 This is a flow chart of data screening according to an embodiment of the present disclosure. Figure 7In the illustrated embodiment, within the original dataset, a determination is first made as to whether the scene identifier of the original sample data in the original dataset matches a preset scene identifier. If so, the original sample data is determined to be target data and added to the special dataset. Oversampling is then performed during the model training phase to increase the proportion of target data in the overall dataset. If the scene identifier of the original sample data does not match the preset scene identifier, inference is performed on the original sample data using multiple different large models to obtain inference results. The error between the inference results and the manually annotated actual values ​​is used as a loss value to determine whether the loss value exceeds a preset loss threshold. If not, the original sample data is determined to be of medium quality and added to the medium-quality dataset. After this determination is performed on all sample data in the original dataset, all medium-quality original sample data in the original dataset are considered a medium-quality dataset. Undersampling is then performed on the medium-quality dataset to reduce the number of samples from the dominant category in the dataset and balance the distribution of samples from different categories. If the loss value exceeds the preset loss threshold, the ratio of the number of models meeting the abnormality condition to the total number of models is calculated. A determination is then made as to whether this ratio exceeds a ratio threshold. If so, the manually annotated original sample data is deemed to have a label problem and the original sample data is marked as useless. All the original sample data marked as useless in the original data set are added to the useless data set to form a useless data set, and all the samples in the useless data set are discarded. If the loss value is less than or equal to the ratio threshold, the original sample data is determined to be a high-quality sample and added to the high-quality data set. All the original sample data marked as high-quality in the original data set are added to the high-quality data set to form a high-quality data set. For all samples in the high-quality data set, oversampling is performed to expand the proportion of high-quality samples in the training data set, thereby improving the reliability of the training data. Since undersampling processes a large number of samples in the medium-quality data set, the convergence time of the model is shortened, the training time of the model is reduced, and the training efficiency of the model is improved.

[0113] The disclosed embodiments provide a data classification method, which obtains original sample data from an original data set, providing an original data set for data classification; uses at least two large models to infer the original sample data respectively, obtains corresponding inference results, and obtains inference results of multiple large models on the original sample data, providing a reference basis for data classification; determines the loss value of the sample label of the inference result and the corresponding original sample data respectively, and determines a direct reference value for evaluating the quality level of the sample data; classifies the quality level of the original sample data according to the loss value, completes the division of the quality level of the original sample data, and obtains reliable model training data. The high-quality training data helps to improve the performance and training efficiency of the model.

[0114] Corresponding to the methods provided in the above-mentioned embodiments, the present disclosure also provides a data classification device. Since the device provided in the embodiment of the present disclosure corresponds to the methods provided in the above-mentioned embodiments, the implementation method is also applicable to the device provided in this embodiment and will not be described in detail in this embodiment.

[0115] Figure 8 FIG. 8 is a structural diagram of a data classification device 800 according to an embodiment of the present disclosure. Figure 8 As shown, the data classification device includes:

[0116] The acquisition module 810 is used to obtain original sample data from the original data set;

[0117] The inference module 820 is used to use at least two large models to perform inference on the original sample data to obtain corresponding inference results;

[0118] A determination module 830 is used to determine the loss value of the inference results and the sample labels of the corresponding original sample data;

[0119] The classification module 840 is used to classify the quality level of the original sample data according to the loss value.

[0120] In some embodiments, before being used to use at least two large models to perform inference on the original sample data, the inference module 820 is further used to:

[0121] Obtain the scene identifier of the original sample data;

[0122] If the scene identifier is a preset scene identifier, the original sample data is identified as special data, and a copy of the original sample data is created in the original data set according to the first ratio;

[0123] Among them, using at least two large models to infer the original sample data includes:

[0124] If the scene identifier is not a preset scene identifier, at least two large models are used to perform inference on the original sample data respectively.

[0125] In some embodiments, the classification module 840 is configured to:

[0126] If the loss values ​​of the original sample data are all less than or equal to the loss threshold, the quality level of the original sample data is marked as medium quality;

[0127] If there is at least one loss value in the loss value of the original sample data that is greater than the loss threshold, the ratio of the number of large models that meet the abnormal condition to the total number of at least two large models is calculated, and the quality level of the original sample data is classified according to the ratio, where the abnormal condition is that the loss value is greater than the loss threshold.

[0128] In some embodiments, after identifying the quality level of the original sample data as medium quality, the classification module 840 is further configured to:

[0129] All samples in the original data set with medium quality are retained as medium quality data sets;

[0130] Deleting the second proportion of samples from the medium quality data set to obtain a first quality data set;

[0131] The first quality dataset is used to replace the medium quality dataset in the original dataset.

[0132] In some embodiments, the classification module 840 classifies the quality level of the original sample data according to the ratio value in the following manner:

[0133] Compare the ratio value with the ratio threshold;

[0134] If the ratio value is less than or equal to the ratio threshold, the quality level of the original sample data is identified as high quality;

[0135] If the ratio value is greater than the ratio threshold, the quality level of the original sample data is identified as low quality, and the original sample data is deleted from the original data set.

[0136] In some embodiments, after identifying the quality level of the original sample data as high quality, the classification module 840 is further configured to:

[0137] All samples in the original dataset with a high-quality quality level are retained as high-quality datasets;

[0138] Oversampling the high-quality dataset to generate samples of a third proportion in the high-quality dataset and combining the samples of the high-quality dataset to form a second-quality dataset;

[0139] The high-quality dataset in the original dataset is replaced with the second-quality dataset.

[0140] In summary, the data classification device obtains original sample data from the original dataset; uses at least two large models to perform inference on the original sample data, obtaining corresponding inference results; determines the loss value between the inference results and the sample labels of the corresponding original sample data; and classifies the quality level of the original sample data based on the loss value. This device solves the problem of uneven quality and unreliable training samples in the original dataset, obtains high-quality and reliable training data, improves the efficiency of model training, and enhances model performance.

[0141] The embodiments provided above in this disclosure describe the methods and devices provided in these embodiments. To implement the various functions in the methods provided in these embodiments, electronic devices may include hardware structures and software modules, and implement these functions in the form of hardware structures, software modules, or a combination of hardware structures and software modules. Certain of these functions may be implemented in the form of hardware structures, software modules, or a combination of hardware structures and software modules.

[0142] Figure 9 is a block diagram of an electronic device 900 for implementing the above data classification method according to an exemplary embodiment.

[0143] For example, the electronic device 900 may be a mobile phone, a computer, a messaging device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0144] Reference Figure 9 , the electronic device 900 may include one or more of the following components: a processing component 902 , a memory 904 , a power component 906 , a multimedia component 908 , an audio component 910 , an input / output (I / O) interface 912 , a sensor component 914 , and a communication component 916 .

[0145] The processing component 902 generally controls the overall operation of the electronic device 900, such as operations associated with display, phone calls, data communications, camera operation, and recording operations. The processing component 902 may include one or more processors 920 to execute instructions to perform all or part of the steps of the above-described method. In addition, the processing component 902 may include one or more modules to facilitate interaction between the processing component 902 and other components. For example, the processing component 902 may include a multimedia module to facilitate interaction between the multimedia component 908 and the processing component 902.

[0146] The memory 904 is configured to store various types of data to support operations on the electronic device 900. Examples of such data include instructions for any application or method operating on the electronic device 900, contact data, phone book data, messages, pictures, videos, etc. The memory 904 can be implemented by any type of volatile or non-volatile storage device, or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk, or optical disk.

[0147] The power supply component 906 provides power to the various components of the electronic device 900. The power supply component 906 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 900.

[0148] The multimedia component 908 includes a screen that provides an output interface between the electronic device 900 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touches, slides, and gestures on the touch panel. The touch sensor can not only sense the boundaries of a touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 908 includes a front camera and / or a rear camera. When the electronic device 900 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera can receive external multimedia data. Each front camera and rear camera can be a fixed optical lens system or have focal length and optical zoom capabilities.

[0149] The audio component 910 is configured to output and / or input audio signals. For example, the audio component 910 includes a microphone (MIC), and when the electronic device 900 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 904 or transmitted via the communication component 916. In some embodiments, the audio component 910 also includes a speaker for outputting audio signals.

[0150] I / O interface 912 provides an interface between processing component 902 and peripheral interface modules, such as a keyboard, click wheel, buttons, etc. These buttons may include but are not limited to: a home button, volume buttons, a start button, and a lock button.

[0151] The sensor assembly 914 includes one or more sensors for providing various aspects of status assessment for the electronic device 900. For example, the sensor assembly 914 can detect the open / closed state of the electronic device 900, the relative positioning of components, such as the display and keypad of the electronic device 900. The sensor assembly 914 can also detect changes in the position of the electronic device 900 or a component of the electronic device 900, the presence or absence of user contact with the electronic device 900, the orientation or acceleration / deceleration of the electronic device 900, and temperature changes of the electronic device 900. The sensor assembly 914 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 914 may also include a light sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 914 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0152] The communication component 916 is configured to facilitate wired or wireless communication between the electronic device 900 and other devices. The electronic device 900 can access a wireless network based on a communication standard, such as WiFi, 2G or 3G, 4G LTE, 5G NR (NewRadio) or a combination thereof. In an exemplary embodiment, the communication component 916 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 916 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0153] In an exemplary embodiment, the electronic device 900 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to perform the above-described methods.

[0154] In an exemplary embodiment, a non-transitory computer-readable storage medium including instructions is also provided, such as a memory 904 including instructions, and the instructions can be executed by the processor 920 of the electronic device 900 to perform the above method. For example, the non-transitory computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0155] An embodiment of the present disclosure further provides a non-transitory computer-readable storage medium storing computer instructions, wherein the computer instructions are used to enable a computer to execute the data classification method described in the above embodiment of the present disclosure.

[0156] An embodiment of the present disclosure further provides a computer program product, including a computer program, which executes the data classification method described in the above embodiment of the present disclosure when a processor is used.

[0157] Figure 10 FIG. 1 is a schematic structural diagram of a chip 1000 for implementing the above-mentioned data classification method according to an exemplary embodiment.

[0158] Reference Figure 10 The chip 1000 includes at least one communication interface 1001 and a processor 1002; the communication interface 1001 is used to receive signals input to the chip 1000 or signals output from the chip 1000, and the processor 1002 communicates with the communication interface 1001 and implements the data classification method described in the above embodiment through logic circuits or execution code instructions.

[0159] It should be noted that the terms "first," "second," and the like in the specification and claims of the present disclosure and the above-mentioned drawings are used to distinguish similar objects and are not necessarily used to describe a specific order or precedence. It should be understood that the numbers used in this manner are interchangeable where appropriate so that the embodiments of the present disclosure described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present disclosure. Instead, they are merely examples of apparatus and methods consistent with certain aspects of the present disclosure as detailed in the appended claims.

[0160] Throughout this specification, reference to terms such as "one embodiment," "some embodiments," "illustrative embodiments," "examples," "specific examples," or "some examples" means that a specific feature, structure, material, or characteristic described in conjunction with an embodiment or example is included in at least one embodiment or example of the present disclosure. In this specification, the illustrative use of the above terms does not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0161] Any process or method description in a flowchart or otherwise described herein may be understood to represent a module, segment or portion of code that includes one or more executable instructions for implementing the steps of a specific logical function or process, and the scope of the preferred embodiments of the present disclosure includes additional implementations in which functions may be performed out of the order shown or discussed, including performing functions in a substantially simultaneous manner or in the reverse order depending on the functions involved, which should be understood by those skilled in the art to which the embodiments of the present disclosure belong.

[0162] The logic and / or steps represented in the flowcharts or otherwise described herein, for example, can be considered as a sequenced list of executable instructions for implementing the logical functions, and can be embodied in any computer-readable medium for use by, or in conjunction with, an instruction execution system, apparatus, or device (e.g., a computer-based system, a system including a processing module, or other system that can fetch and execute instructions from an instruction execution system, apparatus, or device). For purposes of this specification, a "computer-readable medium" can be any device that can contain, store, communicate, propagate, or transport a program for use by, or in conjunction with, an instruction execution system, apparatus, or device. More specific examples (non-exhaustive list) of computer-readable media include the following: an electrical connection having one or more wires (control method), a portable computer disk cartridge (magnetic device), random access memory (RAM), read-only memory (ROM), erasable and programmable read-only memory (EPROM or flash memory), fiber optic devices, and a portable compact disc read-only memory (CDROM). Furthermore, the computer-readable medium may even be paper or other suitable medium on which the program is printed, since the program may be obtained electronically, for example, by optically scanning the paper or other medium and then editing, interpreting or otherwise processing it in a suitable manner if necessary, and then storing it in a computer memory.

[0163] It should be understood that the various parts of the embodiments of the present disclosure can be implemented using hardware, software, firmware, or a combination thereof. In the above embodiments, multiple steps or methods can be implemented using software or firmware stored in a memory and executed by a suitable instruction execution system. For example, if implemented using hardware, as in another embodiment, any one of the following technologies known in the art or a combination thereof can be used to implement: a discrete logic circuit having a logic gate circuit for implementing a logic function on a data signal, an application-specific integrated circuit having a suitable combination of logic gate circuits, a programmable gate array (PGA), a field programmable gate array (FPGA), etc.

[0164] Those skilled in the art will understand that all or part of the steps in the method of the above embodiment can be completed by instructing related hardware through a program, and the program can be stored in a computer-readable storage medium. When the program is executed, it includes one or a combination of the steps of the method embodiment.

[0165] In addition, the functional units in the various embodiments of the present disclosure may be integrated into a processing module, or each unit may exist physically separately, or two or more units may be integrated into a module. The above-mentioned integrated module may be implemented in the form of hardware or in the form of a software functional module. If the integrated module is implemented in the form of a software functional module and sold or used as an independent product, it may also be stored in a computer-readable storage medium. The above-mentioned storage medium may be a read-only memory, a magnetic disk, an optical disk, etc.

[0166] Although the embodiments of the present disclosure have been shown and described above, it is understood that the above embodiments are exemplary and are not to be construed as limitations on the present disclosure. A person skilled in the art may change, modify, replace and vary the above embodiments within the scope of the present disclosure.

Claims

1. A data classification method, characterized in that: The method comprises: Obtain original sample data from the original dataset; Use at least two large models to respectively infer the original sample data to obtain corresponding inference results; Determine the loss value of the inference result and the sample label of the corresponding original sample data respectively; The quality level of the original sample data is classified according to the loss value.

2. The method according to claim 1, characterized in that Before using at least two large models to respectively infer the original sample data, the method further includes: Obtaining a scene identifier of the original sample data; If the scene identifier is a preset scene identifier, the original sample data is identified as special data, and a copy of the original sample data is created in the original data set according to the first ratio; The using at least two large models to respectively infer the original sample data includes: If the scene identifier is not the preset scene identifier, at least two large models are used to perform inference on the original sample data respectively.

3. The method according to claim 1, characterized in that The classifying the quality level of the original sample data according to the loss value includes: If the loss values ​​of the original sample data are all less than or equal to the loss threshold, the quality level of the original sample data is marked as medium quality; If there is at least one loss value among the loss values ​​of the original sample data that is greater than the loss threshold, the ratio of the number of large models that meet the abnormal condition to the total number of the at least two large models is calculated, and the quality level of the original sample data is classified according to the ratio, wherein the abnormal condition is that the loss value is greater than the loss threshold.

4. The method according to claim 3, characterized in that After identifying the quality level of the original sample data as medium quality, the method further includes: Retain all samples in the original data set whose quality level is medium as the medium quality data set; Deleting a second proportion of samples from the medium quality data set to obtain a first quality data set; The medium-quality dataset in the original dataset is replaced by the first-quality dataset.

5. The method according to claim 3, characterized in that The classifying the quality level of the original sample data according to the ratio value includes: Comparing the ratio value with a ratio threshold; If the ratio value is less than or equal to the ratio threshold, the quality level of the original sample data is identified as high quality; If the ratio value is greater than the ratio threshold, the quality level of the original sample data is identified as low quality, and the original sample data is deleted from the original data set.

6. The method according to claim 5, characterized in that After identifying the quality level of the original sample data as high quality, the method further includes: Retain all samples in the original data set with a quality level of high quality as a high-quality data set; Performing oversampling processing on the high-quality data set to generate samples of a third proportion in the high-quality data set and combining the samples of the high-quality data set to form a second-quality data set; The second quality dataset is used to replace the high quality dataset in the original dataset.

7. A data classification device, characterized in that: The device comprises: The acquisition module is used to obtain the original sample data from the original data set; The inference module is used to use at least two large models to infer the original sample data and obtain corresponding inference results; A determination module, configured to determine the loss values ​​of the inference results and the corresponding sample labels of the original sample data; A classification module is used to classify the quality level of the original sample data according to the loss value.

8. An electronic device, characterized in that: include: at least one processor; as well as a memory communicatively connected to the at least one processor; wherein, The memory stores instructions that can be executed by the at least one processor, and the instructions are executed by the at least one processor to enable the at least one processor to perform the method according to any one of claims 1 to 6.

9. A non-transitory computer-readable storage medium storing computer instructions, characterized in that: The computer instructions are used to cause the computer to execute the method according to any one of claims 1 to 6.

10. A computer program product, characterized in that The invention comprises a computer program which, when executed by a processor, implements the method according to any one of claims 1 to 6.

11. A chip, characterized in that: The method comprises at least one processor and a communication interface; the communication interface is used to receive a signal input to the chip or a signal output from the chip, and the processor communicates with the communication interface and implements the method according to any one of claims 1 to 6 through a logic circuit or executing code instructions.

Images