A data classification method and system

Abstract

This application relates to the field of data classification, and more specifically to a data classification method and system. The method includes: classifying data to be classified using a current classification model pre-trained based on training samples to obtain category labels and confidence scores; when the current classification model needs updating, updating the current classification model and classifying new data to be classified based on the updated model; wherein updating the current classification model includes: classifying the data to be classified based on the category labels and confidence scores to obtain classification results; expanding the training samples using the classification results to obtain new training samples; and retraining the current classification model based on the new training samples to obtain an updated classification model. This application enables low-cost, automated, and continuous optimization of the classification model.

Description

Technical Field

[0001] This invention relates to the field of data classification, and more specifically, to a data classification method and system. Background Technology

[0002] In various data classification tasks, especially in large-scale, multimodal (such as text, image, audio, and video) and dynamically evolving business scenarios, the training and maintenance of classification models face multiple challenges. Current mainstream model training methods typically rely on pre-collected, large-scale labeled datasets, obtaining a static model through a one-time training process before deployment. However, in real-world business environments, while unlabeled data is readily available and abundant, manual annotation is costly, and the scale of high-quality labeled data is limited. This makes it difficult to update the model in a timely manner when data distribution or the content to be recognized changes, leading to performance degradation. Furthermore, traditional static training processes lack the dynamic utilization of the latest online data, failing to establish an efficient continuous optimization mechanism.

[0003] To address the aforementioned issues, existing technological solutions include: adding training data or optimizing the structure of a single model; combining multiple independent models to complete classification tasks; and using manual annotation combined with periodic retraining. While these solutions can improve model performance to a certain extent, they generally suffer from shortcomings such as a lack of data feedback mechanisms, excessive manual intervention, high costs, and poor long-term stability. They are therefore difficult to implement in scenarios where data is constantly updated and labeled data is scarce, making it challenging to achieve low-cost, automated, and continuous optimization. Summary of the Invention

[0004] This invention provides a data classification method and system for achieving low-cost, automated, and continuous optimization of models.

[0005] According to a first aspect of this application, a data classification method is provided, the method comprising: The current classification model, pre-trained based on training samples, is used to classify the data to be classified, and the category labels and confidence scores are obtained. When the current classification model needs to be updated, the current classification model is updated, and the new data to be classified is classified based on the updated classification model; Updating the current classification model includes: The data to be classified is graded based on the category labels and confidence scores to obtain the grading results. The training samples are expanded using the classification results to obtain new training samples; The current classification model is retrained based on the new training samples to obtain an updated classification model.

[0006] Understandably, to address the technical challenge of achieving low-cost, automated, and continuous optimization of classification models in scenarios where data is constantly updated and labeled data is scarce, traditional data classification methods struggle to achieve this. This application triggers model updates during the data classification process and directly uses the graded data to expand training samples and retrain the model, creating a closed-loop linkage between data classification and model optimization. This eliminates the need for extensive manual labeling, allowing for iterative model updates. It maximizes the use of real-world business data, significantly reducing data labeling and model maintenance costs. Furthermore, it enables the classification model to dynamically adjust and optimize with data distribution, effectively preventing model accuracy decay and improving the long-term accuracy and stability of data classification results. Simultaneously, it adapts to real-world business scenarios with continuously updated data, enhancing the model's scenario adaptability and achieving low-cost, automated, and continuous optimization of the classification model.

[0007] Optionally, updating the current classification model further includes: When the updated classification model fails to meet the preset convergence condition, the data to be classified is reclassified using the updated classification model to obtain new category labels and confidence scores. Based on the new category labels and confidence scores, the process of grading the data to be classified is returned. When the updated classification model satisfies the convergence condition, the updated classification model is obtained.

[0008] Understandably, by setting convergence conditions and iteratively executing the process from classification to grading and then to retraining, the model is ensured to continuously use the latest data to be classified for self-correction before reaching stable performance. This effectively alleviates the problem of static model failure caused by one-time training and improves the model's long-term adaptability in dynamic business environments.

[0009] Optionally, the classification result includes a first confidence sample and a second confidence sample carrying category classification labels, wherein the confidence of the first confidence sample is greater than the confidence of the second confidence sample; the training samples are divided into a training set and a test set; The step of expanding the training samples using the grading results to obtain new training samples includes: Add the first confidence sample to the training set; The second confidence sample is subjected to category classification label verification, and the verified correct second confidence sample is added to the training set.

[0010] Understandably, by differentiating samples with different confidence levels, only the first confidence samples with relatively high confidence are directly added to the training set, while the second confidence samples with relatively low confidence are added to the training set after verification. This effectively filters out low-quality labeled samples, improves the overall quality of training samples, avoids noise samples affecting the model training effect, reduces the workload of manual labeling, lowers labeling costs, and allows the expanded training samples to truly improve the model classification performance.

[0011] Optionally, the step of validating the category classification label on the second confidence sample includes: Perform category prediction on the second confidence sample to obtain the predicted category label; The consistency between the category classification label and the category prediction label of the second confidence sample is verified.

[0012] Understandably, to address the issue of insufficient reliability of second-confidence sample labels, high-quality samples are further screened through consistency verification of dual labels. This improves the quality of training sample annotation from the source, solves the problem of imbalance between data and label quality, avoids second-confidence samples with incorrect labels from entering the training set and causing model classification bias, ensures the classification accuracy of the model after retraining, and replaces manual full verification, thereby improving sample screening efficiency and reducing labor costs.

[0013] Optionally, the classification result further includes a third confidence sample carrying a category classification label, wherein the confidence of the second confidence sample is greater than the confidence of the third confidence sample; The step of expanding the training samples using the grading results to obtain new training samples further includes: The third confidence samples belonging to the rare category are subjected to category classification label verification, and the verified third confidence samples are added to the test set; wherein the rare category is determined according to statistical results, which are obtained by statistically analyzing the category classification labels of the data to be classified.

[0014] Understandably, in order to address the class imbalance problem caused by the imbalance between data and label quality, targeted validation of the third confidence samples of rare classes and supplementation of the test set are used to compensate for the insufficient sample size of rare classes. This enriches the coverage of rare classes in complex scenarios, allowing the model to fully learn the features of rare classes during the training and testing phases, improve the model's classification ability of rare classes, avoid the problem of missed or false classification of rare classes due to class imbalance, and improve the model's generalization ability.

[0015] Optionally, the step of classifying the data to be classified based on the category label and confidence level to obtain the classification result includes: The category labels of the data to be classified are statistically analyzed to obtain statistical results. Based on the statistical results, common and rare categories are determined; Based on the first and second confidence thresholds corresponding to the common category, the data to be classified belonging to the common category is graded to obtain first confidence samples, second confidence samples, and third confidence samples carrying category classification labels; Based on the first and second confidence thresholds corresponding to the rare categories, the data to be classified belonging to the rare categories are graded to obtain first confidence samples, second confidence samples, and third confidence samples carrying category classification labels. Specifically, the first confidence threshold of the common category is greater than its second confidence threshold, and the first confidence threshold of the rare category is greater than its second confidence threshold; furthermore, the first confidence threshold of the common category is greater than the first confidence threshold of the rare category, and the second confidence threshold of the common category is greater than the second confidence threshold of the rare category.

[0016] Understandably, to address the problem of low classification accuracy and difficulty in effective model learning due to the small sample size of rare categories, different confidence thresholds are set for common and rare categories. This lowers the classification threshold for rare category samples, allowing more rare category samples to participate in model training and testing, thus solving the core pain point of class imbalance and improving the model's ability to identify and classify rare categories.

[0017] Optionally, when classifying the data to be classified using the current classification model, the predicted probability of the category of the data to be classified is also obtained; Obtain the highest and second-highest predicted category probabilities corresponding to the data to be classified; The confidence level is obtained based on the highest category prediction probability and the second highest category prediction probability.

[0018] Understandably, obtaining the confidence level by using the highest and second-highest predicted probabilities is a better reflection of the model's discrimination boundary clarity than a single probability value. This helps to accurately separate highly certain samples from easily confused samples, providing a more reliable basis for subsequent classification and verification, and improving the accuracy of data screening.

[0019] Optionally, the new training samples are divided into a new training set and a new test set; When the updated classification model satisfies the convergence condition, the misclassified samples of the updated classification model on the new test set are also obtained. The misclassified samples were stratified and sampled according to their categories to obtain the sampled samples; The sampled data is then subjected to category correction; The corrected sampled samples are added to the new training samples, and the updated classification model that meets the convergence condition is fine-tuned using the new training samples with the added sampled samples to obtain the final model.

[0020] Understandably, to address the issues of misclassification of key categories and suboptimal classification accuracy even after model convergence, stratified sampling, label correction, and model fine-tuning of misclassified samples in the new test set are employed to precisely resolve key misclassification problems. This thoroughly filters out dirty data in the training samples and corrects labels on difficult samples, further improving the model's classification accuracy and recall. Simultaneously, it enables the model to adapt to complex classification scenarios in real-world data, enhancing its multi-scenario transferability and long-term stable performance.

[0021] Optionally, the current classification model is obtained in advance through the following method: The training samples include a training set constructed based on synthetic training data and a test set constructed based on real labeled data; A preset classification model is trained using the training set, and the trained preset classification model is tested using the test set to obtain the trained current classification model.

[0022] Understandably, in order to address the issue of high data annotation costs, the approach of constructing the main training set with synthetic training data and the test set with a small amount of real labeled data significantly reduces the reliance on high-quality manually labeled data, thereby lowering the annotation costs in the early stages of model training. At the same time, the use of synthetic training data ensures the diversity of training sample categories, avoiding the problem of poor model generalization ability due to insufficient real labeled data. Meanwhile, it can quickly obtain a usable initial version of the model in scenarios with extremely limited annotation resources, significantly shortening the model deployment cycle and reducing the R&D costs in the cold start phase.

[0023] Optionally, the step of training a preset classification model using the training set and testing the trained preset classification model using the test set to obtain the trained current classification model includes: At least one candidate predefined classification model is trained using the training set, and the trained candidate predefined classification model is tested using the test set to obtain test results; Based on the test results, the candidate preset classification model with the best performance is obtained as the trained current classification model.

[0024] Understandably, to address the issues of poor adaptability and low transferability across multiple scenarios of a single pre-defined classification model, performance evaluation of multiple candidate pre-defined classification models and selection of the optimal model are employed. This allows the selected classification model to better adapt to the characteristics of the target data classification scenario, improves the model's scenario adaptability and initial classification accuracy, avoids the problems of low subsequent optimization efficiency and poor classification results caused by improper model selection, reduces the cost of model retraining, and improves the overall implementation efficiency of the data classification method.

[0025] Optionally, the synthetic training data is synthesized based on a synthesis strategy; The method further includes: The synthesis strategy is adjusted using the test results, and new synthetic training data is resynthesized using the adjusted synthesis strategy. The training samples are updated based on the new synthetic training data; The updated training samples are used to optimize the current classification model.

[0026] Understandably, the data synthesis strategy is adjusted in reverse based on the model's performance on the test set, so that the synthesized data is more closely aligned with the model's weaknesses. This forms a small closed loop of synthesis, training, evaluation, and optimization, improving the effectiveness and relevance of the synthesized data. This, in turn, enhances the model's initial classification ability, reduces the number of iterations required for subsequent model retraining, improves model optimization efficiency, and ensures class balance in the synthesized data, preventing the model from experiencing classification shifts due to biases in the synthesized data.

[0027] According to a second aspect of this application, a data classification system is provided, the system comprising: The classification module is used to classify the data to be classified using the current classification model pre-trained based on training samples, and to obtain the category label and confidence score. The update module is used to update the current classification model when the current classification model needs to be updated. The classification module is also used to classify new data to be classified based on the updated classification model; Updating the current classification model includes: The data to be classified is graded based on the category labels and confidence scores to obtain the grading results. The training samples are expanded using the classification results to obtain new training samples; The current classification model is retrained based on the new training samples to obtain an updated classification model.

[0028] According to a third aspect of this application, an electronic device is provided, comprising: Memory, used to store one or more computer programs; A processor, when the one or more computer programs are executed by the processor, implements the data classification method described in the first aspect above.

[0029] According to a fourth aspect of this application, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the data classification method described in the first aspect above.

[0030] Based on any of the above aspects, the data classification method, system, electronic device, and storage medium provided in the embodiments of this application have the following beneficial effects: (1) By introducing a model update triggering mechanism in the classification process, this application can automatically start the model optimization process when potential performance degradation is detected, avoid the accuracy decay caused by data distribution drift after long-term model operation, initially realize the dynamic connection between classification and training, and realize low-cost, automated and continuous optimization of the classification model.

[0031] (2) By setting convergence conditions and repeatedly executing the process of classification, grading and retraining, this application ensures that the model continuously uses the latest data to be classified for self-correction before reaching stable performance, effectively alleviating the problem of static model failure caused by one-time training and improving the long-term adaptability of the model in dynamic business environments.

[0032] (3) This application uses the first confidence sample with relatively high confidence directly for training, and verifies the label of the second confidence sample with relatively low confidence before including it in the training set. This avoids the introduction of noise by blindly adopting low-quality pseudo-labels and prevents valuable boundary samples from being discarded, thus significantly improving the utilization efficiency of unlabeled data and reducing the dependence on manual labeling.

[0033] (4) This application further screens high-quality samples through the consistency verification of dual labels, and provides an automated quality filtering mechanism for second confidence samples without human intervention. While controlling noise injection, it retains semantically valid training samples, enhances the stability of model retraining, ensures the classification accuracy of the model after retraining, and replaces manual full verification, thereby improving sample screening efficiency and reducing labor costs.

[0034] (5) This application adds verified and correct rare category third confidence samples to the test set to enrich the coverage of rare categories in complex scenarios, so that the test set can reflect the long tail difficulties in real business, avoid the problem of missing or misjudging rare categories due to class imbalance, and improve the generalization ability of the model.

[0035] (6) This application uses different confidence thresholds for common and rare categories to classify samples, thereby reducing the adoption threshold of rare categories, preventing them from being systematically ignored in the data flywheel, effectively alleviating the performance bias caused by class imbalance, and improving the model's ability to identify and classify long-tail categories.

[0036] (7) This application obtains confidence by using the highest and second-highest predicted probabilities, which can better reflect the clarity of the model's discrimination boundary than a single probability value. This helps to accurately separate highly certain samples from easily confused samples, providing a more reliable basis for subsequent classification and verification, and improving the accuracy of data screening.

[0037] (8) After the model converges, this application performs stratified sampling of misjudged samples on the test set by category and performs label correction, focusing resources on the categories that actually have problems, fine-tuning the model through high-quality correction samples, further improving the model's classification accuracy and recall, while enabling the model to adapt to complex classification scenarios in real data, and improving the model's multi-scenario transfer capability and long-term stable performance.

[0038] (9) This application uses synthetic training data to construct a large-scale training set and a small amount of real labeled data to construct a test set. This can quickly obtain a usable initial version of the model in scenarios where labeling resources are extremely limited, significantly shortening the model launch cycle and reducing the R&D cost in the cold start phase.

[0039] (10) In the synthetic training phase, this application trains multiple candidate pre-defined classification models in parallel and evaluates them based on a unified test set to ensure that the most suitable base model for the current task is selected, thereby avoiding performance bottlenecks caused by improper model architecture selection and improving the starting quality and subsequent iteration efficiency of the entire optimization process.

[0040] (11) This application adjusts the data synthesis strategy in reverse according to the model’s performance on the test set, so that the synthesized data is more in line with the model’s weaknesses, forming a small closed loop of synthesis, training, evaluation and optimization of synthesis, improving the effectiveness and relevance of the synthesized data, thereby improving the model’s initial classification ability, reducing the number of iterations of subsequent model retraining, improving the model optimization efficiency, and ensuring the class balance of the synthesized data, avoiding the model from having a classification shift due to the bias of the synthesized data. Attached Figure Description

[0041] To more clearly illustrate the technical solutions in the embodiments of this application, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0042] Figure 1 This is an illustrative application scenario diagram of the data classification method provided in this embodiment.

[0043] Figure 2 A flowchart of the data classification method provided in this embodiment.

[0044] Figure 3 This is a schematic diagram illustrating the training steps of the current classification model provided in this embodiment.

[0045] Figure 4 This is a schematic diagram illustrating the update steps of the current classification model provided in this embodiment.

[0046] Figure 5 This is a schematic diagram of the sub-steps of step S220 provided in this embodiment.

[0047] Figure 6 This is a schematic diagram of the functional modules of the data classification system provided in this embodiment.

[0048] Figure 7 This is a schematic diagram of the structure of the electronic device provided in this embodiment. Detailed Implementation

[0049] The accompanying drawings are for illustrative purposes only and should not be construed as limiting the scope of this application. To better illustrate the following embodiments, some components in the drawings may be omitted, enlarged, or reduced, and do not represent the actual dimensions of the product; it is understandable to those skilled in the art that some well-known structures and their descriptions may be omitted in the drawings.

[0050] To enable those skilled in the art to better understand the present application, the technical solutions in the embodiments of the present application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present application, and not all embodiments. Based on the embodiments in the present application, all other embodiments obtained by those of ordinary skill in the art without creative effort should fall within the scope of protection of the present application.

[0051] It should be noted that the terms "first," "second," etc., in the specification, claims, and accompanying drawings of this application are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of this application described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0052] In data classification scenarios with limited labeled data, the inventors have discovered that existing technologies offer several ways to improve classification performance. For example, semi-supervised learning combined with pseudo-label strategies can be used to automatically label large amounts of unlabeled data and then directly add them to the training set for iterative optimization; cross-task transfer learning can be used to fine-tune models trained on similar tasks to the current task domain; or continuous data augmentation can be used for iterative training, transforming and expanding limited labeled samples to improve the model's generalization ability. While these techniques can improve model performance to some extent under conditions of insufficient data, they generally suffer from drawbacks such as a lack of closed-loop optimization processes and poor long-term model performance stability. Therefore, in application environments where labeled data is scarce and data distribution is dynamically changing, it is difficult to achieve low-cost, sustainable model optimization.

[0053] This embodiment provides a technical solution that can solve the above problems. The specific implementation of this application will be described in detail below with reference to the accompanying drawings.

[0054] An exemplary diagram illustrating an application scenario of a data classification method provided in this application embodiment is shown below. Figure 1 As shown, the application scenario includes at least a server 100 and a terminal 200 that can communicate with the server 100.

[0055] Understandably, the server 100 can be an independent electronic device or a cluster of multiple electronic devices; the terminal 200 can be a smartphone terminal, personal computer, tablet computer, vehicle terminal, etc., but is not limited to these.

[0056] In one possible implementation, server 100 and terminal 200 may each execute the data classification method provided in the embodiments of this application. Alternatively, the data classification method provided in the embodiments of this application may be executed partly in server 100 and partly in terminal 200.

[0057] like Figure 2 As shown, this embodiment provides a data classification method, which can be further divided into the following steps: S100. Use the current classification model, which has been pre-trained based on the training samples, to classify the data to be classified, and obtain the category label and confidence level.

[0058] Understandably, the classification model is a type of predictive model in machine learning and artificial intelligence. Its core function is to automatically determine which category the data belongs to based on the characteristics of the input data. Correspondingly, the data to be classified can be raw input data collected in actual business scenarios that has not yet been manually labeled but needs to be classified by the classification model. This includes, but is not limited to, images, text, audio / video, or multimodal data.

[0059] Specifically, the classification model can be a computer vision classification model, such as an image classification model or a face recognition model, and the data to be classified can be image data; the classification model can also be a natural language processing classification model, such as a text classification model, and the data to be classified can be text data.

[0060] Specifically, the category label is the category result output by the classification model after predicting the data to be classified, that is, which category the classification model believes the data to be classified belongs to.

[0061] Specifically, the confidence level is a quantitative indicator of the credibility of the classification model's output category labels, reflecting the certainty of the model's judgment.

[0062] In the specific implementation process, the method of this embodiment further includes the following steps: A110. When classifying the data to be classified using the current classification model, the predicted probability of the category of the data to be classified is also obtained.

[0063] In the specific implementation process, when the current classification model classifies the data to be classified, it will generate a category prediction probability for each category. Usually, the category corresponding to the highest category prediction probability is output as the category classification label.

[0064] A120. Obtain the highest category prediction probability and the second highest category prediction probability corresponding to the data to be classified; Specifically, after obtaining the category prediction probability for each category of the data to be classified, the highest category prediction probability and the second highest category prediction probability can be obtained from it.

[0065] A130. Obtain the confidence level based on the highest category prediction probability and the second highest category prediction probability.

[0066] In practice, the difference between the predicted probability of the highest-ranking class and the predicted probability of the second-highest-ranking class can be calculated as the confidence level. Using this difference to measure the confidence level reflects the degree to which the current classification model distinguishes the predicted class from the second-highest-ranking class; the larger the difference, the more certain the prediction.

[0067] In specific implementation, the current classification model is pre-trained based on training samples, such as... Figure 3 As shown, the specific training process can include the following detailed steps: B110. The training samples include a training set constructed based on synthetic training data and a test set constructed based on real labeled data; Understandably, the training samples comprise two functionally distinct subsets: a training set and a test set, typically with a smaller sample size in the test set than in the training set. The training set is constructed based on synthetic training data, while the test set is constructed based on real-world labeled data.

[0068] Specifically, the synthetic training data is sample data with preset category labels that can be simulated by a pre-set synthesis strategy, and it is not directly collected from real business scenarios.

[0069] In practical applications, during the cold start phase of a model, real labeled data is often scarce and costly to obtain. By synthesizing a large amount of training data that is class-balanced and covers typical scenarios, class balance can be ensured, avoiding excessive repetition of certain types of samples that could lead to model bias. This allows for the rapid training of an initial classification model with basic discriminative capabilities.

[0070] Specifically, the synthesis strategy can take into account one or more influencing factors such as data type, data size, and prediction category. For example, for image-based training data, the synthesis strategy can employ strategies such as background replacement, color perturbation, and affine transformation; for text-based training data, the synthesis strategy can employ strategies such as synonym replacement, structural rewriting, and data expansion.

[0071] Specifically, the real-world labeled data can be high-quality samples collected from actual business systems and whose category labels have been manually verified. A test set based on this real-world labeled data can be used to objectively evaluate the performance of the current classification model in real-world scenarios.

[0072] Model training relies on large amounts of data to learn complex patterns, while performance evaluation only requires samples with sufficient statistical significance. Therefore, in this embodiment, synthetic training data is used to provide a large number of training samples for the training phase, while a small amount of real labeled data is used for testing. This allows limited real labeled resources to be concentrated on building a reliable evaluation benchmark, while the synthetic training data bears the main training load, thus achieving efficient model development under the condition of limited labeling costs.

[0073] B120. Train a preset classification model using the training set, and test the trained preset classification model using the test set to obtain the trained current classification model.

[0074] Specifically, the preset classification model can be selected before the task starts. It can be implemented using an existing neural network architecture, and its type can be adapted according to the data modality of the target task. For example, for an image classification task, the preset classification model may include a convolutional neural network; for a text classification task, the preset classification model may use an existing pre-trained language model.

[0075] Specifically, during training, synthetic training data from the training set and their preset category labels are input into the preset classification model. The model parameters are iteratively updated until the loss function converges or a preset number of training epochs is reached. Since the training set is generated by a synthetic strategy and has a balanced category distribution, it effectively avoids model bias caused by the long-tail distribution in real data, thus improving the initial version of the classification model's basic recognition ability for each category.

[0076] Furthermore, after the model training is completed, it is used for inference on the test set to obtain the predicted labels and confidence scores for each test sample, and evaluation metrics such as accuracy, recall, and definiteness are calculated. Since the test set consists of high-quality labeled data collected from real business scenarios and manually verified, its evaluation results can truly reflect the model's generalization performance in actual application environments, rather than just its fitting effect on synthetic data.

[0077] More specifically, in one embodiment, if multiple preset classification models exist, steps B110 and B120 are performed on each of them respectively. This allows for a horizontal comparison of the preset classification models based on the comprehensive performance indicators on the test set, and the best-performing model is selected as the current classification model. That is, the selected model serves as the initial version of step S100, used for prediction and confidence analysis of the data to be classified.

[0078] Specifically, the steps of step B120 may include: B121. Train at least one candidate pre-defined classification model using the training set, and test the trained candidate pre-defined classification model using the test set to obtain the test results.

[0079] Understandably, this embodiment trains multiple candidate pre-defined classification models in parallel and obtains their performance under a unified evaluation benchmark, thereby providing an objective basis for subsequent model selection and avoiding performance limitations at the starting point of the overall optimization process due to improper selection of a single model.

[0080] Specifically, the at least one candidate pre-selected classification model can be one of several classification models with different network structures, and / or different parameter sizes, and / or different feature extraction capabilities, pre-selected during the task initialization phase. These candidate pre-selected classification models can cover mainstream architecture types to meet the diverse needs of performance, efficiency, and generalization ability.

[0081] Specifically, during the training phase, the training set constructed based on the synthetic strategy is input into each candidate pre-defined classification model, using the same training configuration, such as learning rate, batch size, optimizer, and number of training epochs, for independent training. This ensures that each model completes parameter learning under fair conditions. Since the training set consists of class-balanced synthetic training data, it effectively eliminates model performance differences caused by data bias, allowing the evaluation results to more accurately reflect the merits of the model architecture itself.

[0082] Furthermore, after each candidate pre-defined classification model is trained, it is uniformly deployed on the test set constructed from real labeled data for inference. The predicted label, confidence score, and inference time of each sample in the test set are recorded, and multi-dimensional test results are calculated based on these. The test results may include, but are not limited to, indicators such as overall accuracy, recall rate of each category, F1 score, confusion matrix, category miss rate, inference latency, and memory usage, forming a comprehensive performance profile.

[0083] B122. Based on the test results, obtain the candidate preset classification model with the best performance as the trained current classification model.

[0084] Specifically, the performance can ideally be based on a single optimal metric, or it can be a comprehensive performance priority set according to actual business needs, or it can be determined by combining the metric with the situation of misclassified samples. For example, in some high-risk scenarios, more attention may be paid to the recall rate of a few key categories; while in general product classification scenarios, the overall F1 score or accuracy may be the main metric.

[0085] In practice, the training process of the current classification model may also include the following steps: The synthesis strategy is adjusted using the test results, and new synthetic training data is resynthesized using the adjusted synthesis strategy. The training samples are updated based on the new synthetic training data; The updated training samples are used to optimize the current classification model.

[0086] Understandably, this embodiment aims to construct an iterative optimization mechanism for synthetic training data that is oriented towards model weakness feedback. By analyzing the performance defects of the classification model on the test set, it guides the fine-tuning of the synthesis strategy in reverse, thereby generating more targeted and challenging synthetic training data and continuously improving the robustness and generalization ability of the model.

[0087] Specifically, the test results can include macro-level indicators such as overall accuracy and recall, as well as fine-grained error analysis conclusions. For example, high-confusion category pairs can be identified through confusion matrices, such as dresses and skirts; sample-level backtracking can reveal that misclassified samples generally exhibit characteristics such as severe background interference, target occlusion, or abnormal lighting; or statistical analysis can show that certain rare categories suffer from systematic missed classifications. These analytical results directly reveal the shortcomings of current synthetic training data in terms of semantic coverage, scene diversity, or category discrimination.

[0088] Specifically, the test results are further analyzed based on the test results, and the synthesis strategy is adjusted based on the analysis conclusions, such as increasing the number of samples of easily confused categories, enhancing the data perturbation intensity, and introducing cross-category adversarial samples.

[0089] Specifically, after adjusting the synthesis strategy, a new batch of synthetic training data will be generated based on the new strategy, and then merged or replaced with the original training set to form updated training samples. This update process can be incremental, i.e., only adding new samples, or full, i.e., regenerating the entire training set, depending on resource and convergence speed requirements.

[0090] Furthermore, the updated training samples are used to optimize the already trained classification model. Since the new data focuses on historical weaknesses in the model, this training effectively corrects discriminative boundary vulnerabilities and improves the ability to identify easily confused, difficult, and long-tailed categories. After training, the performance is evaluated again on the same test set, and the above-described cycle of analysis, adjustment, synthesis, training, and evaluation is repeated until key indicators stabilize or meet preset thresholds.

[0091] Based on this, this embodiment establishes a test feedback-driven synthetic data self-evolution mechanism, so that model development in the cold start phase is no longer a one-way process, but a closed-loop optimization process with self-diagnosis and repair capabilities. This significantly improves the quality ceiling of the initial model and lays a more solid foundation for subsequent flywheel iterations based on real unlabeled data.

[0092] S200. When the current classification model needs to be updated, the current classification model is updated. It is understood that this embodiment can be applied to online business scenarios and respond to dynamic changes in the online business environment, such as hero appearance updates caused by game version iterations in game scenarios. When the potential degradation of the current classification model performance is detected, the model optimization process is automatically triggered, thereby realizing the continuous evolution and adaptive adjustment of classification capabilities.

[0093] Specifically, the criteria for determining whether the current classification model needs updating can be derived from the confidence distribution and / or recognition error rate and / or prediction stability output by the model when inferring about the data to be classified. For example, if it is detected that the confidence of the data to be classified is significantly lower than the historical average within a certain period, or the prediction results of a specific hero category in a game scenario fluctuate frequently, it can be determined that the model can no longer accurately identify the current data distribution. The data to be classified corresponding to the significantly reduced confidence or incorrect prediction results are then automatically sent back to the label correction stage as hard examples, thereby initiating the update mechanism to update the model.

[0094] Furthermore, this update process can be executed periodically or triggered by events. Event-driven triggers can include new version release notifications, the mean of the confidence sliding window falling below a threshold, etc., ensuring that the model is always synchronized with the latest business status. By transforming difficult online examples into high-value training resources, this embodiment effectively avoids the defect of traditional methods where the model becomes fixed upon deployment, significantly reducing the risk of recognition accuracy decay due to version iterations.

[0095] Specifically, such as Figure 4 As shown, the specific steps for updating the current classification model in step S200 may include: S210. Classify the data to be classified based on the category labels and confidence levels to obtain classification results; Understandably, this embodiment can refine the classification of the data to be classified based on the quality differences in the model prediction results, providing a basis for subsequent differential processing and avoiding noise injection or information waste caused by a one-size-fits-all data utilization strategy.

[0096] Specifically, the classification not only relies on confidence scores but also incorporates the statistical distribution characteristics of each category to achieve adaptive threshold classification of categories.

[0097] Specifically, step S210 may include the following steps: S211. Perform statistical analysis on the category labels of the data to be classified to obtain statistical results; Understandably, the statistics are used to quantify the frequency of occurrence of each category in the data to be classified, forming a category distribution profile. For example, the category labels of all data to be classified are accumulated within a preset time window, and the sample size or proportion of each category is calculated as the basis for subsequent category division.

[0098] S212. Determine the common category and the rare category based on the statistical results; Specifically, a threshold can be set; categories exceeding the threshold are classified as common, while others are classified as rare. This classification can be dynamically updated to adapt to changes in popularity brought about by version iterations.

[0099] S213. Based on the first confidence threshold and the second confidence threshold of the ordinary category, the data to be classified belonging to the ordinary category is classified into levels to obtain third confidence samples, third confidence samples, and third confidence samples carrying category classification labels. Specifically, in the ordinary category, a first confidence threshold is set. Second confidence threshold ,in For the confidence level Its classification rules are as follows: like Then the corresponding data to be classified is used as the first confidence sample.

[0100] like Then the corresponding data to be classified is used as the second confidence level sample.

[0101] like Then the corresponding data to be classified is used as the third confidence level sample.

[0102] S214. Based on the first confidence threshold and the second confidence threshold of the rare category, the data to be classified belonging to the rare category is classified to obtain the first confidence sample, the second confidence sample, and the third confidence sample carrying the category classification label. Specifically, the first confidence threshold of the common category is greater than its second confidence threshold, and the first confidence threshold of the rare category is greater than its second confidence threshold; furthermore, the first confidence threshold of the common category is greater than the first confidence threshold of the rare category, and the second confidence threshold of the common category is greater than the second confidence threshold of the rare category.

[0103] Specifically, relative to the first and second confidence thresholds for the common category, the rare category can be set with a reduction range. Determine its first confidence threshold Second confidence threshold , specifically: For example, in a game hero classification scenario, "popular hero A" accounts for 30% of the screenshots uploaded by players, while "unpopular hero Z" accounts for only 0.2%. If the same first confidence threshold is applied to both, valid samples of "hero Z" may not even be able to enter the training set, and the model's ability to recognize it will continue to deteriorate. Therefore, a more lenient adoption standard needs to be set for rare categories, making it easier for samples of rare categories to enter the training set, thereby improving the model's performance in that category and significantly improving the overall class balance of the model.

[0104] S220. The training samples are expanded using the classification results to obtain new training samples; wherein the training samples are divided into a training set and a test set; Understandably, in this embodiment, instead of simply merging all samples, a differentiated, high-value data augmentation strategy is implemented based on the grading results to ensure that the new samples can both improve training effectiveness and enhance evaluation coverage.

[0105] Specifically, the first confidence samples are used directly for training; the second confidence samples are added to the training set after validation; and some third confidence samples, especially rare categories, are used to expand the test set to build a more challenging evaluation environment.

[0106] Specifically, such as Figure 5 As shown, the specific steps of step S220 may include: S221. Add the first confidence sample to the training set; Specifically, since the first confidence sample has high predictive reliability, the category classification label obtained after classification can be regarded as an approximate true label. It can be directly included in the training set to safely expand the data scale without additional verification overhead.

[0107] S222. Perform category classification label verification on the second confidence sample, and add the verified correct second confidence sample to the training set.

[0108] Specifically, the steps for verifying the category classification label of the second confidence sample in step S222 may include: Perform category prediction on the second confidence sample to obtain the predicted category label; The consistency between the category classification label and the category prediction label of the second confidence sample is verified.

[0109] In practice, the second confidence level sample is used to predict the category again to obtain the predicted category label. Then, the original category label is compared with the predicted category label. If they match, the label is considered reliable and adopted; otherwise, it is discarded or sent to a manual review queue. This mechanism effectively filters out erroneous pseudo-labels caused by occasional model fluctuations.

[0110] Understandably, the category classification label verification in this step can be achieved by introducing a visual language model or a large language model with strong semantic understanding capabilities, combined with a small number of task-related examples, to perform intelligent and context-aware secondary judgment on the category prediction labels of the second confidence samples, thereby achieving high-precision label confirmation without human intervention.

[0111] Specifically, visual language models refer to a class of multimodal artificial intelligence models that can simultaneously understand image content and natural language semantics. Through large-scale image-text pair pre-training, they possess cross-modal alignment capabilities and can answer questions, describe scenes, or perform fine-grained classification tasks based on images.

[0112] Large language models refer to deep neural network models trained on massive amounts of text data, possessing powerful language generation, reasoning, and contextual understanding capabilities. Although traditional large language models only process text, they can also participate in semantic judgment for multimodal tasks after being connected to an image encoder or invoked through tools.

[0113] For example, in a game hero classification scenario, a prompt template containing positive and negative examples is first constructed. For instance: "You are a professional game hero identification assistant. Please determine the hero category based on the following image and description."

[0114] Example 1 (positive example): [Image: Clearly showing the skill effects of 'Hero A'] → Tag: Hero A Example 2 (Negative Example): [Image: 'Hero B' and 'Hero C' look similar but have different clothing details] → Tag: Hero B Sample to be judged: [Input the current second confidence level sample image] Please output the most likely hero name. By inputting the prompt template into the visual language model, its independent category judgment for the current sample can be obtained, i.e., the category prediction label. Subsequently, this result is compared with the category classification label output by the original classification model: if they match, the label is considered highly reliable and adopted; if they do not match, it is considered a potential error and is not added to the training set, or sent to the manual review queue.

[0115] Specifically, the limited number of examples is typically selected from a validated, high-quality sample library, covering typical correct cases and easily confused negative examples, ensuring that the visual language model fully understands the task boundaries and key discrimination points. The number of examples is generally 2–5, balancing the prompting effect with inference cost.

[0116] Furthermore, this verification process not only determines whether the label is correct, but also outputs reasons for confidence, providing interpretability support for subsequent analysis.

[0117] By leveraging a collaborative mechanism between a large language model / visual language model and a small number of examples, this embodiment achieves low-cost, high-precision, and scalable automated verification of second-confidence samples. It effectively solves the problem of pseudo-label noise accumulation in traditional semi-supervised methods and provides a reliable data source for expanding the training set.

[0118] S223. The third confidence samples belonging to the rare category are subjected to category classification label verification, and the verified third confidence samples are added to the test set; wherein the rare category is determined according to statistical results, which are obtained by statistically analyzing the category classification labels of the data to be classified.

[0119] Specifically, because rare categories have scarce samples during training and inference, the model's discrimination boundary is often ambiguous, leading to low confidence outputs even when faced with valid input. Directly discarding these samples would result in the permanent loss of crucial scene information. Therefore, this embodiment prioritizes sampling third-confidence samples from rare categories for category classification label verification, ensuring that less common but important categories receive sufficient attention.

[0120] Specifically, for the rare category third-confidence samples obtained through sampling, existing visual language models or large language models are used for label verification. The specific steps may include: The third confidence level sample is used to predict the category, and the predicted category label is obtained. The consistency between the category classification label and the category prediction label of the third confidence sample is verified.

[0121] Furthermore, these verified samples are not used directly for training, but are added to the test set to build an evaluation environment that more closely reflects real-world business challenges.

[0122] Specifically, step S220 in this embodiment achieves dynamic evolution of the test set and adaptive expansion in complex scenarios through a dual-track strategy of training set expansion and test set evolution, which significantly improves the credibility, robustness and business guidance value of model evaluation.

[0123] S230. Retrain the current classification model based on the new training samples to obtain an updated classification model.

[0124] S240. When the updated classification model does not meet the preset convergence condition, the data to be classified is reclassified using the updated classification model to obtain new category classification labels and confidence scores, and based on the new category classification labels and confidence scores, the step of classifying the data to be classified is returned. Understandably, this embodiment constructs a flywheel-style iterative optimization mechanism to ensure that the model continuously absorbs the latest data feedback before reaching stable performance.

[0125] Specifically, the preset convergence condition can be set based on one or more conditions such as accuracy, false negative rate, and number of iterations. Alternatively, it can be based on the overall confidence level, the stability of the distribution of each category (e.g., KL divergence below 0.05), or the recall rate of key problem categories meeting business requirements. If these conditions are not met, the updated classification model is used to reprocess the data to be classified, generating more reliable prediction results, thereby triggering a new round of grading and expansion.

[0126] For example, in a game hero classification scenario, the first-round model misclassifies "Hero X" as "Hero Y" (confidence level 0.55); after verification and correction, it is added to the training set; the second-round model predicts the sample as "Hero X" (confidence level 0.88), successfully correcting the error. This process can be repeated multiple times to gradually purify the training data and strengthen the discrimination boundary.

[0127] For example, in the game hero classification scenario, if the model's recall rate for "newly launched assassin heroes" is only 58%, then this category is marked as a "key issue category" and included in the key issue set.

[0128] Specifically, when reprocessing the data to be classified using the updated classification model, the confidence threshold strategy can be dynamically adjusted: for key question categories, the first / second confidence threshold can be appropriately reduced to retain more potentially valid samples; at the same time, the prediction strategy for class prediction of second and third confidence samples can be optimized, such as optimizing the prompt template content of the large language model, or adding contrast cases in a small number of examples, thereby strengthening the large language model's ability to identify the class boundary.

[0129] Specifically, after obtaining the new category classification label and confidence level in step S240, the process returns to step S210 for hierarchical processing, and steps S210-S240 are executed sequentially until the convergence condition is met, at which point step S250 is executed.

[0130] S250. When the updated classification model satisfies the convergence condition, the updated classification model is obtained.

[0131] Understandably, the execution of step S250 marks the completion of this round of flywheel iteration, outputting a classification model with stable performance under the current data distribution.

[0132] Specifically, the converged classification model will be used in subsequent online services and will serve as the current classification model for the next triggered update. Simultaneously, its performance on the test set can be used as a baseline to monitor future performance degradation.

[0133] This embodiment utilizes S210–S250 to form an automated, category-aware, and verification-driven continuous optimization closed loop, enabling the classification model to efficiently utilize unlabeled online data under the condition of limited labeling costs, and achieve dynamic evolution and long-term stability of model performance.

[0134] Furthermore, in a preferred embodiment, after convergence, human-machine collaborative refinement can be performed to perform stratified sampling, label correction, and fine-tuning of misjudged samples in the test set, thereby further improving the model accuracy.

[0135] Specifically, the new training samples are divided into a new training set and a new test set; the method in this embodiment also includes: S260. When the updated classification model satisfies the convergence condition, the misclassified samples of the updated classification model on the new test set are also obtained. Understandably, meeting the convergence condition indicates that the model has reached a preliminary stable state through the previous data flywheel iteration. However, to further improve its robustness and reliability in real business, a refined manual collaborative refinement stage can be added.

[0136] Specifically, although the overall model metrics have converged, in actual deployment, misclassification of a small number of key categories can still lead to serious business consequences. Therefore, the updated classification model can be used to perform full inference on the expanded new test set, and all samples whose predicted labels do not match the true labels can be extracted as misclassified samples.

[0137] S270. Perform stratified sampling on the misjudged samples according to their categories to obtain sampled samples; Understandably, misjudged samples are often concentrated in a few key issue categories. If full manual review is too costly, a stratified sampling strategy should be adopted to prioritize the coverage density of high-risk or high-frequency misjudged categories.

[0138] In the specific implementation process, the number of misjudged samples in each category can be counted first, and stratified sampling can be carried out according to the proportion or by setting a minimum sampling size.

[0139] Furthermore, the sampling process can incorporate confidence level distributions, prioritizing the selection of misjudged samples from the second and third confidence level samples to avoid the first confidence level sample erroneously dominating the analysis direction.

[0140] S280. Perform category correction on the sampled samples; Understandably, step S280 can use a human-machine collaboration mechanism to diagnose and correct the labels of the sampled data, distinguish between dirty data and difficult samples, and implement differentiated processing.

[0141] Specifically, samples that cannot be correctly classified due to quality issues can be classified as dirty data and directly removed; samples that are clear and effective but cause model confusion due to similar category semantics or subtle differences in features can be classified as difficult samples and enter the label correction process.

[0142] For difficult samples, a large language model or visual language model can be invoked, combined with a small number of task-related examples to generate auxiliary judgment suggestions for human reference. Finally, the correct label is confirmed by humans to form a high-confidence correction result.

[0143] S290. The corrected sampling samples are added to the new training samples, and the updated classification model that meets the convergence condition is fine-tuned using the new training samples with the added sampling samples to obtain the final model.

[0144] Specifically, difficult samples, identified through collaboration between human reviewers and large-scale models, are added to new training data as high-value training data due to their combination of realism, challenge, and label accuracy. Because of their limited quantity but dense information, fine-tuning is typically employed instead of full retraining. Fine-tuning methods may involve freezing the backbone network parameters and updating only the classification head or the last few layers to absorb key knowledge with minimal computational cost.

[0145] Furthermore, the fine-tuned model is evaluated again on an expanded test set to verify whether the key problem category metrics have significantly improved. If there are still residual problems, the process returns to S260 to start a new round of refinement; if all key problems have been resolved and the overall performance is stable, the model is output as the final model.

[0146] In this embodiment, S260–S290 together constitute a final optimization mechanism for high reliability. Through the process of misjudgment focusing, hierarchical sampling, human-machine collaborative correction, and precise fine-tuning, residual errors are further eliminated after the model converges, ensuring that it has long-term stable and high-precision classification capabilities in real complex scenarios, which is especially suitable for business systems with stringent reliability requirements.

[0147] S300: Classify the new data to be classified based on the updated classification model.

[0148] Understandably, step S300 involves applying the continuously trained and refined updated classification model to real-world business scenarios to perform efficient and accurate category discrimination on the constantly generated new data.

[0149] The updated classification model mentioned in step S300 can be the updated classification model obtained in step S230, the updated classification model obtained in step S250, or the final model obtained in step S290.

[0150] Specifically, the new data to be classified refers to the original input data newly collected from the online business system after the model update, which has not participated in this round of training or verification, such as newly uploaded game screenshots by users, newly added product images on e-commerce platforms, and the latest user consultation text received by the customer service system.

[0151] Furthermore, the classification results obtained after classifying the new data not only serve the current business requests, but the predicted category labels and confidence levels are also recorded and fed into the data pipeline as potential inputs for the next flywheel iteration. If a large amount of new data exhibits low confidence or concentrated misclassifications, the model update mechanism is automatically triggered, forming a long-term adaptive closed loop of service, feedback, optimization, and re-service.

[0152] In summary, step S300 is not only the final output of the model's capabilities, but also the entry point for the continuous learning system. By deploying the highly reliable updated model into real-world scenarios, it ensures both current business accuracy and real-time performance, while also laying the groundwork for future performance evolution, truly enabling the classification system to operate stably and evolve autonomously in dynamic environments over the long term.

[0153] like Figure 6 As shown in the embodiments of this application, a data classification system is also provided. Optionally, the system includes: Classification module 411, update module 412, wherein: Classification module 411 is used to classify the data to be classified using the current classification model pre-trained based on training samples, and obtain the category label and confidence score; In this embodiment, the classification module 411 can be used to perform... Figure 2 For a detailed description of the classification module 411, please refer to the description of step S100 shown.

[0154] The update module 412 is used to update the current classification model when the current classification model needs to be updated. In this embodiment, the update module 412 can be used to perform... Figure 2 For a detailed description of step S200 shown, please refer to the description of step S200.

[0155] The classification module 412 is also used to classify the new data to be classified based on the updated classification model; In this embodiment, the classification module 412 can also be used to perform... Figure 2 For a more detailed description of the classification module 412, please refer to the description of step S300 shown.

[0156] Updating the current classification model includes: The data to be classified is graded based on the category labels and confidence scores to obtain the grading results. The training samples are expanded using the classification results to obtain new training samples; The current classification model is retrained based on the new training samples to obtain an updated classification model.

[0157] This application also provides an electronic device, the structure of which is as follows: Figure 7 As shown, the electronic device includes a memory 611, a processor 612, a communication module 613, and an input / output interface 614, etc. Optionally, the memory 611, the processor 612, the communication module 613, and the input / output interface 614 can be connected and communicate with each other through a bus 615.

[0158] The memory 611 is used to store one or more computer programs and to transfer the code of the computer programs to the processor 612; when the one or more computer programs are executed by the processor 612, a data classification method in this application embodiment is implemented.

[0159] Optionally, the electronic device can be connected to a network via communication module 613 to communicate with other devices, such as terminals or servers, to achieve data interaction. The electronic device can be various forms of digital computers, exemplarily such as desktop computers, servers, workbenches, mainframes, or other types of computers. The electronic device can also be various forms of mobile terminals, exemplarily such as smartphones, tablets, wearable devices (such as helmets, glasses, watches, etc.), and other similar mobile terminals.

[0160] Optionally, the electronic device can connect to required input / output devices, such as a keyboard or display device, via the input / output interface 614. The electronic device itself may have a display device, and other display devices can also be connected externally via the input / output interface 614. Optionally, a storage device, such as a hard disk, can also be connected via the input / output interface 614 to store data from the electronic device, read data from the storage device, or store data from the storage device in the memory 611. It is understood that the input / output interface 614 can be a wired interface or a wireless interface. Depending on the actual application scenario, the device connected to the input / output interface 614 can be a component of the electronic device or an external device connected to the electronic device when needed.

[0161] Optionally, the memory 611 may be a volatile memory and / or a non-volatile memory. The volatile memory may be a random access memory, etc., and the non-volatile memory may be a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, or a flash memory, etc.

[0162] Optionally, the computer program stored in the memory 611 can be divided into one or more modules, which are stored in the memory 611 and executed by the processor 612 to perform the method provided in this embodiment. The one or more modules can be a series of computer program instruction segments capable of performing specific functions, which describe the execution process of the computer program in the electronic device.

[0163] Optionally, the processor 612 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of the processor 612 include, but are not limited to, a central processing unit, a graphics processing unit, a digital signal processor, various special-purpose artificial intelligence computing chips, various processors running machine learning model algorithms, and can also be any suitable controller, microcontroller, processor, etc. The processor 612 executes the various methods and processes of this embodiment, exemplarily, such as a data classification method according to an embodiment of this application.

[0164] Optionally, the bus 615 may include a path for transmitting information. Depending on its function, the bus 615 may be divided into an address bus, a data bus, a control bus, etc.

[0165] In an optional implementation, this application embodiment also provides a computer storage medium storing a computer program thereon, which, when executed by a computer, enables the computer to perform the methods described in the above-described method embodiments. Part or all of the computer program may be loaded and / or installed on the memory 611 of an electronic device. When the computer program is executed by the processor 612, one or more steps of a data classification method according to an embodiment of this application can be performed.

[0166] Optionally, the computer-readable storage medium may be a random access memory, a read-only memory, a programmable read-only memory, an erasable programmable read-only memory, an electrically erasable programmable read-only memory, etc.

[0167] Obviously, the above embodiments of the present invention are merely examples for clearly illustrating the technical solution of the present invention, and are not intended to limit the specific implementation of the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the claims of the present invention should be included within the protection scope of the claims of the present invention.

Claims

1. A data classification method, characterized in that, The method includes: The current classification model, pre-trained based on training samples, is used to classify the data to be classified, and the category labels and confidence scores are obtained. When the current classification model needs to be updated, the current classification model is updated, and the new data to be classified is classified based on the updated classification model; Updating the current classification model includes: The data to be classified is graded based on the category labels and confidence scores to obtain the grading results. The training samples are expanded using the classification results to obtain new training samples; The current classification model is retrained based on the new training samples to obtain an updated classification model.

2. The method according to claim 1, characterized in that, The update of the current classification model also includes: When the updated classification model fails to meet the preset convergence condition, the data to be classified is reclassified using the updated classification model to obtain new category labels and confidence scores. Based on the new category labels and confidence scores, the process of grading the data to be classified is returned. When the updated classification model satisfies the convergence condition, the updated classification model is obtained.

3. The method according to claim 1, characterized in that, The classification result includes a first confidence sample and a second confidence sample carrying category classification labels, wherein the confidence of the first confidence sample is greater than the confidence of the second confidence sample; The training samples are divided into a training set and a test set; The step of expanding the training samples using the grading results to obtain new training samples includes: Add the first confidence sample to the training set; The second confidence sample is subjected to category classification label verification, and the verified correct second confidence sample is added to the training set.

4. The method according to claim 3, characterized in that, The verification of the category classification label for the second confidence sample includes: Perform category prediction on the second confidence sample to obtain the predicted category label; The consistency between the category classification label and the category prediction label of the second confidence sample is verified.

5. The method according to claim 3, characterized in that, The classification result also includes a third confidence sample carrying a category classification label, wherein the confidence of the second confidence sample is greater than the confidence of the third confidence sample; The step of expanding the training samples using the grading results to obtain new training samples further includes: The third confidence samples belonging to the rare category are subjected to category classification label verification, and the verified third confidence samples are added to the test set; wherein the rare category is determined according to statistical results, which are obtained by statistically analyzing the category classification labels of the data to be classified.

6. The method according to any one of claims 1 to 5, characterized in that, The step of classifying the data to be classified based on the category labels and confidence scores to obtain the classification results includes: The category labels of the data to be classified are statistically analyzed to obtain statistical results. Based on the statistical results, common and rare categories are determined; Based on the first and second confidence thresholds of the common category, the data to be classified belonging to the common category is graded to obtain first confidence samples, second confidence samples, and third confidence samples carrying category classification labels; Based on the first and second confidence thresholds for rare categories, the data to be classified belonging to rare categories are graded to obtain first confidence samples, second confidence samples, and third confidence samples carrying category classification labels. Specifically, the first confidence threshold of the common category is greater than its second confidence threshold, and the first confidence threshold of the rare category is greater than its second confidence threshold; furthermore, the first confidence threshold of the common category is greater than the first confidence threshold of the rare category, and the second confidence threshold of the common category is greater than the second confidence threshold of the rare category.

7. The method according to any one of claims 1 to 5, characterized in that, When classifying the data to be classified using the current classification model, the predicted probability of the category of the data to be classified is also obtained; Obtain the highest and second-highest predicted category probabilities corresponding to the data to be classified; The confidence level is obtained based on the highest category prediction probability and the second highest category prediction probability.

8. The method according to claim 2, characterized in that, The new training samples are divided into a new training set and a new test set; When the updated classification model satisfies the convergence condition, the misclassified samples of the updated classification model on the new test set are also obtained. The misclassified samples were stratified and sampled according to their categories to obtain the sampled samples; The sampled data is then subjected to category correction; The corrected sampled samples are added to the new training samples, and the updated classification model that meets the convergence condition is fine-tuned using the new training samples with the added sampled samples to obtain the final model.

9. The method according to any one of claims 1 to 5, characterized in that, The current classification model is obtained in advance through the following method: The training samples include a training set constructed based on synthetic training data and a test set constructed based on real labeled data; A preset classification model is trained using the training set, and the trained preset classification model is tested using the test set to obtain the trained current classification model.

10. The method according to claim 9, characterized in that, The step of training a preset classification model using the training set and testing the trained preset classification model using the test set to obtain the trained current classification model includes: At least one candidate predefined classification model is trained using the training set, and the trained candidate predefined classification model is tested using the test set to obtain test results; Based on the test results, the candidate preset classification model with the best performance is obtained as the trained current classification model.

11. The method according to claim 10, characterized in that, The synthetic training data is synthesized based on a synthetic strategy. The method further includes: The synthesis strategy is adjusted using the test results, and new synthetic training data is resynthesized using the adjusted synthesis strategy. The training samples are updated based on the new synthetic training data; The updated training samples are used to optimize the current classification model.

12. A data classification system, characterized in that, The system includes: The classification module is used to classify the data to be classified using the current classification model pre-trained based on training samples, and to obtain the category label and confidence score. The update module is used to update the current classification model when the current classification model needs to be updated. The classification module is also used to classify new data to be classified based on the updated classification model; Updating the current classification model includes: The data to be classified is graded based on the category labels and confidence scores to obtain the grading results. The training samples are expanded using the classification results to obtain new training samples; The current classification model is retrained based on the new training samples to obtain an updated classification model.

Images