Classification model training method, system and device, storage medium and product

Abstract

The invention discloses a classification model training method, system and device, a storage medium and a product, and relates to the technical field of artificial intelligence models. The training method comprises the following steps: acquiring a marked data set; loading a first classification model, and training the first classification model by using the marked data set; evaluating the trained first classification model to evaluate the performance of the model; determining defects of the first classification model according to the evaluation result; generating synthetic data according to defects of the first classification model; screening the synthetic data to obtain high-quality synthetic data; and merging the high-quality synthetic data with the marked data set, and training the trained first classification model again by using the merged data set to obtain a target classification model. According to the method, the recognition capability of the target classification model on various categories is improved, and the problem of sample data imbalance is relieved.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence model technology, and in particular to a classification model training method, system, device, storage medium and product. Background Art

[0002] With the development of artificial intelligence technology, classification models are increasingly used in all walks of life, and the high performance of classification models depends on high-quality labeled data. Usually, due to the high cost of obtaining real data or technical security issues, it is difficult to obtain a large amount of real data. Data scarcity leads to weak generalization ability of classification models, and real data often contains sensitive information (such as personal information). For example, real data in the fields of medical care, finance, etc., direct use may lead to privacy leakage; at the same time, real data may contain noise, outliers, etc., which affect the performance of artificial intelligence models. Therefore, synthetic data has been increasingly widely used in the field of artificial intelligence models. The cost of obtaining synthetic data is relatively low, which avoids privacy leakage, and the data is purer, which is more beneficial to improving the performance of the model; synthetic data can effectively expand the data set and improve the generalization ability of the model.

[0003] In order to obtain labeled data samples, data needs to be labeled. Traditional data labeling methods mainly rely on manual work, which is not only costly and inefficient, but also has a quality that is affected by the professional knowledge and subjective factors of the labelers, making it difficult to ensure consistency.

[0004] Currently, in many fields, especially emerging fields, high-quality labeled data is often scarce, which leads to poor performance of classification models. Summary of the invention

[0005] The present invention provides a classification model training method, system, device, storage medium and product to solve at least one of the problems of poor classification model performance caused by scarcity of high-quality labeled data and low efficiency and poor consistency caused by manual labeling.

[0006] In a first aspect, the present invention provides a classification model training method, comprising: Get a labeled dataset; Loading a first classification model, and training the first classification model using the labeled data set; Evaluate the first classification model after training to assess model performance; Determine the defects of the first classification model according to the evaluation results; generating synthetic data according to the defects of the first classification model; Screening the synthetic data to obtain high-quality synthetic data; The high-quality synthetic data is merged with the labeled data set, and the trained first classification model is retrained using the merged data set to obtain a target classification model.

[0007] Furthermore, the process of acquiring the labeled data set includes: Get the original data set; Extracting part of the data from the original data set to form a sampled data subset; Using a data classification agent to preliminarily classify the data in the sampled data subset, and using a classification verification agent to verify the preliminary classification results to generate initial labeled data; Loading a second classification model, and using the initial labeled data to train and verify the second classification model to obtain verified data; All verified data are clustered and fused to obtain clustering results and classification of each data, and a labeled data set is constructed based on the classified data.

[0008] Furthermore, the process of acquiring the labeled data set also includes: Using different scoring agents to score the data in the labeled data set, and grading the quality or credibility of each data in the labeled data set according to the scoring results; wherein different scoring agents are configured with different scoring criteria; The third classification model is loaded, and the third classification model is trained and verified using the classified data to obtain high-quality labeled data, and then the final labeled data set is constructed based on the high-quality labeled data.

[0009] Furthermore, before using the classified data to train and verify the third classification model, the process of acquiring the labeled data set also includes: The classified data are deduplicated, clustered and screened, and the third classification model is trained and verified using the screened data.

[0010] In a second aspect, the present invention further provides a classification model training system, including a model defect training subsystem, wherein the model defect training subsystem includes: A first acquisition unit, used to acquire a labeled data set; A first model training agent is used to load a first classification model and train the first classification model using the labeled data set; An evaluation unit, used to evaluate the trained first classification model to assess model performance; A defect detection agent, used to determine defects of the first classification model according to the evaluation results; A data synthesis agent, used to generate synthetic data according to the defects of the first classification model; A data screening agent, used for screening the synthetic data to obtain high-quality synthetic data; The second model training agent is used to merge the high-quality synthetic data with the labeled data set, and use the merged data set to retrain the trained first classification model to obtain a target classification model.

[0011] Furthermore, the classification model training system also includes an automatic marking subsystem and an automatic screening subsystem; the automatic marking subsystem includes: A second acquisition unit, used to acquire an original data set; A data sampling agent, used to extract part of the data from the original data set to form a sampled data subset; A data classification agent, used for preliminarily classifying the data in the sampled data subset; The first classification verification agent is used to verify the preliminary classification results and generate initial labeled data; A third model training agent is used to load the second classification model and train the second classification model using the initial labeled data; The second classification verification agent is used to verify the second classification model trained by the third model training agent to obtain verified data; A clustering and fusion processing unit is used to perform clustering and fusion processing on all verified data to obtain clustering results and classification of each data; Label fusion agent, used to construct a labeled dataset based on the classified data; The automatic screening subsystem comprises: A third acquisition unit, used to acquire a labeled data set from the automatic labeling subsystem; A plurality of scoring agents, each of which is used to score the data in the labeled data set, and to grade the quality or credibility of each data in the labeled data set according to the scoring result; wherein different scoring agents are configured with different scoring criteria; The fourth model training agent is used to load the third classification model and train the third classification model using the classification data; The third classification verification agent is used to verify the third classification model trained by the fourth model training agent to obtain high-quality labeled data, and then construct the final labeled data set based on the high-quality labeled data.

[0012] Furthermore, the classification model training system also includes a basic module, and the basic module includes: A data distribution processing unit, used to distribute data to multiple processing nodes to achieve parallel processing; Data tool API management unit, used to provide a unified API interface to facilitate the calling of various data processing tools; Visual management unit, used to visualize the data processing flow of each subsystem; Model management unit, used to manage models, including model storage, version control, and online / offline switching; The evaluation access unit is used to provide interfaces for accessing various evaluation indicators and data sets.

[0013] In a third aspect, the present invention also provides an electronic device comprising a memory, a processor, and a computer program / instructions stored on the memory, wherein the processor executes the computer program / instructions to implement the steps in the classification model training method as described above in the present application.

[0014] In a fourth aspect, the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the steps in the classification model training method described above in the present application.

[0015] In a fifth aspect, the present invention also provides a computer program product, including a computer program / instructions, which, when executed by a processor, implement the steps in the classification model training method described above in the present application.

[0016] The present invention uses high-quality synthetic data to expand the labeled data set, which greatly improves the generalization ability of the classification model; the synthetic data protects data privacy without affecting the model training effect, and the synthetic data is purer and free of noise, which improves the accuracy of the model; To address the problem that the classification model has weak recognition ability for minority categories, the present invention generates synthetic data (generates samples of minority categories) based on the defects of the classification model (i.e., recognition weaknesses), and then uses the synthetic data and the original labeled data set to perform incremental training on the classification model, thereby improving the classification model's recognition ability for various categories and alleviating the problem of sample data imbalance. BRIEF DESCRIPTION OF THE DRAWINGS

[0017] In order to more clearly illustrate the technical solution of the present invention, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only one embodiment of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0018] Figure 1 This is a flow chart of the classification model training method in an embodiment of the present invention; Figure 2 This is a structural diagram of a classification model training system in an embodiment of the present invention; Figure 3 Schematic diagram of the structure of an electronic device in an embodiment of the present invention. DETAILED DESCRIPTION

[0019] The following is a clear and complete description of the technical solutions in the present invention in conjunction with the drawings in the embodiments of the present invention. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without creative work are within the scope of protection of the present invention.

[0020] The technical solution of the present application is described in detail with specific embodiments below. The following specific embodiments can be combined with each other, and the same or similar concepts or processes may not be described in detail in some embodiments.

[0021] Embodiment 1 like Figure 1 As shown, a classification model training method provided by an embodiment of the present invention includes the following steps: Step S1: obtaining a labeled data set; Step S2: loading a first classification model, and training the first classification model using the labeled data set; Step S3: evaluating the trained first classification model to evaluate the model performance; Step S4: determining the defects of the first classification model according to the evaluation results; Step S5: generating synthetic data according to the defects of the first classification model; Step S6: screening the synthetic data to obtain high-quality synthetic data; Step S7: merging the high-quality synthetic data with the labeled data set, and using the merged data set to re-train the trained first classification model to obtain a target classification model.

[0022] By evaluating the trained first classification model, we can accurately locate the categories in which the first classification model performs poorly, and thus obtain the defects (or classification weaknesses) of the first classification model. Then, based on the defects of the first classification model, we can generate synthetic data that is highly targeted and can effectively make up for the defects of the first classification model. At the same time, we can also filter the synthetic data, remove low-quality data, and retain high-quality synthetic data. Finally, we use high-quality synthetic data and labeled data sets to incrementally train the first classification model, which enhances the classification ability of the first classification model in terms of defects. Through continuous iterative training, the performance of the first classification model is continuously improved.

[0023] The present invention utilizes synthetic data to effectively expand the training data set, improves the generalization ability of the first classification model, and at the same time reduces the dependence on real data and protects the privacy of data protection.

[0024] The obtained target classification model is used to classify or identify targets in image data, which greatly improves the classification accuracy.

[0025] In order to solve the problems of high cost, low efficiency and poor consistency of manual labeling, the present invention adopts multiple intelligent agents (ie, agents) to realize automatic data labeling and generate labeled data (ie, data with labels). In a specific embodiment of the present invention, the process of obtaining a labeled data set in step S1 includes: step S1.1: obtaining an original data set; step S1.2: extracting part of the data from the original data set to form a sampled data subset; step S1.3: using a data classification agent to perform preliminary classification of the data in the sampled data subset to generate labeled data; step S1.4: using a classification verification agent to verify the preliminary classification results to generate initial labeled data; by verifying the correctness of the preliminary classification, the quality of the labeled data is ensured; step S1.5: loading a second classification model, training and verifying the second classification model using the initial labeled data to obtain verified data; repeating step S1.5 multiple times (i.e., N times), and after each repetition of step S1.5, the accuracy of the second classification model will be further improved until satisfactory performance is achieved, which provides better input data for subsequent clustering; step S1.6: clustering and fusion processing are performed on all verified data to obtain clustering results and classification of each data, and a labeled data set is constructed based on the classified data.

[0026] In this embodiment, a clustering model (such as a BGE model) is used to cluster all verified data, and the data is divided into different categories based on a certain similarity measure or clustering algorithm; then all clustering results are summarized and fused to obtain clustering results and classifications of each data, and finally a unified labeling system is generated.

[0027] The present invention automatically completes data annotation through data sampling, classification, training and verification, clustering and label fusion, covering the classification and clustering results of each data. In the entire process of automatic data annotation, through the collaboration of automation tools and various functional intelligent bodies, a large amount of data can be efficiently processed and accurate labeling results can be generated, which greatly improves the efficiency of data processing and the consistency of labeling, and is particularly suitable for the preprocessing and labeling of large-scale data sets.

[0028] In order to further improve the quality of labeled data, the present invention further screens and verifies the labeled data obtained by automatic annotation to generate high-quality labeled data. In a specific embodiment of the present invention, the process of obtaining a labeled data set also includes: step S1.7: using different scoring agents to score the data in the labeled data set obtained in step S1.6, and according to the scoring results, the quality or credibility of each data in the labeled data set is graded; step S1.8: loading a third classification model, using the graded data to train and verify the third classification model, obtaining high-quality labeled data, and then constructing the final labeled data set based on the high-quality labeled data.

[0029] In this embodiment, different scoring agents are configured with different scoring criteria, such as scoring criteria based on data quality, scoring criteria based on feature consistency, scoring criteria based on classification accuracy, etc. In order to further improve the accuracy of the data, various tools are also used to process the hierarchical data, including: using deduplication tools to deduplicate the hierarchical data, remove duplicate data, and ensure the independence and information content of the data; using clustering tools to perform cluster analysis on the deduplicated data, which will help improve the consistency of the data; using retrieval tools to retrieve the data after cluster analysis according to specific retrieval algorithms or rules, and further screen out data that meets specific conditions.

[0030] The third classification model is cyclically trained and verified using graded data or screened data to improve its classification capability so that it can process different categories of data more accurately. The trained third classification model is verified to evaluate its performance on new data to ensure that it can stably process various types of data.

[0031] After multiple rounds of training and verification, high-quality labeled data are finally screened out. These labeled data have been fully verified, which effectively improves the quality of the data. These high-quality labeled data are then applied to downstream tasks, such as model training and evaluation, to ensure the reliability and consistency of the data.

[0032] The first classification model, the second classification model and the third classification model in the embodiment of the present invention may be the same classification model or different classification models.

[0033] Embodiment 2 like Figure 2As shown, a classification model training system provided by an embodiment of the present invention includes a model defect training subsystem, wherein the model defect training subsystem includes a first acquisition unit, a first model training agent, an evaluation unit, a defect detection agent, a data synthesis agent, a data screening agent, and a second model training agent. Through the collaboration of multiple agents to synthesize data for the weaknesses of the classification model, the synthesized data is used to train the weaknesses of the classification model, thereby enhancing the classification ability of the classification model in terms of weaknesses.

[0034] The first acquisition unit is used to acquire a labeled data set, which can be acquired through manual annotation, or through a subsequent automatic annotation subsystem, or through an automatic annotation subsystem and an automatic screening subsystem; The first model training agent is used to load the first classification model and train the first classification model using a labeled data set so that the first classification model has a certain classification capability; The evaluation unit (e.g., evaluation list agent) is used to evaluate the trained first classification model to assess the model performance; The defect detection agent is used to determine the defects of the first classification model based on the evaluation results, that is, to accurately locate the categories in which the first classification model has poor recognition performance (i.e., classification weaknesses); The data synthesis agent is used to generate synthetic data according to the defects of the first classification model; in this embodiment, the data synthesis agent includes multiple data synthesis agents, and the multiple data synthesis agents work together to generate synthetic data that is highly targeted and can effectively make up for the defects of the first classification model; The data screening agent is used to screen the generated synthetic data, remove low-quality data, and retain high-quality synthetic data; The second model training agent is used to merge high-quality synthetic data with the labeled data set, and the merged data set is used to retrain the trained first classification model to obtain the target classification model, thereby enhancing the classification ability of the first classification model in weak aspects.

[0035] The present invention uses high-quality synthetic data to expand the labeled data set, which greatly improves the generalization ability of the classification model; the synthetic data protects data privacy without affecting the model training effect, and the synthetic data is purer and free of noise, which improves the accuracy of the model.

[0036] In a specific embodiment of the present invention, the classification model training system also includes an automatic labeling subsystem and an automatic screening subsystem, the automatic labeling subsystem is used for automatic labeling and management of data, and the automatic screening subsystem is used to filter and select data according to specific conditions; wherein the automatic labeling subsystem includes a second acquisition unit, a data sampling agent, a data classification agent, a first classification verification agent, a third model training agent, a second classification verification agent, a clustering and fusion processing unit, and a label fusion agent; the automatic screening subsystem includes a third acquisition unit, multiple scoring agents, a fourth model training agent and a third classification verification agent.

[0037] The second acquisition unit is used to acquire the original data set; the data sampling agent is used to randomly or according to a certain strategy extract part of the data from the original data set to form a sampled data subset; the data classification agent is used to perform preliminary classification of the data in the sampled data subset to generate labeled data; the first classification verification agent is used to verify the preliminary classification results during the preliminary data classification process to verify the correctness of the current classification results and ensure the quality of the generated initial labeled data; the third model training agent is used to load the second classification model, train the second classification model with the initial labeled data, and update the parameters of the second classification model to enable it to have better classification capabilities; the second classification verification agent is used to verify the second classification model trained by the third model training agent, evaluate the performance of the second classification model, and feed it back to the training process for adjustment and optimization. After N times of training and verification, the second classification model achieves satisfactory performance, thereby obtaining verified data; the clustering and fusion processing unit is used to perform clustering and fusion processing on all verified data to obtain clustering results and classification of each data; the label fusion agent is used to construct a labeled data set based on each classified data.

[0038] After the above series of processes, the automatic labeling subsystem finally generates a complete and automated labeling system, covering the classification and clustering results of various types of data. The entire automatic labeling process can efficiently process a large amount of data and generate accurate labeling results through the collaboration of automation tools and agents, greatly improving the efficiency of data processing, especially suitable for the preprocessing and labeling of large-scale data sets.

[0039] The third acquisition unit is used to obtain a labeled data set from the automatic labeling subsystem, and each scoring agent is used to score the data in the labeled data set, and grade the quality or credibility of each data in the labeled data set according to the scoring results; wherein different scoring agents are configured with different scoring criteria; the fourth model training agent is used to load the third classification model, and train the third classification model using the graded data; the third classification verification agent is used to verify the third classification model trained by the fourth model training agent, to obtain high-quality labeled data, and then construct the final labeled data set based on the high-quality labeled data.

[0040] In a specific embodiment of the present invention, the automatic screening subsystem also includes a deduplication tool, a clustering tool and a retrieval tool; the deduplication tool is used to deduplicate the hierarchical data and remove duplicate data to ensure the independence and information volume of the data; the clustering tool is used to perform cluster analysis on the deduplicated data, which will help improve the consistency of the data; the retrieval tool is used to retrieve the data after cluster analysis according to specific retrieval algorithms or rules to further screen out data that meets specific conditions.

[0041] After multiple rounds of training and verification, the automatic screening subsystem finally screens out high-quality labeled data. These data have been fully verified and can be used for downstream tasks such as model training and evaluation to ensure the reliability and consistency of the data. The entire automatic screening process effectively improves the quality of the data through automated scoring, deduplication, clustering and retrieval tools, combined with the collaboration of training and verification agents, and ultimately generates high-precision, high-quality labeled data.

[0042] In a specific embodiment of the present invention, the classification model training system also includes a basic module, which includes a data distribution processing unit, a data tool API (application programming interface) management unit, a visualization management unit, a model management unit, an evaluation access unit and a high-concurrency LLM (large language model) / VLLM (very large language model) deployment unit.

[0043] The data distribution processing unit is used to distribute large-scale data to multiple nodes for parallel processing to accelerate the computing process; it can support multiple data formats and storage methods. Parallel processing improves processing efficiency, greatly shortens data processing time, and can utilize more cheap computing resources, reducing hardware costs.

[0044] The data tool API management unit is used to provide a unified interface to facilitate users to call various data processing tools, supporting multiple programming languages ​​and data formats. Users do not need to have an in-depth understanding of the underlying implementation details of each tool, they only need to call the API, which simplifies the development process. Commonly used data processing logic is encapsulated into an API for easy reuse, improving code reusability.

[0045] The Workflow visualization management unit is used to visualize complex data processing processes, and to display the dependencies and execution order between tasks in an intuitive way; it provides drag-and-drop operations to facilitate users to customize and modify workflows; it supports real-time monitoring of workflow execution status and provides abnormal alarm functions. Through the visualization interface, the development threshold is lowered, the construction and deployment of workflows are accelerated, and development efficiency is improved; clear workflow diagrams are convenient for team members to understand and maintain, reducing communication costs and enhancing maintainability; real-time monitoring and abnormal alarms help to discover and solve problems in a timely manner, and improve system stability.

[0046] The model management unit is used to manage various types of models in a unified manner, including model storage, version control, online / offline switching, etc.; it provides model evaluation and comparison functions to help users select the optimal model. The model management unit facilitates centralized management of models to avoid models being scattered in different locations; through version control, it is convenient to roll back to the previous model version, improving model reusability; through evaluation and comparison, the most suitable model for the current task is selected to optimize model selection.

[0047] The evaluation access unit is used to provide a unified interface to facilitate access to various evaluation indicators and data sets; it supports custom evaluation tasks. Through standardized evaluation methods, the model can be objectively evaluated; the performance of different models can be compared, and the optimal model can be selected to facilitate model comparison.

[0048] The high-concurrency LLM / VLLM deployment unit is used to deploy LLM or VLLM in a high-concurrency environment to meet the needs of large-scale natural language processing; it achieves high throughput and low latency by optimizing model architecture, hardware configuration, and distributed computing technologies. The high-concurrency LLM / VLLM deployment unit meets the needs of large-scale users accessing at the same time, provides real-time response, maximizes the utilization of hardware resources, reduces costs, and improves the fault tolerance and reliability of the system through distributed deployment.

[0049] By generating high-quality synthetic data, the present invention effectively solves the problems of insufficient data, data privacy, data quality, etc. in traditional data processing, provides strong support for the training of machine learning models, and has broad application prospects in the fields of medicine, finance, autonomous driving, etc.

[0050] Embodiment 3 The embodiment of the present invention further provides an electronic device, such as Figure 3 As shown, the electronic device includes: a memory, a processor, and a computer program / instructions stored in the memory, and the processor executes the computer program / instructions to implement the classification model training method in Example 1.

[0051] Although not shown, the electronic device includes a processor that can perform various appropriate operations and processes according to the programs and / or data stored in the read-only memory (ROM) or the programs and / or data loaded from the storage part into the random access memory (RAM). The processor can be a multi-core processor or can include multiple processors. In some embodiments, the processor can include a general-purpose main processor and one or more special coprocessors, such as a central processing unit, a graphics processing unit (GPU), a neural network processor (NPU), a digital signal processor (DSP), etc. In RAM, various programs and data required for device operation are also stored. The processor, ROM, and RAM are connected to each other via a bus. The input / output (I / O) interface is also connected to the bus.

[0052] The processor and memory are used together to execute the program / instructions stored in the memory. When the program / instructions are executed by the computer, the methods, steps or functions described in the above embodiments can be implemented.

[0053] Although not shown, an embodiment of the present invention further provides a computer-readable storage medium having a computer program / instruction stored thereon, which, when executed by a processor, implements the classification model training method in Embodiment 1.

[0054] Readable storage media include permanent and non-permanent, removable and non-removable media that can be implemented by any method or technology to store information. Information can be computer-readable instructions, data structures, program modules or other data. Examples of computer storage media include, but are not limited to, phase change memory (PRAM), static random access memory (SRAM), dynamic random access memory (DRAM), other types of random access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technology, compact disk read-only memory (CD-ROM), digital versatile disk (DVD) or other optical storage, magnetic cassettes, disk storage or other magnetic storage devices or any other non-transmission media that can be used to store information that can be accessed by a computing device. As defined herein, computer-readable media does not include temporary computer-readable media (transitory media), such as modulated data signals and carrier waves.

[0055] Although not shown, an embodiment of the present invention further provides a computer program product, including: a computer program / instructions, which, when executed by a processor, implement the classification model training method in Embodiment 1 of the present application.

[0056] Although preferred embodiments of the present invention have been described, additional changes and modifications may be made to these embodiments by those skilled in the art once the basic inventive concepts are known. Therefore, the appended claims are intended to be interpreted as including the preferred embodiments and all changes and modifications that fall within the scope of the present invention. Obviously, those skilled in the art may make various changes and modifications to the present invention without departing from the spirit and scope of the present invention. Thus, if these modifications and variations of the present invention fall within the scope of the claims of the present invention and their equivalents, the present invention is also intended to include these modifications and variations.

Claims

1. A classification model training method, characterized in that: The training method comprises: Get a labeled dataset; Loading a first classification model, and training the first classification model using the labeled data set; Evaluate the first classification model after training to assess model performance; Determine the defects of the first classification model according to the evaluation results; generating synthetic data according to the defects of the first classification model; Screening the synthetic data to obtain high-quality synthetic data; The high-quality synthetic data is merged with the labeled data set, and the trained first classification model is retrained using the merged data set to obtain a target classification model.

2. The classification model training method according to claim 1, characterized in that: The process of acquiring the labeled data set includes: Get the original data set; Extracting part of the data from the original data set to form a sampled data subset; Using a data classification agent to preliminarily classify the data in the sampled data subset, and using a classification verification agent to verify the preliminary classification results to generate initial labeled data; Loading a second classification model, and using the initial labeled data to train and verify the second classification model to obtain verified data; All verified data are clustered and fused to obtain clustering results and classification of each data, and a labeled data set is constructed based on the classified data.

3. The classification model training method according to claim 2, characterized in that: The process of acquiring the labeled data set also includes: Using different scoring agents to score the data in the labeled data set, and grading the quality or credibility of each data in the labeled data set according to the scoring results; wherein different scoring agents are configured with different scoring criteria; The third classification model is loaded, and the third classification model is trained and verified using the classified data to obtain high-quality labeled data, and then the final labeled data set is constructed based on the high-quality labeled data.

4. The classification model training method according to claim 3, characterized in that: Before using the classification data to train and verify the third classification model, the process of acquiring the labeled data set further includes: The classified data are deduplicated, clustered and screened, and the third classification model is trained and verified using the screened data.

5. A classification model training system, characterized in that: The training system includes a model defect training subsystem, and the model defect training subsystem includes: A first acquisition unit, used to acquire a labeled data set; A first model training agent is used to load a first classification model and train the first classification model using the labeled data set; An evaluation unit, used to evaluate the trained first classification model to assess model performance; A defect detection agent, used to determine defects of the first classification model according to the evaluation results; A data synthesis agent, used to generate synthetic data according to the defects of the first classification model; A data screening agent, used for screening the synthetic data to obtain high-quality synthetic data; The second model training agent is used to merge the high-quality synthetic data with the labeled data set, and use the merged data set to retrain the trained first classification model to obtain a target classification model.

6. The classification model training system according to claim 5, characterized in that: The classification model training system further includes an automatic marking subsystem and an automatic screening subsystem; the automatic marking subsystem includes: A second acquisition unit, used to acquire an original data set; A data sampling agent, used to extract part of the data from the original data set to form a sampled data subset; A data classification agent, used for preliminarily classifying the data in the sampled data subset; The first classification verification agent is used to verify the preliminary classification results and generate initial labeled data; A third model training agent is used to load the second classification model and train the second classification model using the initial labeled data; The second classification verification agent is used to verify the second classification model trained by the third model training agent to obtain verified data; A clustering and fusion processing unit is used to perform clustering and fusion processing on all verified data to obtain clustering results and classification of each data; Label fusion agent, used to construct a labeled dataset based on the classified data; The automatic screening subsystem comprises: A third acquisition unit, used to acquire a labeled data set from the automatic labeling subsystem; A plurality of scoring agents, each of which is used to score the data in the labeled data set, and to grade the quality or credibility of each data in the labeled data set according to the scoring result; wherein different scoring agents are configured with different scoring criteria; The fourth model training agent is used to load the third classification model and train the third classification model using the classification data; The third classification verification agent is used to verify the third classification model trained by the fourth model training agent to obtain high-quality labeled data, and then construct the final labeled data set based on the high-quality labeled data.

7. The classification model training system according to claim 5 or 6, characterized in that: The classification model training system also includes a basic module, which includes: A data distribution processing unit, used to distribute data to multiple processing nodes to achieve parallel processing; Data tool API management unit, used to provide a unified API interface to facilitate the calling of various data processing tools; Visual management unit, used to visualize the data processing flow of each subsystem; Model management unit, used to manage models, including model storage, version control, and online / offline switching; The evaluation access unit is used to provide interfaces for accessing various evaluation indicators and data sets.

8. An electronic device comprising a memory, a processor, and a computer program / instruction stored in the memory, characterized in that: The processor executes the computer program / instructions to implement the steps in the classification model training method as described in any one of claims 1 to 4.

9. A computer-readable storage medium having a computer program / instruction stored thereon, characterized in that: When the computer program / instructions are executed by a processor, the steps in the classification model training method as described in any one of claims 1 to 4 are implemented.

10. A computer program product comprising a computer program / instructions, characterized in that When the computer program / instructions are executed by a processor, the steps in the classification model training method as described in any one of claims 1 to 4 are implemented.

Images