Classification model processing method, apparatus, device, storage medium, and program product
By using an intelligent system to perform difference analysis and inductive processing, it automatically identifies and filters out incorrectly labeled data and optimizes prompt information, thus solving the problems of low iteration efficiency and insufficient accuracy in existing technologies and achieving efficient classification model iteration and improved accuracy.
Patent Information
- Application Number
- CN202611451169.8
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-09-17
- Publication Date
- 2026-11-10
AI Technical Summary
Existing classification model iteration schemes rely on the quantity and quality of labeled data, resulting in low iteration efficiency and insufficient classification accuracy, and require a large amount of manual intervention.
By using intelligent systems to perform difference analysis and summarization, the system can automatically identify and filter out incorrectly labeled data, optimize prompts, reduce manual intervention, and improve iteration efficiency and accuracy.
It enables automatic filtering of incorrectly labeled data during the classification model iteration process, reducing data quality requirements, minimizing manual intervention, and improving iteration efficiency and classification accuracy.
Smart Images

Figure CN123069082A_ABST
Abstract
Description
Technical Field
[0001] This article relates to the field of information processing technology, and in particular to a classification model processing method, apparatus, device, storage medium, and program product. Background Technology
[0002] With the rapid development of model technology, classification models need to continuously adapt to ever-changing business requirements in application scenarios such as content compliance verification, dialogue request recognition, and multimodal content classification. To reduce computational overhead and shorten the iteration cycle of classification models, a prompt-driven classification model iteration scheme has emerged. This scheme first manually labels the data, and then uses the labeled data to iteratively optimize the prompts of the classification model. However, related classification model iteration schemes rely heavily on the quantity and quality of labeled data, and have shortcomings in terms of classification model quality and iteration efficiency. Summary of the Invention
[0003] To address the aforementioned technical issues, this paper provides a classification model processing method, apparatus, device, storage medium, and program product.
[0004] Firstly, this paper provides a classification model processing method, which includes: Obtain a first data set and a first prompt message; wherein, the first data set includes multiple first data items; the first data items are associated with a first category; Based on the first prompt information, the first data is classified using a classification model to obtain a second category; Based on the second category and the first category, a difference analysis is performed using a first intelligent system to obtain the causes of the difference; wherein, the causes of the difference include the causes of model bias; Based on multiple sets of first data and the corresponding causes of difference, a second intelligent system is used for inductive processing to obtain an inductive result; wherein, the inductive result includes a second data set and a deviation pattern; the second data set is a subset of the first data set; the second data set includes multiple sets of second data; the second data is associated with the causes of model deviation; the deviation pattern characterizes the common features that cause classification deviation in the classification model; Based on the second data set and the deviation pattern, the first prompt information is updated to obtain the second prompt information.
[0005] Secondly, this paper also provides a classification model processing device, which includes: The data acquisition module is used to acquire a first data set and a first prompt message; wherein, the first data set includes multiple first data items; the first data items are associated with a first category; The classification module is used to classify the first data based on the first prompt information using a classification model to obtain a second category; The difference analysis module is used to perform difference analysis based on the second category and the first category using the first intelligent system to obtain the causes of the difference; wherein, the causes of the difference include the causes of model bias; An inductive processing module is used to perform inductive processing using a second intelligent system based on multiple first data points and the corresponding causes of difference in the first data points to obtain an inductive result; wherein, the inductive result includes a second data set and a deviation pattern; the second data set is a subset of the first data set; the second data set includes multiple second data points; the second data points are associated with the causes of model deviation; the deviation pattern characterizes the common features that cause classification deviation in the classification model; The prompt information update module is used to update the first prompt information based on the second data set and the deviation pattern to obtain the second prompt information.
[0006] Thirdly, this article also provides an electronic device, which includes: A storage device on which computer programs are stored; A processing device is configured to execute the computer program in the storage device to implement the classification model processing method described in any embodiment herein.
[0007] Fourthly, this document also provides a computer-readable storage medium storing a computer program that, when executed by a processor, implements the classification model processing method described in any embodiment herein.
[0008] Fifthly, this document also provides a computer program product, which includes a computer program or instructions. When a processor executes the computer program or instructions, it implements the classification model processing method described in any embodiment of this document.
[0009] This paper presents a classification model processing method, apparatus, device, storage medium, and program product. The method involves acquiring a first data set including multiple first data points and corresponding first categories, along with first prompt information. Based on the first prompt information, a classification model is used to classify the first data points to obtain a second category. Based on the second category and the first category, a first intelligent system is used to perform difference analysis to obtain the causes of difference, including the causes of model bias. Based on the multiple first data points and their corresponding causes of difference, a second intelligent system is used to perform inductive processing to obtain an inductive result. The inductive result includes a second data set and a bias pattern. The second data set is a subset of the first data set. The second data set includes multiple second data points. The second data points are associated with the causes of model bias. The bias pattern represents the common characteristics that cause classification bias in the classification model. Based on the second data set and the bias pattern, the first prompt information is updated to obtain a second prompt information. This method achieves difference analysis on the second category predicted by the classification model and its reference first category, and inductive processing of the causes of difference to filter out the second data set corresponding to the causes of model bias from the first data set. The second data set and the summarized bias pattern are then used to update the prompt information to obtain the second prompt information. In this way, the process of optimizing the prompt information can automatically filter out the labeled erroneous data (i.e. dirty data) in the first dataset. This eliminates the need for manual identification and processing of dirty data beforehand, and reduces the possibility of dirty data participating in the prompt information iteration. As a result, the quality requirements of the first dataset are reduced, the manual intervention is reduced, the iteration efficiency of the prompt information and classification model is improved, and the classification accuracy of the classification model is improved.
[0010] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this article are all information and data authorized by the user or fully authorized by all parties. Furthermore, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse. Attached Figure Description
[0011] The above and other features, advantages, and aspects of the embodiments herein will become more apparent when taken in conjunction with the accompanying drawings and the following detailed description. Throughout the drawings, the same or similar reference numerals denote the same or similar elements. It should be understood that the drawings are schematic, and elements are not necessarily drawn to scale.
[0012] Figure 1 This is a schematic diagram of the scenario architecture applicable to the classification model processing method provided in this paper; Figure 2 A flowchart illustrating a classification model processing method provided in this paper; Figure 3 A flowchart illustrating another classification model processing method provided in this paper; Figure 4 This paper provides a schematic diagram of the structure of a classification model processing device. Figure 5 This is a schematic diagram of the structure of an electronic device provided in this article. Detailed Implementation
[0013] The embodiments described herein will now be described in more detail with reference to the accompanying drawings. While some embodiments are shown in the drawings, it should be understood that this document can be implemented in various forms and should not be construed as limited to the embodiments set forth herein; rather, these embodiments are provided to provide a more thorough and complete understanding of the document. It should be understood that the accompanying drawings and embodiments are for illustrative purposes only and are not intended to limit the scope of protection of this document.
[0014] It should be understood that the steps described in the method embodiments herein may be performed in different orders and / or in parallel. Furthermore, the method embodiments may include additional steps and / or omit the steps shown. The scope of this document is not limited in this respect.
[0015] The term "comprising" and its variations as used herein are open-ended, meaning "including but not limited to". The term "based on" means "at least partially based on". The concepts of "first", "second", etc., used herein are used only to distinguish different devices, modules, or units and are not intended to limit the order of functions performed by these devices, modules, or units or their interdependencies. The modifiers "a" and "a plurality" used herein are illustrative and not restrictive, and those skilled in the art should understand that, unless explicitly stated in the context, they should be understood as "one or more". Definitions of other terms will be given in the following description.
[0016] The names of messages or information exchanged between multiple devices in this document are for illustrative purposes only and are not intended to limit the scope of these messages or information.
[0017] In related technologies, the main iterative approach for prompt-driven classification models is as follows: First, manual labeling of the data to be classified according to business requirements constructs a training dataset. Then, using the training data, the model parameters are frozen, and the prompts are iteratively optimized until the iteration termination condition is met. Finally, relevant personnel evaluate the classification model's performance. However, this iterative approach involves three stages: data labeling, prompt iteration, and model performance evaluation. Information transfer between these stages relies on manual communication, resulting in low overall iteration efficiency for the classification model. Furthermore, differences in understanding of business requirements among multiple individuals can lead to dirty data in the labeled training data. Identifying and processing dirty data in this case incurs additional iteration time; failing to process dirty data affects the accuracy of the prompts, ultimately impacting the quality of the classification model delivery. Therefore, the classification model iteration approach in related technologies involves multiple stages, high levels of manual involvement, and dependence on the quantity and quality of labeled data, resulting in shortcomings in terms of classification model quality and iteration efficiency.
[0018] Based on the above, this paper provides a technical solution for processing classification models. During the iterative prompting process, the high processing power of the intelligent system is used to construct a processing flow of difference analysis, difference attribution, classification pattern summarization, data grouping, and prompting information iteration. This automatically identifies and filters dirty data, obtains a second data set and deviation patterns, and uses them to iterate prompting information. This reduces the quality requirements of labeled data, reduces human intervention, and improves iteration efficiency and classification accuracy of the classification model.
[0019] The classification model processing method presented in this paper is mainly applicable to scenarios that optimize prompt information, such as prompt information optimization scenarios of prompt information-driven classification models.
[0020] In one situation, Figure 1 This diagram illustrates the scenario architecture of a classification model processing method presented in this paper. The execution of the classification model processing method involves human-computer interaction between the user and the classification model processing device. This classification model processing method can be executed by the classification model processing device, which can be implemented in software and / or hardware. Figure 1 As shown, when the device is implemented as hardware, it can be various electronic devices with certain data processing capabilities, including but not limited to laptops, desktop computers, servers, or server clusters. When the device is implemented as software, it can be integrated into or installed in the electronic devices listed above. It can be implemented as multiple software programs or software modules, or as a single software program or software module. No specific limitations are made here.
[0021] Figure 2A flowchart illustrating a classification model processing method presented in this paper is shown. Figure 2 As shown, the classification model processing method may include the following steps: Step 210: Obtain the first data set and the first prompt information.
[0022] The first dataset is a pre-collected training dataset. It includes multiple first datasets, each associated with a first category. The first dataset is the data to be classified, labeled with categories; it can be data in various modalities such as text, images, audio, and video. The first category is the labeled classification category. The first prompt is the prompt obtained from the previous iteration. If this is the first iteration, the first prompt can be blank or a pre-set prompt. The prompt refers to the guiding instruction text input to the model, used to clarify the task objective, define output rules, specify content boundaries, and drive the model to generate corresponding results according to expected logic. In this paper, the prompt is the instruction information input to the classification model, carrying classification rules, used to guide the classification model to perform category determination on the input data and output the corresponding classification result. The classification model is a model with classification functionality; this model can be a deep learning model with a large parameter scale.
[0023] Specifically, the electronic device can respond to the request to initiate the prompt message optimization process by reading the first data set and the first prompt message from local / external storage media, or it can receive the first data set and the first prompt message from a client or upstream business terminal. Since the model parameters of the classification model are fixed and the classification function is limited by the prompt message rules, model capability iteration does not rely on large-scale training data. Therefore, the first data set only needs to contain a small amount (e.g., several hundred) of first data points to complete the prompt message optimization iteration, reducing the demand for first data.
[0024] Step 220: Based on the first prompt information, use a classification model to classify the first data to obtain the second category.
[0025] The second category is the classification category output by the classification model.
[0026] Specifically, the electronic device can input the first prompt information and each first data point into the classification model. The classification model predicts the category to which the first data point belongs and outputs the second category corresponding to the first data point.
[0027] In one scenario, step 220 includes: based on the first prompt information, using a first intelligent system to invoke a classification model to classify the first data and obtain a second category. In this scenario, the electronic device inputs each piece of first data and the first prompt information into the first intelligent system, which then schedules the classification model to perform classification processing as needed to obtain the second category for each piece of first data.
[0028] The first intelligent system refers to an intelligent system with functions such as difference analysis and difference attribution. The first intelligent system configures the classification criteria corresponding to the classification task. These criteria may include at least one of the following: pre-set classification judgment standards, classification-related experiential knowledge, and historical classification cases. In one scenario, the intelligent system is an information processing system. The intelligent system uses at least one or more models to process one or more inputs. The intelligent system may include one or more models as components, or it may use models external to the system for processing. Those skilled in the art will recognize that there are various implementations of intelligent systems in the industry, including but not limited to agents, AI assistants, AI workers, copilot systems, and chatbots. Inputs include data in one or more modalities. Modal data includes, but is not limited to, text, audio, images, video, application-specific data, or scientific computing data. Models include, but are not limited to, one or more combinations of the following: large-scale parametric models, deep learning models, neural network models, machine learning models, statistical models, rule-based models, or other models capable of processing inputs. The specific implementation of the model can employ various existing or improved models. Intelligent systems may involve one or more of the following operations: receiving single or multiple rounds of input; recognizing and / or understanding the input; generating answers, decisions, or processing results using one or more models; invoking one or more external tools, interfaces, knowledge bases, or other systems to obtain information or perform operations; processing tasks autonomously, through human-machine collaboration, or by delegating tasks to humans or other systems. Autonomous decision-making refers to execution without human intervention. Human-machine collaboration refers to partial human involvement or confirmation. The above explanation of intelligent systems is illustrative, and all descriptions are examples, not all of which are necessary, nor do they constitute an exhaustive list.
[0029] Step 230: Based on the second category and the first category, perform difference analysis using the first intelligent system to obtain the causes of the differences.
[0030] Here, the causes of discrepancy refer to the reasons for the classification bias between the second category and the first category. These causes include model bias, which refers to the fact that the aforementioned classification bias arises because the classification model outputs an incorrect classification category.
[0031] Specifically, the electronic device inputs the first data and its corresponding first and second categories into the first intelligent system. The first intelligent system analyzes whether there are differences / consistencies between the paired first and second categories, obtaining a set of data with category differences. This set of data includes the first data, the first category, and the second category. Then, the first intelligent system can use its configured classification criteria and data processing functions to perform difference attribution analysis on each set of data to determine whether the difference is caused by an error in the classification model (i.e., model bias) or by an error in the labeling of the first category (i.e., labeling bias), thus obtaining the cause of the difference in that set of data.
[0032] Step 240: Based on multiple first data and the causes of differences corresponding to the first data, the second intelligent system is used for inductive processing to obtain the inductive result; the inductive result includes the second data set and the deviation pattern.
[0033] The second intelligent system is an intelligent system capable of summarizing data and inductively identifying systematic patterns. Inductive processing refers to performing clustering, verification, and summarizing common characteristics of classification biases on multiple primary data sets and their causes of difference. The inductive result is the output of the second intelligent system, which may include at least one of the following: difference type, corresponding data subset, inductive classification pattern, and subsequent processing nodes. Difference type refers to the category to which the bias causing the category difference belongs, determined by summarizing multiple causes of difference; it may include model error type, labeling error type, and labeling doubt type. Classification pattern refers to the common judgment rules / common characteristics of data classification, which is the classification decision basis learned by the model from the data. Subsequent processing nodes may be nodes that optimize prompt information, nodes that automatically correct categories, or verification nodes. The verification node is the execution node used to verify and correct the primary category; for example, it may be a node corresponding to the intelligent system or a manual node. The second data set is a subset of the first data set, including multiple secondary data sets. The secondary data sets are the primary data sets associated with the causes of model biases. That is, the second category of each second data point in the second dataset is due to a classification model error, while its corresponding first category is reliably labeled. The bias pattern characterizes the common features that cause classification bias in the classification model.
[0034] Specifically, the electronic device inputs the aforementioned sets of data with discrepancies (first data, first category, and second category) and their corresponding causes of difference into the second intelligent system. The second intelligent system can then aggregate these data with category differences for global analysis such as clustering and verification, and summarize the common characteristics of these data to obtain the summary results. For example, if the second intelligent system analyzes multiple sets of first data based on classification criteria and finds that the labeled first category is stable and reliable, while the second category output by the classification model consistently makes prediction errors in similar scenarios, then the corresponding difference type can be determined as a model error type. A second data set is constructed from these first data sets, and common characteristics of classification errors / biases can be extracted from the second data set to form corresponding bias patterns. Furthermore, the subsequent processing nodes can be determined as nodes for executing prompt information optimization.
[0035] Step 250: Based on the second data set and the deviation pattern, update the first prompt information and obtain the second prompt information.
[0036] The second prompt message is the new prompt message after the update / optimization.
[0037] Specifically, the electronic device can transfer the second dataset and its corresponding bias patterns to the prompt information optimization node. In this way, according to a pre-set prompt information optimization method, using the second dataset and its corresponding bias patterns as input, at least one candidate prompt information can be obtained. The second dataset is then used to evaluate whether the candidate prompt information is overfitting, and whether its classification accuracy / recall on the second dataset meets specified threshold values. If the evaluation passes, the corresponding candidate prompt information can be used as the second prompt information.
[0038] In one scenario, the method further includes: using the second prompt information from the previous round as the updated first prompt information; and, based on the updated first prompt information and the first data set, sequentially re-executing classification processing, difference analysis, inductive processing, and prompt information update to obtain the second prompt information, until the first data set satisfies the first condition.
[0039] The first condition is a pre-set condition used to determine whether the prompt information meets the business exit requirements on the first data set. For example, the first condition may include the accuracy or recall rate of the first data set reaching a preset indicator threshold (such as a preset accuracy or a preset recall rate); or the first condition may include the accuracy / recall rate of the first data set reaching a preset indicator threshold and the absence of overfitting.
[0040] Specifically, after the electronic device receives the second prompt information, it can reuse it as the first prompt information, and then repeatedly execute steps 220 to 250 until the first data set meets the first condition. At this point, the second prompt information is the final output prompt information corresponding to the first data set. In this way, dirty data can be automatically filtered out in each iteration, and only the second data set corresponding to the causes of model bias can be used to optimize the prompt information. This achieves closed-loop automated iteration of the prompt information, continuously corrects the classification rules, reduces manual intervention, and steadily improves the ability of the prompt information to adapt to the training data of the classification business.
[0041] This paper's classification model processing method involves: acquiring a first data set including multiple first data points and corresponding first categories, along with first prompt information; classifying the first data points using a classification model based on the first prompt information to obtain a second category; performing difference analysis using a first intelligent system based on the second category and the first category to obtain the causes of difference, including model bias; performing inductive processing using a second intelligent system based on multiple first data points and their corresponding causes of difference to obtain an inductive result; the inductive result includes a second data set and a bias pattern; the second data set is a subset of the first data set; the second data set includes multiple second data points; the second data points are associated with the causes of model bias; the bias pattern represents the common characteristics of classification bias generated by the classification model; and updating the first prompt information based on the second data set and the bias pattern to obtain the second prompt information. This method achieves difference analysis on the second category predicted by the classification model and its reference first category, and inductive processing of the causes of difference to filter out the second data set corresponding to the causes of model bias from the first data set, and updates the prompt information using the second data set and the summarized bias pattern to obtain the second prompt information. In this way, the process of optimizing the prompt information can automatically filter out the labeled erroneous data (i.e. dirty data) in the first dataset. This eliminates the need for manual identification and processing of dirty data beforehand, and reduces the possibility of dirty data participating in the prompt information iteration. As a result, the quality requirements of the first dataset are reduced, the manual intervention is reduced, the iteration efficiency of the prompt information and classification model is improved, and the classification accuracy of the classification model is improved.
[0042] In one scenario, the causes of discrepancies also include labeling bias. Accordingly, the inductive results also include a third dataset and multiple third categories.
[0043] The third dataset is a subset of the first dataset and includes multiple third datasets. Each third dataset has a corresponding third category. The third category is a corrected classification category, which may be the correct classification category automatically determined by the second intelligent system. The third dataset is associated with the cause of labeling bias, meaning that the third dataset is the first dataset that is incorrectly labeled, as determined by the second intelligent system's global analysis. Alternatively, the third category corresponding to the third dataset is matched with the classification criteria by the second intelligent system. That is, the cause of the difference corresponding to the third dataset may be a labeling bias or a model bias. As long as the second intelligent system can determine the classification category for it based on the classification criteria (the classification criteria are clear and reliable), the third dataset belongs to the labeling error type.
[0044] Specifically, referring to step 240 above, the inductive result obtained by the second intelligent system after inductive processing may also include a third data set and multiple third categories. In one scenario, if the second intelligent system analyzes multiple sets of first data based on classification criteria and finds that the second category output by the classification model is stable and reliable, while the labeled first category consistently shows labeling bias in similar scenarios, then the corresponding difference type can be determined as a labeling error type. Simultaneously, the classification category opposite to the second category is determined as the third category of the corresponding first data (e.g., the second category is the complaint category, and the third category is the non-complaint category). In another scenario, if the second intelligent system analyzes multiple sets of first data in the same scenario and finds conflicts in labeled first categories or contradictions in predicted second categories, but the second intelligent system can determine the classification category for these first data based on classification criteria, then the difference type corresponding to these first data can also be determined as a labeling error type. Simultaneously, the classification category determined by the second intelligent system is taken as the third category of the corresponding first data. A third data set is constructed from the first data in these two scenarios, and the third category of each third data set can be obtained.
[0045] Since all the third data in the third dataset determined by the global analysis of batch data are labeled incorrectly, the second intelligent system does not need to classify the third dataset into patterns. Furthermore, the second intelligent system can determine its subsequent processing nodes as either automatic category correction nodes or verification nodes. These subsequent nodes can correct the third dataset containing dirty data. In this way, the second intelligent system can automatically identify incorrectly labeled samples in the first dataset and provide reliable reference categories, providing a data foundation for correcting dirty data; it also reduces the need for manual identification and correction of dirty data, further improving the efficiency of iterative prompts.
[0046] In one scenario, the method further includes: replacing the first category corresponding to the third data based on the third category, and updating the first data set.
[0047] Specifically, in this scenario, the electronic device transfers the third data set and each third category to the node that performs automatic category correction, replacing the first category corresponding to each third data with the third category. This automatically corrects the classification of incorrectly labeled third data, reduces the need for manual data cleaning, quickly obtains reliable sample data, improves data utilization, and provides high-quality foundational data for subsequent iterations of prompt information.
[0048] In one scenario, the induction result also includes a fourth data set. This fourth data set is a subset of the first data set and includes multiple fourth data points. These fourth data points are also part of the first data set, but they lack a corresponding third category. This means the second intelligent system cannot determine the category of the fourth data point based on the classification criteria, as these criteria are not applicable. At least one fourth data point from the fourth data set is transmitted to a verification node for verification and correction of its corresponding first category. In this scenario, during the analysis of various attributions and their corresponding data, the second intelligent system identifies unstable classification categories for multiple first data points. For example, multiple first data points in the same scenario may have different classification categories or fail to obtain a stable classification category. These first data points constitute the fourth data set, and those with ambiguous classification boundaries are labeled as suspicious types, thus identifying the verification node as the subsequent processing node. In this way, difficult-to-determine samples can be automatically identified and isolated during the prompt information optimization process, preventing them from interfering with the optimization process and further improving the iteration efficiency of the prompt information.
[0049] In one scenario, the method further includes: receiving a fourth category; replacing the first category with the fourth category and updating the first data set.
[0050] The fourth category corresponds to the fourth data point. The fourth category is the classification category reported by the verification node.
[0051] Specifically, the electronic device can transfer at least one fourth data point from the fourth data set to a verification node, such as a human node. Relevant personnel verify and correct the first category corresponding to the fourth data point. In response to this verification and correction, the electronic device obtains its corresponding labeled fourth category. Then, the fourth category replaces the first category corresponding to the fourth data point in the first data set, serving as a classification case for the aforementioned difficult-to-determine situation, thus updating the first data set. In this way, by manually labeling the fourth data with ambiguous boundaries with reliable categories, classification cases for difficult-to-determine scenarios are supplemented, enriching the first data set. This facilitates subsequent iterations in mining the classification rules for these cases, automatically processing similar data, improving the classification rules carried by the prompt information, and further enhancing the accuracy of the prompt information.
[0052] In one scenario, the method further includes: using the second prompt information from the previous round as the updated first prompt information; and, based on the updated first prompt information and the updated first data set, sequentially re-executing the classification process, difference analysis, inductive process, and prompt information update to obtain the second prompt information.
[0053] Specifically, after updating the first data set, the electronic device uses the second prompt information obtained in the previous round as the first prompt information again, and then repeats steps 220 to 250 until the first data set meets the first condition. At this point, the second prompt information is the final output prompt information corresponding to the first data set. In this way, a full iteration can be performed again based on the data with corrected classification categories and / or newly added classification cases to fully explore the classification patterns of various scenarios, further improve the classification rules within the prompt information, and enhance the overall classification effect.
[0054] Figure 3 A flowchart illustrating another classification model processing method provided in this paper is shown. This method, based on the embodiments described above, adds a generalization verification process. For example... Figure 3 As shown, the classification model processing method may include the following steps: Step 301: Obtain the first data set and the first prompt information.
[0055] Step 302: Based on the first prompt information, use a classification model to classify the first data to obtain the second category.
[0056] Step 303: Based on the second category and the first category, perform difference analysis using the first intelligent system to obtain the causes of the differences.
[0057] Step 304: Based on multiple causes of difference and the first data corresponding to the causes of difference, use the second intelligent system to perform inductive processing to obtain the inductive result.
[0058] Step 305: Based on the second data set and the deviation pattern, update the first prompt information to obtain the second prompt information. Subsequent iterations can be performed based on the second prompt information and the first data set until the first data set satisfies the first condition.
[0059] Step 306: Based on the third category, replace the first category and update the first data set. Subsequent iterations can be performed based on the second hint information from the previous round and the updated first data set until the first data set satisfies the first condition.
[0060] Step 307: Receive the fourth category, and based on the fourth category, replace the first category and update the first data set. Subsequent iterations can be performed based on the second prompt information from the previous round and the updated first data set until the first data set satisfies the first condition.
[0061] Step 308: Based on the fifth data set and the third prompt information, the first intelligent system is used to perform verification processing to obtain the verification result.
[0062] The fifth dataset includes multiple sets of fifth data. This fifth data consists of unlabeled, unclassified data obtained from the online business platform. The third prompt is the second prompt at the end of the loop. The end of the loop here corresponds to the first dataset satisfying the first condition. That is, the third prompt is the second prompt output from the last iteration of the prompts in the preceding steps. The validation process examines the classification adaptability of the third prompt to online data outside the first dataset, assessing whether the third prompt can adapt to various classification scenarios. The validation result is obtained from generalization validation, which may include whether new classification patterns emerge, whether overfitting exists, and whether the evaluation metrics (accuracy, recall, etc.) corresponding to the fifth dataset reach preset thresholds.
[0063] Specifically, electronic devices can obtain multiple real online fifth data points from online business terminals, forming a fifth data set. Simultaneously, they can obtain the aforementioned third prompt information. Then, the fifth data set and the third prompt information are input into a first intelligent system. The first intelligent system, combining classification criteria and considering dimensions such as classification mode, overfitting, and evaluation indicators, analyzes each fifth data point, the third prompt information, and the category (i.e., the fifth category) obtained by the classification model for the fifth data point, as well as the first data set, to obtain verification results.
[0064] In one scenario, step 308 includes: obtaining a fifth data set and a third prompt; classifying the fifth data set based on the third prompt using a classification model to obtain a fifth category; and performing generalization verification using a first intelligent system based on the fifth data set and the fifth category to obtain a verification result.
[0065] Specifically, the electronic device can input the obtained third prompt information and each fifth data point into a classification model. The classification model predicts the category to which the fifth data point belongs and outputs the fifth category corresponding to each fifth data point. Then, the electronic device inputs each fifth data point and its corresponding fifth category into a first intelligent system for generalization verification to obtain the verification results. For example, the first intelligent system can summarize and analyze the fifth data points and the fifth categories to obtain one or more corresponding category features, and compare the similarity of these category features with the category features corresponding to the first data set to determine whether the fifth data set contains new category features not included in the first data set. New category features can be used to summarize new classification patterns. Alternatively, the first intelligent system can classify the fifth data points based on classification criteria to obtain a reference category, and then use it to determine whether the fifth category is correct. It can also statistically analyze the percentage of correctly classified data and the distribution of the classification scenarios to which the correctly classified data belongs to, in order to determine whether the third prompt information is overfitting. Furthermore, the first intelligent system can estimate the precision and recall of the fifth data set based on the fifth category and reference category of each fifth data point, and combine this with indicator thresholds to determine whether the evaluation indicators of the fifth data set have reached the indicator thresholds.
[0066] Step 309: In response to the verification result satisfying the second condition, based on the verification result and the fifth data set, obtain the sixth data and the sixth category to form the sixth data set.
[0067] The second condition is a pre-set condition used to determine whether the generalization verification passes. The second condition may include: the fifth dataset contains category features not covered by the first dataset, and these category features are matched to a category by the first intelligent system and the classification criteria. That is, a new classification pattern appears in the fifth dataset, and the first intelligent system cannot determine the classification category of the newly appearing category feature based on the classification criteria. The second condition may also include: the fifth category corresponding to the fifth data deviates from the classification criteria corresponding to the third prompt information, i.e., overfitting occurs, causing the classification criteria to not fit the fifth dataset well. The second condition may also include: the evaluation index of the fifth dataset not reaching the index threshold. The second condition may also include a combination of two or three of the above conditions. The sixth category corresponds to the sixth dataset.
[0068] Specifically, when the electronic device detects that the verification result meets the second condition, it extracts at least one fifth piece of data related to the verification result from the fifth data set as the sixth piece of data; that is, it extracts at least one sixth piece of data from the fifth data set based on the verification result. Then, through a human node or the first intelligent system, the corresponding classification category is determined for the sixth piece of data, which is designated as the sixth category. These sixth pieces of data and their corresponding sixth categories constitute the sixth data set. This sixth data set contains online data that failed the verification of the third prompt information and can characterize the classification rules corresponding to these online data.
[0069] Step 310: Use the third prompt information as the updated first prompt information, and update the first data set based on the sixth data set.
[0070] Specifically, if the verification result meets the second condition, it indicates that the generalization verification has failed, and the generalization defect can be eliminated by adjusting the prompt information. Therefore, the electronic device reverts the third prompt information as the first prompt information and updates the first data set using the sixth data set, for example, by adding the sixth data set to the first data set; or by incorporating the sixth data set into the first data set, etc.
[0071] Step 311: Based on the updated first prompt information and the updated first data set, re-execute the classification process, difference analysis, inductive process and prompt information update in sequence to obtain the second prompt information.
[0072] Specifically, the electronic device can update the first prompt message and the first data set, then return to steps 301-307 to continue optimizing the prompt message until the updated first data set meets the first condition. It then re-enters steps 308-311 until the verification result meets the third condition, outputting the final round of third prompt message as the exit prompt message for the classification model adaptation. Here, the third condition is the condition for the classification model to undergo final manual verification / deployment / delivery, and it is mutually exclusive with the second condition.
[0073] In one scenario, the method further includes: if the verification result indicates that the classification criteria cannot be adapted to the new classification pattern, the process can be handed over to a verification node, such as a human node. After the new classification criteria or the classification category of the fifth data corresponding to the new classification pattern is manually confirmed, the electronic device updates the prompt information or the first data set and returns to the aforementioned steps until the third prompt information is output.
[0074] The classification model processing method described above enables generalization verification based on the fifth dataset and the third prompt information. It tests the classification adaptability of the prompt information in online unlabeled data scenarios, overcoming the limitations of scenarios where training samples are not covered. When uncovered category features are detected, evaluation indicators are not met, or categories deviate from the classification criteria, the verification results are used to automatically filter valid samples, update the dataset, and iterate the prompt information again. This automatically identifies and adapts to the distribution of newly added online samples, reducing the workload of manual labeling and rule adjustment for online data. It further improves the automation level and classification accuracy of prompt information iteration and enhances the generalization performance of the classification model.
[0075] The following are embodiments of the classification model processing apparatus provided herein. This apparatus and the classification model processing methods of the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the classification model processing apparatus, please refer to the embodiments of the above classification model processing methods.
[0076] Figure 4 A schematic diagram of the structure of a classification model processing device provided in this paper is shown. Figure 4 As shown, the classification model processing device 400 may include: The data acquisition module 410 is used to acquire a first data set and a first prompt message; wherein, the first data set includes multiple first data items; the first data items are associated with a first category; The classification module 420 is used to classify the first data based on the first prompt information using a classification model to obtain a second category; The difference analysis module 430 is used to perform difference analysis based on the second category and the first category using the first intelligent system to obtain the causes of the difference; wherein, the causes of the difference include the causes of model bias. The inductive processing module 440 is used to perform inductive processing using a second intelligent system based on multiple first data and the corresponding causes of differences in the first data to obtain inductive results; wherein, the inductive results include a second data set and a deviation pattern; the second data set is a subset of the first data set; the second data set includes multiple second data; the second data is associated with the deviation causes of the model; the deviation pattern represents the common characteristics of the classification model that produce classification deviations; The prompt message update module 450 is used to update the first prompt message and obtain the second prompt message based on the second data set and the deviation pattern.
[0077] The classification model processing device described in this paper acquires a first data set including multiple first data points and corresponding first categories, and a first prompt message; based on the first prompt message, it uses a classification model to classify the first data points to obtain a second category; based on the second category and the first category, it uses a first intelligent system to perform difference analysis to obtain the difference causes, including the causes of model bias; based on the multiple first data points and the difference causes corresponding to the first data points, it uses a second intelligent system to perform inductive processing to obtain an inductive result; the inductive result includes a second data set and a bias pattern; the second data set is a subset of the first data set; the second data set includes multiple second data points; the second data points are associated with the model bias causes; the bias pattern characterizes the common features that cause classification bias in the classification model; based on the second data set and the bias pattern, it updates the first prompt message to obtain a second prompt message. This achieves difference analysis on the second category predicted by the classification model and its reference first category, and inductive processing of the difference causes to filter out the second data set corresponding to the model bias causes from the first data set, and updates the prompt message using the second data set and the summarized bias pattern to obtain the second prompt message. In this way, the process of optimizing the prompt information can automatically filter out the labeled erroneous data (i.e. dirty data) in the first dataset. This eliminates the need for manual identification and processing of dirty data beforehand, and reduces the possibility of dirty data participating in the prompt information iteration. As a result, the quality requirements of the first dataset are reduced, the manual intervention is reduced, the iteration efficiency of the prompt information and classification model is improved, and the classification accuracy of the classification model is improved.
[0078] In one case, the causes of the difference also include the causes of labeling bias; The induction results also include a third data set and multiple third categories; wherein, the third data set is a subset of the first data set; the third data set includes multiple third data; the third data has a corresponding third category; the third data is associated with the cause of the bias labeling, or the third category corresponding to the third data is matched with the classification criteria by the second intelligent system.
[0079] In one scenario, the classification model processing device 400 further includes a first data set update module, used for: Based on the third category, replace the first category and update the first data set; where the first category corresponds to the third data.
[0080] In one scenario, the induction result also includes a fourth data set; wherein the fourth data set is a subset of the first data set; the fourth data set includes multiple fourth data sets; the fourth data sets do not have a corresponding third category; and at least one fourth data set is transmitted to the verification node.
[0081] In one scenario, the first data set update module is also used for: Receive the fourth category; where the fourth category corresponds to the fourth data, and the fourth category is the classification category returned by the verification node; Based on the fourth category, replace the first category and update the first data set; where the first category corresponds to the fourth data.
[0082] In one scenario, the classification model processing device 400 further includes a loop triggering module, used for: Use the second prompt message from the previous round as the updated first prompt message; Based on the updated first prompt information and the updated first data set, the classification process, difference analysis, inductive process, and prompt information update are re-executed in sequence to obtain the second prompt information.
[0083] In one scenario, the loop triggering module is also used for: Use the second prompt message from the previous round as the updated first prompt message; Based on the updated first prompt information and the first data set, the classification process, difference analysis, inductive process and prompt information update are re-executed in sequence to obtain the second prompt information, until the first data set meets the first condition.
[0084] In one scenario, the classification model processing device 400 further includes a generalization verification module, used for: Based on the fifth data set and the third prompt information, the first intelligent system performs verification processing to obtain the verification result; wherein, the fifth data set includes multiple fifth data; the third prompt information is the second prompt information for the end of the loop; the end of the loop corresponds to the first data set satisfying the first condition; In response to the verification result satisfying the second condition, based on the verification result and the fifth data set, the sixth data and the sixth category corresponding to the sixth data are obtained to form the sixth data set; wherein, the second condition includes at least one of the following: the fifth data set contains category features not covered by the first data set, and the category features are matched to the classification category by the first intelligent system and the classification basis; the evaluation index of the fifth data set does not reach the index threshold; the fifth category corresponding to the fifth data deviates from the classification basis, and the classification basis corresponds to the third prompt information; The third prompt message is used as the updated first prompt message, and the first data set is updated based on the sixth data set; Based on the updated first prompt information and the updated first data set, the classification process, difference analysis, inductive process, and prompt information update are re-executed in sequence to obtain the second prompt information.
[0085] The classification model processing device provided in this paper can execute the classification model processing method provided in this paper, and has the corresponding functional modules and beneficial effects of the method.
[0086] It is worth noting that in the embodiments of the above-mentioned classification model processing device, the modules included are divided according to functional logic, but are not limited to the above division, as long as the corresponding functions can be achieved; in addition, the specific names of each functional module are only for easy differentiation and are not used to limit the scope of protection of this document.
[0087] This document also provides an electronic device, which may include a storage device storing a computer program thereon; and a processing device for executing the computer program in the storage device to implement the classification model processing method described in any embodiment of this document, and having the corresponding functional modules and beneficial effects of the method execution.
[0088] Figure 5 A schematic diagram of the structure of an electronic device provided in this article is shown.
[0089] like Figure 5 As shown, the electronic device 500 may include a processing unit 501 (e.g., an arithmetic processor, a graphics processor, etc.), which can perform various appropriate actions and processes according to a program stored in read-only memory (ROM) 502 or a program loaded from storage device 508 into random access memory (RAM) 503. The RAM 503 also stores various programs and data required for the operation of the electronic device 500. The processing unit 501, ROM 502, and RAM 503 are interconnected via a bus 504. An input / output interface (I / O interface) 505 is also connected to the bus 504.
[0090] Typically, the following devices can be connected to I / O interface 505: input devices 506 including, for example, touch screens, touchpads, keyboards, mice, cameras, microphones, accelerometers, gyroscopes, etc.; output devices 507 including, for example, liquid crystal displays (LCDs), speakers, vibrators, etc.; storage devices 508 including, for example, magnetic tapes, hard disks, etc.; and communication devices 509. Communication device 509 allows electronic device 500 to communicate wirelessly or wiredly with other devices to exchange data.
[0091] It should be noted that, Figure 5 The electronic device 500 shown is merely an example and should not impose any limitations on the functionality or scope of this document. That is, although... Figure 5 An electronic device 500 with various devices is shown; however, it should be understood that it is not required to implement or possess all of the devices shown. More or fewer devices may be implemented or possessed alternatively.
[0092] Specifically, according to the embodiments herein, the process described in the above-referenced flowchart can be implemented as a computer software program. In one case, the embodiments herein include a computer program product comprising a computer program or instructions. In this case, the computer program or instructions can be downloaded and installed from a network via communication device 509, or installed from storage device 508, or installed from ROM 502. When processing device 501 executes the aforementioned computer program or instructions, it performs the functions defined in the classification model processing method herein, possessing the corresponding functional modules and beneficial effects of the execution method.
[0093] This paper also provides a computer-readable storage medium that stores one or more computer programs. When a processor executes the one or more computer programs, it implements the classification model processing method described in this paper and has the corresponding functional modules and beneficial effects of the execution method.
[0094] It should be noted that the computer-readable medium mentioned above can be a computer-readable signal medium or a computer-readable storage medium, or any combination thereof. Computer-readable storage media can be, for example, an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, apparatus, or device, or any combination thereof. More specific examples of computer-readable storage media may include, but are not limited to: electrical connections having one or more wires, portable computer disks, hard disks, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, portable compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination thereof. In this document, a computer-readable storage medium can be any tangible medium that contains or stores a program used or in conjunction with an instruction execution system, apparatus, or device. In this document, a computer-readable signal medium can include data signals propagated in baseband or as part of a carrier wave, carrying computer-readable program code. The transmitted data signal can take various forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than a computer-readable storage medium, which can send, propagate, or transmit a program for use by or in connection with an instruction execution system, apparatus, or device. The program code contained on the computer-readable medium can be transmitted using any suitable medium, including but not limited to: wires, optical fibers, radio frequency (RF), etc., or any suitable combination thereof.
[0095] In one scenario, the client and server can communicate using any currently known or future-developed network protocol, such as Hypertext Transfer Protocol (HTTP), and can interconnect with digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include Local Area Networks (LANs), Wide Area Networks (WANs), the Internet (e.g., the Internet), and peer-to-peer networks (e.g., ad hoc peer-to-peer networks), as well as any currently known or future-developed networks.
[0096] The aforementioned computer-readable storage medium may be included in the aforementioned electronic device; or it may exist independently and not assembled into the electronic device.
[0097] In this document, computer program code for performing the operations described herein may be written in one or more programming languages or a combination thereof. These programming languages include, but are not limited to, object-oriented programming languages such as Java, Smalltalk, and C++, as well as conventional procedural programming languages such as the "C" language or similar programming languages. The program code may be executed entirely on the user's computer, partially on the user's computer, as a software package, partially on the user's computer and partially on a remote computer, or entirely on a remote computer or server. In cases involving remote computers, the remote computer may be connected to the user's computer via any type of network—including a local area network (LAN) or a wide area network (WAN)—or may be connected to an external computer (e.g., via the Internet using an Internet service provider).
[0098] The flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of devices, methods, and computer program products according to various embodiments herein. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing the specified logical function. It should also be noted that in some alternative implementations, the functions indicated in the blocks may occur in a different order than those indicated in the drawings. For example, two consecutively indicated blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams and / or flowcharts, and combinations of blocks in the block diagrams and / or flowcharts, may be implemented using a dedicated hardware-based system that performs the specified function or operation, or using a combination of dedicated hardware and computer instructions.
[0099] The functions described above in this document can be performed, at least in part, by one or more hardware logic components. For example, exemplary types of hardware logic components that can be used, without limitation, include: Field Programmable Gate Array (FPGA), Application Specific Integrated Circuit (ASIC), Application Specific Standard Parts (ASSP), System on Chip (SOC), Complex Programmable Logic Device (CPLD), and so on.
[0100] The above description is merely a preferred embodiment and an explanation of the technical principles employed. Those skilled in the art should understand that the scope of disclosure herein is not limited to technical solutions formed by specific combinations of the above-described technical features, but should also cover other technical solutions formed by arbitrary combinations of the above-described technical features or their equivalents without departing from the above-disclosed concept. For example, technical solutions formed by substituting the above features with (but not limited to) technical features disclosed herein that have similar functions.
[0101] Furthermore, while the operations are described in a specific order, this should not be construed as requiring these operations to be performed in the specific order shown or in a sequential order. In certain environments, multitasking and parallel processing may be advantageous. Similarly, while several specific implementation details are included in the above discussion, these should not be construed as limiting the scope of this document. Certain features described in the context of a single embodiment may also be implemented in combination in a single embodiment. Conversely, various features described in the context of a single embodiment may also be implemented individually or in any suitable sub-combination in multiple embodiments.
[0102] Although the subject matter has been described using language specific to structural features and / or methodological logic, it should be understood that the subject matter defined in the appended claims is not necessarily limited to the specific features or actions described above. Rather, the specific features and actions described above are merely illustrative examples of implementing the claims.
Claims
1. A classification model processing method, comprising: Obtain a first data set and a first prompt message; wherein, the first data set includes multiple first data items; the first data items are associated with a first category; Based on the first prompt information, the first data is classified using a classification model to obtain a second category; Based on the second category and the first category, a difference analysis is performed using a first intelligent system to obtain the causes of the difference; wherein, the causes of the difference include the causes of model bias. Based on multiple sets of first data and the corresponding causes of difference, a second intelligent system is used for inductive processing to obtain an inductive result; wherein, the inductive result includes a second data set and a deviation pattern; the second data set is a subset of the first data set; the second data set includes multiple sets of second data; the second data is associated with the causes of model deviation; the deviation pattern characterizes the common features that cause classification deviation in the classification model; Based on the second data set and the deviation pattern, the first prompt information is updated to obtain the second prompt information.
2. The method according to claim 1, wherein the cause of the difference further includes the cause of labeling deviation; The inductive results also include a third dataset and multiple third categories; among which... The third data set is a subset of the first data set; the third data set includes multiple third data sets; the third data sets have corresponding third categories; the third data sets are associated with the causes of the labeling deviation, or the third category corresponding to the third data sets is matched with the classification criteria by the second intelligent system.
3. The method according to claim 2, further comprising: Based on the third category, the first category is replaced, and the first data set is updated; wherein the first category corresponds to the third data.
4. The method according to claim 1, wherein the induction result further includes a fourth data set; wherein, The fourth data set is a subset of the first data set; the fourth data set includes multiple fourth data sets; the fourth data sets do not have a corresponding third category; at least one of the fourth data sets is transmitted to the verification node.
5. The method according to claim 4, further comprising: Receive the fourth category; wherein the fourth category corresponds to the fourth data, and the fourth category is the classification category returned by the verification node; Based on the fourth category, the first category is replaced, and the first data set is updated; wherein the first category corresponds to the fourth data.
6. The method according to claim 3 or 5, further comprising: Use the second prompt message from the previous round as the updated first prompt message; Based on the updated first prompt information and the updated first data set, the classification process, difference analysis, inductive process, and prompt information update are re-executed in sequence to obtain the second prompt information.
7. The method according to claim 1, further comprising: Use the second prompt message from the previous round as the updated first prompt message; Based on the updated first prompt information and the first data set, the classification process, difference analysis, induction process and prompt information update are re-executed in sequence to obtain the second prompt information until the first data set meets the first condition.
8. The method according to claim 1, further comprising: Based on the fifth data set and the third prompt information, the first intelligent system is used for verification processing to obtain the verification result; wherein, the fifth data set includes multiple fifth data; the third prompt information is a second prompt information indicating the end of the loop; the end of the loop corresponds to the first data set satisfying a first condition; In response to the verification result satisfying the second condition, based on the verification result and the fifth data set, a sixth data set and a sixth category are obtained, constituting the sixth data set; wherein, the sixth category corresponds to the sixth data set; the second condition includes at least one of the following: the fifth data set contains category features not covered by the first data set, and the category features are matched to a category by the first intelligent system and the classification criteria; the evaluation index of the fifth data set does not reach the index threshold; the fifth category corresponding to the fifth data set deviates from the classification criteria, and the classification criteria correspond to the third prompt information; The third prompt information is used as the updated first prompt information, and the first data set is updated based on the sixth data set; Based on the updated first prompt information and the updated first data set, the classification process, difference analysis, inductive process, and prompt information update are re-executed in sequence to obtain the second prompt information.
9. A classification model processing device, comprising: The data acquisition module is used to acquire a first data set and a first prompt message; wherein, the first data set includes multiple first data items; the first data items are associated with a first category; The classification module is used to classify the first data based on the first prompt information using a classification model to obtain a second category; The difference analysis module is used to perform difference analysis based on the second category and the first category using the first intelligent system to obtain the causes of the difference; wherein, the causes of the difference include the causes of model bias; An inductive processing module is used to perform inductive processing using a second intelligent system based on multiple first data points and the corresponding causes of difference in the first data points to obtain an inductive result; wherein, the inductive result includes a second data set and a deviation pattern; the second data set is a subset of the first data set; the second data set includes multiple second data points; the second data points are associated with the causes of model deviation; the deviation pattern characterizes the common features that cause classification deviation in the classification model; The prompt information update module is used to update the first prompt information based on the second data set and the deviation pattern to obtain the second prompt information.
10. An electronic device, comprising: A storage device on which computer programs are stored; A processing device for executing the computer program in the storage device to implement the classification model processing method according to any one of claims 1-8.
11. A computer-readable storage medium storing a computer program, wherein a processor executes the computer program to implement the classification model processing method of any one of claims 1-8.
12. A computer program product comprising a computer program or instructions, wherein a processor, when executing the computer program or instructions, implements the classification model processing method of any one of claims 1-8.
