Data processing method and classifier training method

Abstract

The embodiment of the invention provides a data processing method and a classifier training method, and the data processing method comprises the steps: marking candidate sample data in a candidate sample data set according to a data marking strategy, and obtaining at least two groups of data marking information corresponding to each candidate sample data, each group of data labeling information comprises a plurality of label values; performing multi-dimensional verification on the at least two groups of data labeling information corresponding to each candidate sample data, and determining target sample data in the candidate sample data set according to a multi-dimensional verification result; fusing the at least two groups of data annotation information corresponding to the target sample data to obtain target data annotation information corresponding to the target sample data; and constructing a target sample pair based on the target sample data and the target data annotation information, and storing the target sample pair to a target sample set.

Description

Technical Field

[0001] The embodiments in this specification relate to the field of data processing technology, and in particular to data processing methods and classifier training methods. Background Technology

[0002] With the development of computer and internet technologies, large language models are being applied in an increasing number of scenarios. The reasoning ability of these models largely depends on the quality and quantity of reasoning-intensive corpora in the pre-training data. Therefore, high-quality corpus selection is fundamental to ensuring the model's predictive ability. In existing technologies, most inference data retrieval operations rely on classifiers. Classifiers automatically label and retrieve data from the entire dataset, and the accuracy of the classifier's classification results depends on the quality of the samples used during its training phase. In practical applications, the samples used for classifier training are typically defined by experts using a reasoning labeling system to define the scope of the inference data. Then, samples are labeled based on this system to complete the classifier training. This process is costly, has poor scalability, and is difficult to rapidly label large-scale data. Therefore, an effective solution is urgently needed to address these issues. Summary of the Invention

[0003] In view of this, embodiments of this specification provide data processing methods. One or more embodiments of this specification also relate to classifier training methods, data processing apparatus, classifier training apparatus, a computing device, a computer-readable storage medium, and a computer program product, to address the technical deficiencies existing in the prior art.

[0004] According to a first aspect of the embodiments of this specification, a data processing method is provided, comprising:

[0005] The candidate sample data in the candidate sample dataset is labeled according to the data labeling strategy, and at least two sets of data labeling information are obtained for each candidate sample data, wherein each set of data labeling information contains multiple label values.

[0006] For each candidate sample data, perform multi-dimensional verification on at least two sets of data annotation information corresponding to the candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results.

[0007] At least two sets of data annotation information corresponding to the target sample data are fused to obtain the target data annotation information corresponding to the target sample data.

[0008] Target sample pairs are constructed based on the target sample data and the target data annotation information, and the target sample pairs are stored in the target sample set.

[0009] According to a second aspect of the embodiments of this specification, a classifier training method is provided, comprising:

[0010] The candidate sample data in the candidate sample dataset is labeled according to the data labeling strategy, and at least two sets of data labeling information are obtained for each candidate sample data, wherein each set of data labeling information contains multiple label values.

[0011] For each candidate sample data, perform multi-dimensional verification on at least two sets of data annotation information corresponding to the candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results.

[0012] At least two sets of data annotation information corresponding to the target sample data are fused to obtain the target data annotation information corresponding to the target sample data.

[0013] Target sample pairs are constructed based on the target sample data and the target data annotation information, and the target sample pairs are stored in the target sample set;

[0014] The target classifier is obtained by training the classifier using the target sample pairs contained in the target sample set.

[0015] According to a third aspect of the embodiments of this specification, another data processing method is provided, including:

[0016] Obtain the set of data to be processed;

[0017] The data to be processed contained in the dataset to be processed is input into the target classifier for classification processing. Based on the classification processing result, the inference data is determined in the dataset to be processed. The target classifier is obtained by the above method.

[0018] The business model is trained using the inference data until a target business model that meets the training stopping condition is obtained.

[0019] According to a fourth aspect of the embodiments of this specification, a data processing apparatus is provided, comprising:

[0020] The annotation module is configured to annotate the candidate sample data in the candidate sample dataset according to the data annotation strategy, and obtain at least two sets of data annotation information corresponding to each candidate sample data, wherein each set of data annotation information contains multiple label values;

[0021] The verification module is configured to perform multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results.

[0022] The fusion module is configured to fuse at least two sets of data annotation information corresponding to the target sample data to obtain target data annotation information corresponding to the target sample data.

[0023] The construction module is configured to construct target sample pairs based on the target sample data and the target data annotation information, and store the target sample pairs in the target sample set.

[0024] According to a fifth aspect of the embodiments of this specification, a classifier training apparatus is provided, comprising:

[0025] The annotation module is configured to annotate the candidate sample data in the candidate sample dataset according to the data annotation strategy, and obtain at least two sets of data annotation information corresponding to each candidate sample data, wherein each set of data annotation information contains multiple label values;

[0026] The verification module is configured to perform multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results.

[0027] The fusion module is configured to fuse at least two sets of data annotation information corresponding to the target sample data to obtain target data annotation information corresponding to the target sample data.

[0028] The construction module is configured to construct target sample pairs based on the target sample data and the target data annotation information, and store the target sample pairs in the target sample set;

[0029] The training module is configured to train a classifier using target sample pairs contained in the target sample set to obtain a target classifier.

[0030] According to a sixth aspect of the embodiments of this specification, another data processing apparatus is provided, comprising:

[0031] The acquisition module is configured to acquire a set of data to be processed.

[0032] The processing module is configured to input the data to be processed contained in the data set to be processed into the target classifier for classification processing, and determine the inference data in the data set to be processed based on the classification processing result, wherein the target classifier is obtained by the above method;

[0033] The training module is configured to train the business model using the inference data until a target business model that meets the training stopping condition is obtained.

[0034] According to a seventh aspect of the embodiments of this specification, a computing device is provided, comprising:

[0035] Memory and processor;

[0036] The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method or classifier training method.

[0037] According to an eighth aspect of the embodiments of this specification, a computer-readable storage medium is provided that stores computer-executable instructions, which, when executed by a processor, implement the steps of the above-described data processing method or classifier training method.

[0038] According to a ninth aspect of the embodiments of this specification, a computer program product is provided, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described data processing method or classifier training method.

[0039] The data processing method provided in this embodiment, in order to reduce the cost of sample construction and improve the scalability of sample construction, can first label the candidate sample data in the candidate sample data set according to the data labeling strategy after determining the candidate sample data set. At this time, at least two sets of data labeling information corresponding to each candidate sample data can be determined. Each set of data labeling information contains multiple label values, and each label value represents a different type. On this basis, multi-dimensional verification can be performed on the at least two sets of data labeling information corresponding to each candidate sample data, so as to determine the target sample data that passes the verification in the candidate sample data set based on the multi-dimensional verification results. Since each sample data corresponds to at least two sets of data labeling information, the at least two sets of data labeling information corresponding to the target sample data can be fused to obtain the target data labeling information corresponding to the target sample data. Based on this, target sample pairs can be constructed based on the target sample data and target data labeling information, and the target sample pairs can be stored in the target sample set. This allows the high-quality target sample set to be used for model training. This data processing method can not only reduce the cost of data labeling, but also ensure the quality of data labeling, adapt to different labeling scenarios, and thus improve the efficiency of training the model using the target sample set. Attached Figure Description

[0040] Figure 1 This is a flowchart illustrating a data processing method provided in one embodiment of this specification;

[0041] Figure 2 This is a flowchart illustrating a classifier training method provided in one embodiment of this specification;

[0042] Figure 3 This is a flowchart of another data processing method provided in one embodiment of this specification;

[0043] Figure 4 This is a timing flowchart of a data processing method provided in one embodiment of this specification;

[0044] Figure 5 This is a schematic diagram of the structure of a data processing apparatus provided in one embodiment of this specification;

[0045] Figure 6 This is a schematic diagram of the structure of a classifier training device provided in one embodiment of this specification;

[0046] Figure 7 This is a schematic diagram of another data processing apparatus provided in one embodiment of this specification;

[0047] Figure 8 This is a structural block diagram of a computing device provided in one embodiment of this specification. Detailed Implementation

[0048] Many specific details are set forth in the following description to provide a full understanding of this specification. However, this specification can be implemented in many other ways than those described herein, and those skilled in the art can make similar extensions without departing from the spirit of this specification. Therefore, this specification is not limited to the specific implementations disclosed below.

[0049] The terminology used in one or more embodiments of this specification is for the purpose of describing particular embodiments only and is not intended to be limiting of the one or more embodiments of this specification. The singular forms “a,” “described,” and “the” as used in one or more embodiments of this specification and the appended claims are also intended to include the plural forms unless the context clearly indicates otherwise. It should also be understood that the term “and / or” as used in one or more embodiments of this specification refers to and includes any or all possible combinations of one or more associated listed items.

[0050] It should be understood that although the terms first, second, etc., may be used to describe various information in one or more embodiments of this specification, such information should not be limited to these terms. These terms are only used to distinguish information of the same type from one another. For example, first may also be referred to as second without departing from the scope of one or more embodiments of this specification, and similarly, second may also be referred to as first. Depending on the context, the word "if" as used herein may be interpreted as "when," "when," or "in response to a determination."

[0051] Furthermore, it should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in one or more embodiments of this specification are all information and data authorized by the user or fully authorized by all parties. Moreover, the collection, use and processing of related data must comply with the relevant laws, regulations and standards of the relevant countries and regions, and corresponding operation entry points are provided for users to choose to authorize or refuse.

[0052] First, the terms and concepts used in one or more embodiments of this specification will be explained.

[0053] Large Language Model (LLM): A deep learning-based artificial intelligence model with massive parameters (usually reaching billions or even hundreds of billions) specifically designed to understand and generate human language.

[0054] A classifier is a model or algorithm in machine learning whose task is to assign input data to predefined categories or labels. It is a typical application of supervised learning.

[0055] Feature engineering refers to the process of extracting, constructing, transforming, and selecting the most useful features (variables) for training machine learning models from raw data. It is a crucial part of machine learning projects and directly affects model performance.

[0056] This specification provides a data processing method. One or more embodiments of this specification also relate to a classifier training method, a data processing apparatus, a classifier training apparatus, a computing device, a computer-readable storage medium, and a computer program product, which will be described in detail in the following embodiments.

[0057] In practical applications, the reasoning ability of current large-scale language models largely depends on the quality and quantity of reasoning-intensive corpora in the pre-training data. Research shows that data that can significantly improve a model's reasoning ability is sparsely distributed and difficult to identify in general web page text. This reasoning data typically contains cognitively intensive content such as complex logical arguments, multi-step reasoning processes, and causal relationship analysis, but it only accounts for a small proportion of the massive amount of general web page text. Therefore, how to efficiently and accurately retrieve reasoning data from terabytes of general web page text has become a key technical problem for improving the model's reasoning ability.

[0058] Traditional inference data retrieval processes typically rely on expert annotation. The process involves: first, annotation experts develop an "inference labeling system" to define the scope of inference data; then, general web page text is manually annotated based on this system; next, a binary classifier (inference / non-inference) is trained on the annotated data; and finally, the classifier is used to automatically annotate and retrieve the entire dataset. However, this traditional method suffers from serious systemic flaws. Expert annotation is costly and has poor scalability, making it difficult to handle terabytes of web page data. More importantly, the entire iteration cycle is long, typically taking months from annotation to model deployment, and it presents challenges in discovering and annotating sparse class data.

[0059] Traditional methods face even greater challenges in terms of annotation quality. Due to the lack of a unified definition of "reasoning," annotators can only rely on experience, leading to label drift and increasing noise in the training set. Poor consistency among annotators (low κ value) and unstable data quality directly affect the training performance of subsequent classifiers. Particularly when dealing with samples with ambiguous boundaries, annotators often disagree, resulting in different annotations for the same text at different times or from different annotators.

[0060] The design flaws of the existing inference labeling system further exacerbate these problems. Conceptually, the concept of inference lacks an operational definition, merely describing it as "data that enhances reasoning ability," forcing labelers to rely solely on subjective experience. The essential characteristics of inference are not clearly defined, making it difficult to establish unified labeling standards. Structurally, the existing labeling system places concepts of different natures—such as topics (e.g., "medical health"), genres (e.g., "news reports"), and logical strength (e.g., "syllogism")—into the same classification tree, resulting in high heterogeneity within labels and making it difficult for models to learn effective feature representations. Furthermore, positive class labels are overly subdivided (e.g., "formal logic theory → basic laws"), while negative classes only have a coarse "non-inference" label, causing a severe long-tail problem and making models prone to overfitting to a few high-frequency categories.

[0061] More seriously, the existing tagging system suffers from significant category overlap and blurred boundaries. For example, tags such as "industry analysis," "development review," and "business analysis" overlap, and the same text may match multiple tags. This not only affects annotation consistency but also makes sample statistics difficult. At the logical level, the tagging system juxtaposes weak logical content (such as "news interpretation" and "sentiment analysis") with strong logical tags (such as "mathematical derivation"), making it easy for models to learn pseudo-logical features and generate numerous misjudgments in practical applications. Furthermore, the tagging system lacks annotation for essential reasoning patterns such as deduction, induction, abduction, and multi-hop reasoning, causing models to only learn superficial topic signals and exhibiting weak generalization ability.

[0062] To address the high cost of manual annotation, the industry has attempted to use large-scale model APIs to replace expert annotation. However, this approach faces new technical challenges. Due to the irregular boundaries of existing labeling systems, APIs struggle to accurately grasp annotation standards, and the randomness of a single API call leads to inconsistent annotation results for the same text multiple times. More importantly, the lack of effective quality control mechanisms makes it difficult to guarantee the reliability of annotation results. In terms of large-scale processing, issues such as API call concurrency limitations and latency, fault tolerance and continuation mechanisms for large-scale data processing, and the balance between cost control and annotation quality all require careful resolution.

[0063] Training inference data classifiers based on labeled data presents numerous technical challenges. Label noise leads to overfitting and poor generalization; class imbalance severely impacts model performance; and insufficient long-tail class samples result in low recall. In terms of model selection and optimization, key challenges include choosing an appropriate model architecture to balance accuracy and efficiency, designing effective loss functions to handle the transition from multi-label to binary classification, and achieving high recall while maintaining high accuracy.

[0064] The data processing method provided in this embodiment, in order to reduce the cost of sample construction and improve the scalability of sample construction, can first label the candidate sample data in the candidate sample data set according to the data labeling strategy after determining the candidate sample data set. At this time, at least two sets of data labeling information corresponding to each candidate sample data can be determined. Each set of data labeling information contains multiple label values, and each label value represents a different type. On this basis, multi-dimensional verification can be performed on the at least two sets of data labeling information corresponding to each candidate sample data, so as to determine the target sample data that passes the verification in the candidate sample data set based on the multi-dimensional verification results. Since each sample data corresponds to at least two sets of data labeling information, the at least two sets of data labeling information corresponding to the target sample data can be fused to obtain the target data labeling information corresponding to the target sample data. Based on this, target sample pairs can be constructed based on the target sample data and target data labeling information, and the target sample pairs can be stored in the target sample set. This allows the high-quality target sample set to be used for model training. This data processing method can not only reduce the cost of data labeling, but also ensure the quality of data labeling, adapt to different labeling scenarios, and thus improve the efficiency of training the model using the target sample set.

[0065] See Figure 1 , Figure 1 A flowchart of a data processing method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0066] Step S102: Label the candidate sample data in the candidate sample data set according to the data labeling strategy to obtain at least two sets of data labeling information corresponding to each candidate sample data, wherein each set of data labeling information contains multiple label values.

[0067] The data processing method provided in this embodiment can be applied to any sample construction scenario of a classifier to be trained. The trained classifier can be used for data classification in any scenario, such as the classification of inference data and non-inference data, the classification of AI images and non-AI images, the classification of correct text and incorrect text, etc. It is used to automatically construct sample pairs for training classifiers, thereby improving the predictive ability of classifiers, reducing sample construction costs, and covering more application scenarios.

[0068] This embodiment describes the data processing method provided in this embodiment by taking the construction of samples required for training a classifier that can process both inference data and non-inference data as an example. Other scenarios for constructing sample data required by classifiers can refer to the same or corresponding descriptions in this embodiment, and will not be elaborated on further here.

[0069] Specifically, the candidate sample data set refers to a collection of multiple candidate sample data. In different scenarios, the candidate sample data can be of different types, such as images, text, audio, and video. Correspondingly, the data annotation strategy refers to the strategy for labeling each candidate sample data. This strategy can be set according to actual needs, adding labels to each candidate sample data across multiple labeling dimensions and setting a label value for each label. This is used to subsequently compose annotation information, facilitating the construction of sample pairs. Specifically, at least two sets of data annotation information refer to the data annotation information composed of the label values ​​associated with each labeling result after performing at least two labeling processes on the same candidate sample data in the same way. Each set of data annotation information includes multiple label values, each label value being the value of each type of label corresponding to the candidate sample data. The types of the multiple label values ​​can be set according to actual needs; this embodiment does not impose any limitations on this.

[0070] Therefore, to reduce the cost of sample construction and improve its scalability, after determining the candidate sample dataset, the candidate sample data in the dataset can be labeled according to a data labeling strategy. At this point, at least two sets of data labeling information can be determined for each candidate sample data. Each set of data labeling information contains multiple label values, and each label value represents a different type. Based on this, multi-dimensional verification can be performed on the at least two sets of data labeling information corresponding to each candidate sample data. Based on the multi-dimensional verification results, the target sample data that passes the verification can be determined from the candidate sample dataset. Since each sample data corresponds to at least two sets of data labeling information, the at least two sets of data labeling information corresponding to the target sample data can be fused to obtain the target data labeling information corresponding to the target sample data. Based on this, target sample pairs can be constructed based on the target sample data and target data labeling information, and the target sample pairs can be stored in the target sample set. This allows the high-quality target sample set to be used for model training. This data processing method not only reduces the cost of data labeling but also ensures the quality of data labeling, adapts to different labeling scenarios, and thus improves the efficiency of training the model using the target sample set.

[0071] Furthermore, to ensure the uniqueness of candidate sample data within the candidate sample dataset and avoid duplicate sample construction, the candidate sample dataset can be constructed by calculating hash values. In this embodiment, before the step of labeling the candidate sample data in the candidate sample dataset according to the data labeling strategy to obtain at least two sets of data labeling information corresponding to each candidate sample data, the following steps are also included:

[0072] Initial sample data is acquired and preprocessed to obtain a word sequence. If the number of words in the word sequence exceeds a set threshold, the word sequence is truncated to obtain a target word sequence. The target word sequence is processed using a hash algorithm to obtain sequence identifier information, and the initial sample data is stored in a candidate sample data set according to the sequence identifier information. The sequence identifier information is used to deduplicate candidate sample data in the candidate sample data set.

[0073] Specifically, initial sample data refers to sample data that has not yet been stored in the candidate sample set. This data can be text, images, videos, audio, etc. The token sequence refers to the sequence of tokens obtained after word segmentation of the initial sample data; each token is a word. The token count is the number of tokens contained in the token sequence, and the count threshold is the threshold for truncating the token sequence. Sequence identifier information refers to the unique identifier obtained after hashing the sequence, used to ensure the uniqueness of each candidate sample in the candidate sample data set.

[0074] Based on this, after obtaining the initial sample data, in order to store high-quality sample data in the candidate sample data set, it can be processed. At this time, the initial sample data can be preprocessed to obtain a word sequence; then, if the number of words in the word sequence is greater than a set threshold, the word sequence can be truncated to obtain the target word sequence; then, the target word sequence is processed using a hash algorithm to obtain sequence identifier information. Based on this, the initial sample data can be stored in the candidate sample data set according to the sequence identifier information; the sequence identifier information can be used to remove duplicate candidate sample data in the candidate sample data set, thereby ensuring the uniqueness of the data in the set.

[0075] In practical applications, when processing each initial sample data, a word segmenter can be used for standardized word segmentation. When the number of tokens obtained after word segmentation exceeds a set threshold, such as more than 4096 tokens, the token sequence can be truncated, retaining the first 4096 tokens to ensure processing efficiency. The preprocessed text generates its corresponding unique identifier using the SHA256 hash algorithm, which can realize deduplication checking and task continuation mechanism. The system only needs to maintain the hash index, and can achieve efficient duplicate detection with a lookup complexity of 0(1), thus facilitating the subsequent use of high-quality candidate sample data to complete classifier training.

[0076] In summary, by intelligently truncating excessively long word sequences and generating unique identifiers based on hash algorithms, automatic deduplication and efficient storage of candidate sample data are achieved. This significantly reduces redundant data usage, improves processing speed and storage space utilization, while ensuring the uniqueness and integrity of the sample set. This provides a more reliable and optimized data foundation for large-scale text analysis tasks, effectively enhancing the overall system performance and scalability.

[0077] Furthermore, the annotation process can be quickly completed using a model. In this embodiment, the annotation of any candidate sample data in the candidate sample data set includes:

[0078] A data annotation model is determined according to a data annotation strategy, and the prompt words corresponding to the first candidate sample data are determined. The data annotation model is used to annotate the first candidate sample data at least twice according to the prompt words to obtain at least two sets of data annotation information corresponding to the first candidate sample data. Among them, the multiple label values ​​contained in each set of data annotation information are associated with the target analysis category, the reasoning behavior category and / or the context analysis category, and the target analysis category, the reasoning behavior category and / or the context analysis category correspond to at least one of the multiple label values.

[0079] Specifically, the first candidate sample data refers to any candidate sample data contained in the candidate sample data set. The data labeling model refers to a model capable of adding labels to the candidate sample data, which can output label values ​​for multiple set labels corresponding to each candidate sample data. These set labels are communicated to the data labeling model via prompts. The prompts are the alerts set for data labeling in the current scenario. Correspondingly, at least two labeling operations refer to performing at least two independent labeling operations on the candidate sample data. Correspondingly, the target analysis category refers to the category corresponding to the labels related to core intent and reasoning needs in the candidate sample data; the reasoning behavior category refers to the category corresponding to the labels related to reasoning knowledge in the candidate sample data; and the context analysis category refers to the category corresponding to the labels related to the contextual content in the candidate sample data. This can be understood as follows: the labels corresponding to the target analysis category can be associated with the core intent and reasoning needs in the candidate sample data; the labels corresponding to the reasoning behavior category can be associated with the reasoning logic in the candidate sample data; and the labels corresponding to the context analysis category can be associated with the environment and conditions under which the evaluation and reasoning occur in the candidate sample data.

[0080] Based on this, when annotating any candidate sample data in the candidate sample dataset according to the data annotation strategy, the data annotation model can be determined first according to the data annotation strategy, and the prompt words corresponding to the first candidate sample data can be determined. Then, the data annotation model can be used to annotate the first candidate sample data at least twice according to the prompt words, thereby obtaining at least two sets of data annotation information corresponding to the first candidate sample data. Among them, the multiple label values ​​contained in each set of data annotation information are associated with the target analysis category, reasoning behavior category and / or context analysis category, and the target analysis category, reasoning behavior category and / or context analysis category correspond to at least one of the multiple label values. Based on this, after performing the above annotation processing on each candidate sample data, it can be used for subsequent sample construction processing operations.

[0081] In practical applications, in order to complete multi-dimensional labeling of different candidate samples through data labeling strategies, this embodiment constructs a multi-dimensional reasoning labeling system containing twelve core dimensions. This system can comprehensively characterize the reasoning features of the text from three levels.

[0082] Firstly, the goal analysis level primarily identifies the core intent and reasoning needs of the text. The Goal_Type dimension categorizes text goals into eight types, including information delivery (Inform / Report), description / instruction, narration / recount, and reasoning-intensive types such as explanation / analysis, argumentation / persuasion, evaluation / comparison, and solution / propose. This classification effectively distinguishes the intensity of reasoning needs for different goal types, providing a foundation for subsequent reasoning judgments. The Goal_Reasoning_Relevance dimension uses a 0-5 scoring system to assess the reasoning need of the goal type itself. This dimension reflects the inherent reasoning potential of the goal, rather than the actual amount of reasoning performed in the text. This design effectively identifies task types that inherently require reasoning activity to complete. The Goal_Clarity_Focus dimension also uses a 0-5 scoring system to assess the clarity and focus of the text's main goal, which is crucial for judging text quality and the effectiveness of reasoning. The Num_Goals dimension counts the number of main objectives contained in the text, providing a reference for subsequent complexity analysis.

[0083] Secondly, the reasoning behavior analysis delves into the actual reasoning process executed within the text. The R_Type dimension identifies specific mechanisms of reasoning, including deductive reasoning (necessary reasoning from general to specific), inductive reasoning (probabilistic reasoning from specific instances to general conclusions), abductive reasoning (the best explanation reasoning that deduces causes from results), analogical reasoning (reasoning based on similarity), and modal reasoning (reasoning involving modal concepts such as possibility and necessity). Each reasoning type is assigned a confidence score to quantify the reliability of the reasoning identification. The Chain_Depth dimension assesses the depth of the reasoning chain, using a 0-6 scoring system. This dimension quantifies the complexity of reasoning by identifying the number of reasoning operations (the smallest independent steps from known information to new information). The Explicit_R_Type and Explicit_Chain_Depth dimensions specifically analyze explicitly marked reasoning steps in the text, judging the explicitness of reasoning by identifying explicit logical connectors such as "therefore," "so," and "if...then...".

[0084] Thirdly, the contextual analysis level assesses the environment and conditions under which reasoning occurs. The Reasoning_Knowledge_Dependency dimension uses a 0-5 score to evaluate the degree to which the reasoning process depends on external knowledge, ranging from completely self-consistent reasoning to reasoning requiring extensive specialized knowledge. The Text_Knowledge_Level dimension assesses the overall knowledge level and professionalism of the text, ranging from basic common sense to cutting-edge expert content. The R_Form dimension identifies the form of reasoning expression, including different forms such as text, formulas, and code. The Scene dimension determines the domain and scenario in which reasoning occurs, covering multiple professional fields such as logic exercises, mathematical statistics, science and engineering, medical and health, law, and business and finance.

[0085] The label names, label types, value ranges, and meanings corresponding to the above three levels are shown in Table (1) below:

[0086]

[0087]

[0088] The value range refers to the value corresponding to the label contained in each data annotation information, i.e., the label value. For any candidate sample data, the 12 labels mentioned above can be labeled using the data annotation model, thus obtaining the label values ​​corresponding to the 12 labels in a set of data annotation information, which can be used for subsequent sample construction. Furthermore, in order to achieve more accurate labeling results for each candidate sample data, each candidate sample data can be independently labeled two or more times to obtain at least two sets of data annotation information. By comparing at least two sets of data annotation information, the corresponding target data annotation information can be obtained, thereby improving the quality of the final constructed sample set.

[0089] In other words, for any candidate sample data, at least two completely independent API calls can be made. Each call uses the same input data and prompt template, but is executed through an independent session instance to ensure that the two annotation processes do not affect each other.

[0090] Furthermore, to address the instability of API calls, the system implements an exponential backoff retry mechanism. The retry strategy employs a truncated exponential backoff algorithm, with an initial delay of 1 second, doubling with each retry, a maximum of 10 retries, and a maximum delay limit of 60 seconds. The system handles different types of API errors differently: retries are executed for network timeouts and service unavailability errors, while authentication failures and quota exceedances are terminated directly and error information is logged, ensuring rapid completion of error processing.

[0091] This embodiment uses the example of performing two labeling processes on each candidate sample data to illustrate the data processing method. The construction process for other numbers of labeled samples can be found in the description of this embodiment. Specifically, firstly, n initial texts are obtained. Then, the n initial texts can be input into a word segmenter for processing, and only the first 4096 tokens of each initial text are retained. Then, a hash algorithm is used to calculate the hash value corresponding to each of the n initial texts, which serves as their unique identifier. In this way, the texts are stored in the candidate text set.

[0092] Furthermore, after obtaining the candidate text set, the n candidate texts contained in the candidate text set can be combined with a preset prompt and input into the annotation model for labeling. At this time, the label values ​​corresponding to the above 12 labels can be obtained for each candidate text. Each candidate text is labeled twice, so any candidate text can obtain two sets of label information, A and B. The label information in set A contains the label values ​​corresponding to the 12 labels, and the label information in set B also contains the label values ​​corresponding to the 12 labels. On this basis, high-quality sample screening can be carried out.

[0093] In summary, multi-round annotation processing and a prompt-word guidance mechanism significantly improve the accuracy and consistency of data annotation. The generated multiple sets of annotation information ensure a deep correlation between label values ​​and target analysis, reasoning behavior, and contextual categories, effectively avoiding single-batch annotation bias. This design enhances the reliability of the annotation results, provides a high-quality structured data foundation for AI model training, and significantly optimizes the accuracy and decision support capabilities of analysis tasks.

[0094] Step S104: Perform multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results.

[0095] Specifically, after obtaining at least two sets of data annotation information corresponding to each candidate sample data, in order to filter out high-quality target sample data, cross-multi-dimensional verification can be performed on at least two sets of data annotation information corresponding to each candidate sample data. This includes verification of pairwise label value consistency, score distance, or matching degree, etc., so that the target sample data can be determined from the candidate sample data set based on the verification results. In specific implementation, when determining the target sample data from the candidate sample data set based on the verification results, candidate sample data with consistent two sets of data annotation information can be selected as the target sample data, or candidate sample data with higher similarity in the multi-dimensional verification results can be selected as the target sample data. This embodiment does not impose any limitations here.

[0096] Multi-dimensional verification specifically refers to the operation of performing multi-dimensional verification on the label values ​​of the same type in at least two sets of data annotation information corresponding to each candidate sample data. The verification dimensions include, but are not limited to, tolerance dimensions, matching dimensions, or confidence dimensions. Correspondingly, the target sample data specifically refers to the sample data that has passed the multi-dimensional verification and has been selected from the candidate sample data set.

[0097] Furthermore, in order to enable the subsequent construction of sample pairs based on samples containing inference data, sample filtering can be performed in advance based on inference requirement labels. In this embodiment, before the multi-dimensional verification step of at least two sets of data annotation information corresponding to each candidate sample data is executed, the following steps are also included:

[0098] In each candidate sample data set, at least two inference requirement label values ​​are extracted from at least two sets of data annotation information. These inference requirement label values ​​are then fused. Candidate sample data with the first field value associated with the fused inference requirement label value are deleted from the candidate sample data set. A target candidate sample data set is determined based on the deletion result. The step of performing multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data set, and determining the target sample data in the candidate sample data set based on the multi-dimensional verification results, includes: performing multi-dimensional verification on at least two sets of data annotation information corresponding to the candidate sample data included in the target candidate sample data set, and determining the target sample data in the target candidate sample data set based on the multi-dimensional verification results.

[0099] Specifically, the inference requirement label value refers to the label value associated with the inference requirement label in each group of data annotation information corresponding to the candidate sample data. The inference requirement label is used to indicate whether the candidate sample data contains inference logic. Correspondingly, the fusion of inference requirement label values ​​refers to the processing of averaging the label values. The first field value is specifically a set value for detecting whether the candidate sample data contains inference logic, such as 0; it can be understood that when the inference requirement label value in any annotation result of the candidate sample data is 0, it can be determined that the data does not contain inference logic, and can be directly discarded. The target candidate sample data set specifically refers to the sample data set obtained by deleting the candidate sample data that does not contain inference logic.

[0100] Based on this, to improve subsequent processing efficiency, a preliminary screening can be performed before determining the target sample data. At this time, at least two inference requirement label values ​​corresponding to each candidate sample data can be extracted from at least two sets of data annotation information. Then, the at least two inference requirement label values ​​corresponding to each candidate sample data can be fused. The fused inference requirement label value can reflect whether the candidate sample data contains inference data. Therefore, the candidate sample data associated with the first field value of the fused inference requirement label value can be deleted from the candidate sample data set, thereby determining the target candidate sample data set based on the deletion result. On this basis, multi-dimensional verification can be performed only on the at least two sets of data annotation information corresponding to the candidate sample data contained in the target candidate sample data set, so as to determine the target sample data in the target candidate sample data set based on the multi-dimensional verification result, which is used for the subsequent construction of target sample pairs.

[0101] In practical applications, to improve the efficiency of sample construction, special rule verification can be performed first to quickly eliminate candidate sample data that does not contain reasoning logic. During this process, the label values ​​corresponding to the Goal_Reasoning_Relevance label field in at least two annotation information corresponding to each candidate sample data can be extracted. When the average of at least two label values ​​is 0, it indicates that the candidate sample data does not contain reasoning content. Therefore, it can be determined that the candidate sample data can be discarded, thereby improving the processing efficiency of non-reasoning candidate sample data.

[0102] In summary, by fusing inference requirement labels and employing a multi-dimensional verification mechanism, redundant samples with correlated first field values ​​are intelligently eliminated, significantly improving the purity and analytical applicability of the target candidate dataset. This design effectively eliminates data redundancy interference, optimizes the sample selection process, and ensures that the final dataset highly matches the target analysis requirements. It provides a highly reliable and targeted training foundation for AI models, greatly improving decision-making accuracy and system operating efficiency in complex tasks.

[0103] Furthermore, when performing multi-dimensional validation for each candidate sample data, tolerance dimension, matching dimension, and confidence dimension can be combined to validate different types of label values. In this embodiment, the multi-dimensional validation of any candidate sample data includes:

[0104] From at least two sets of data annotation information corresponding to the second candidate sample data, at least two numerical label values, at least two list label values, and at least two structural label values ​​corresponding to the second candidate sample data are determined; the at least two numerical label values ​​are validated in the tolerance dimension, the at least two list label values ​​are validated in the matching dimension, and the at least two structural label values ​​are validated in the confidence dimension; if the validation results of the tolerance dimension, the matching dimension, and the confidence dimension all meet the validation conditions, the second candidate sample data is used as the target sample data.

[0105] Specifically, the second candidate sample data refers to any candidate sample data contained in the set. Correspondingly, numerical label values ​​refer to the label values ​​associated with numerical types in each group of data annotation information, such as Goal_Reasoning_Relevance, Goal_Clarity_Focus, Chain_Depth, Explicit_Chain_Depth, Reasoning_Knowledge_Dependency, and Text_Knowledge_Level; correspondingly, list-type label values ​​refer to the label values ​​associated with list types, such as Goal_Type; correspondingly, structured label values ​​refer to the label values ​​corresponding to structured fields, such as R_Type; correspondingly, the tolerance dimension refers to a dimension set by combining subjectivity and the randomness of API calls, used to ensure quality while avoiding overly strict requirements; the matching dimension refers to the dimension for matching verification according to set standards, used to ensure verification accuracy according to different matching standards; and the confidence dimension refers to the dimension for verification according to confidence levels.

[0106] Based on this, for multi-dimensional verification of any candidate sample data, we can first determine at least two numerical label values, at least two list label values, and at least two structural label values ​​corresponding to the second candidate sample data from at least two sets of data annotation information. Then, we can perform verification on the tolerance dimension for the at least two numerical label values, on the matching dimension for the at least two list label values, and on the confidence dimension for the at least two structural label values. This allows us to filter out target sample data whose verification results in the tolerance dimension, matching dimension, and confidence dimension all meet the verification conditions. This second candidate sample data can then be used as the target sample data. By analogy, after determining multiple target sample data, they can be used for subsequent sample pair construction.

[0107] In summary, through a three-dimensional label verification mechanism encompassing tolerance, matching, and confidence, the logical consistency and numerical accuracy of numerical, list, and structured labels are accurately verified. Automatic identification and removal of conflicting labels significantly enhances the logical rigor and credibility of the target dataset. This provides a high-precision, highly adaptable data foundation for AI model training, effectively mitigating analytical biases caused by label errors and substantially optimizing decision-making efficiency and system robustness in complex tasks.

[0108] Based on this, the verification of the at least two numerical label values ​​in the tolerance dimension includes: constructing statistical label values ​​based on the at least two numerical label values, comparing the statistical label values ​​with a set statistical threshold, and determining the verification result of the tolerance dimension based on the comparison result;

[0109] The step of validating the at least two list-type label values ​​in the matching dimension includes: when the at least two list-type label values ​​are associated with a first matching strategy, detecting whether the at least two list-type label values ​​are equal based on the first matching strategy; when the at least two list-type label values ​​are associated with a second matching strategy, detecting whether the intersection of the at least two list-type label values ​​is an empty set based on the second matching strategy; and determining the validation result of the matching dimension based on the detection result.

[0110] The step of verifying the confidence dimension for the at least two structural label values ​​includes: determining the confidence level corresponding to each of the at least two structural label values; fusing the confidence levels corresponding to the at least two structural label values ​​to obtain an average confidence level; comparing the average confidence level with a preset confidence threshold; and determining the verification result of the confidence dimension based on the comparison result.

[0111] Specifically, the statistical label value refers to the value obtained by averaging, subtracting, or weighting at least two numerical label values. Correspondingly, the statistical threshold is a set threshold used to verify whether the tolerance dimension passes the validation. The first matching strategy is a pre-defined strict matching strategy that mandates at least two list-type label values ​​to be equal; the second matching strategy is a pre-defined lenient matching strategy that requires at least two list-type label values ​​to be approximately equal. The confidence level is the confidence level corresponding to the structured label value. The average confidence level is the confidence level obtained after weighted averaging.

[0112] Based on this, when validating at least two numerical label values ​​in the tolerance dimension, statistical label values ​​can be constructed first based on the at least two numerical label values. Then, the statistical label values ​​can be compared with a set statistical threshold to determine the validation result of the tolerance dimension based on the comparison result. When validating at least two list-type label values ​​in the matching dimension, if the at least two list-type label values ​​are associated with a first matching strategy, the equality of the at least two list-type label values ​​can be detected based on the first matching strategy. If the at least two list-type label values ​​are associated with a second matching strategy, the intersection of the at least two list-type label values ​​can be detected based on the second matching strategy to determine whether it is an empty set. This allows the validation result of the matching dimension to be determined based on the detection result. When validating at least two structural label values ​​in the confidence dimension, the confidence level corresponding to each of the at least two structural label values ​​can be determined first. Then, the confidence levels corresponding to each of the at least two structural label values ​​can be fused to obtain an average confidence level. By comparing the average confidence level with a preset confidence threshold, the validation result of the confidence dimension can be determined based on the comparison result.

[0113] In practical applications, when performing multi-dimensional validation, appropriate consistency measurement algorithms can be used for different types of label values. For example, for numerical label values ​​(Goal_Reasoning_Relevance, Goal_Clarity_Focus, Chain_Depth, Explicit_Chain_Depth, Reasoning_Knowledge_Dependency, Text_Knowledge_Level), a tolerance consistency check can be used: Let the scores of two annotations be s1 and s2, and the consistency condition be |s1-s2|≤1. This threshold setting considers the subjectivity of scoring and the randomness of API calls, ensuring quality while avoiding overly stringent requirements.

[0114] For list-type label values, two matching modes can be distinguished: strict matching and loose matching. For critical fields such as Goal_Type, a strict matching strategy is used: let the two labeled lists be L1 and L2, and the consistency condition be L1 = L2. For auxiliary fields such as Explicit_R_Type, a loose matching strategy is used: the consistency condition is... That is, the intersection of the two lists is not empty.

[0115] For the structured label value R_Type, a confidence-based consistency evaluation can be used. Let the R_Type of the two annotations be R1 = {(type 1i ,conf 1i )} and R2={(type 2j ,conf 2jThe system first calculates the average confidence level of each inference type across the two annotations, and then retains the types with an average confidence level of not less than 0.5. The consistency condition is that the two annotations have at least one common inference type and the difference in confidence level of that type does not exceed 0.3.

[0116] After the above multi-dimensional verification process, candidate sample data that passes the verification results of each dimension can be selected as target sample data for subsequent use in building a high-quality sample set to train the classifier.

[0117] Following the previous example, after obtaining the two sets of labeling information, A and B, for each candidate text, we can select the Goal_Reasoning_Relevance-A label value from the A set of labeling information and the Goal_Reasoning_Relevance-B label value from the B set of labeling information for each candidate text. Then, we can calculate the mean of the Goal_Reasoning_Relevance-A and Goal_Reasoning_Relevance-B label values ​​corresponding to each candidate text and check whether the mean is 0. Based on this process, we can remove candidate texts with a mean of 0 from the set, thus obtaining a candidate text set containing m candidate texts.

[0118] Furthermore, multi-dimensional consistency checks can be performed on the m candidate texts. During this process, the following tag values ​​can be selected from the two sets of tag information for each candidate text A and B: Goal_Reasoning_Relevance, Goal_Clarity_Focus, Chain_Depth, Explicit_Chain_Depth, Reasoning_Knowledge_Dependency, and Text_Knowledge_Level. Then, by calculating whether the absolute value of the difference between tag values ​​of the same type is less than or equal to 1, it is determined whether the candidate text passes the tolerance consistency check. Similarly, the Goal_Type tag value from the two sets of tag information for each candidate text A and B can be selected, and by checking whether the two are equal, and by selecting the Explicit_R_Type tag value, checking whether the intersection of the two is a non-empty set, it is determined whether the candidate text passes the matching dimension consistency check. Similarly, the structured R_Type tag value from the two sets of tag information for each candidate text A and B can be selected, and by calculating the average confidence score and determining its relationship with a threshold, it is determined whether the candidate text passes the confidence dimension consistency check.

[0119] Furthermore, after performing the above verification process on each of the m candidate texts, p candidate texts can be selected as target texts so that sample pairs can be constructed using the p candidate texts for training the classifier.

[0120] In summary, this innovative approach integrates statistical threshold comparison, strategy-based matching detection, and confidence level fusion technologies to perform multi-dimensional automated verification of numerical, list-based, and structured labels. It accurately identifies logical conflicts and numerical deviations in labels, significantly improving data consistency and numerical accuracy. This provides high-purity, high-reliability input data for AI training, effectively eliminating noise interference and greatly enhancing the decision-making accuracy and model generalization ability for complex analytical tasks.

[0121] Step S106: At least two sets of data annotation information corresponding to the target sample data are fused to obtain the target data annotation information corresponding to the target sample data.

[0122] Specifically, after selecting the verified target sample data, at least two sets of data annotation information corresponding to the target sample data can be fused to obtain the target data annotation information corresponding to the target sample data. Subsequently, sample pairs can be constructed by combining labels and samples for model training. The fusion of at least two sets of data annotation information can be understood as fusing label values ​​of the same type in each set of data annotation information. This fusion process includes, but is not limited to, random selection, averaging, and weighted averaging; this embodiment does not impose any limitations on these methods. The target data annotation information is the fused set of data annotation information, which contains multiple target label values.

[0123] Furthermore, when fusing at least two sets of data annotation information, different methods can be used to fuse different types of label values, thereby ensuring the accuracy of the fusion result. In this embodiment, fusing at least two sets of data annotation information corresponding to the target sample data to obtain the target data annotation information corresponding to the target sample data includes:

[0124] In the at least two sets of data annotation information corresponding to the target sample data, at least two target numerical label values, at least two target list label values, and at least two target structural label values ​​corresponding to the target sample data are determined; average aggregation is performed on the at least two target numerical label values, union aggregation is performed on the at least two target list label values, and priority aggregation is performed on the at least two target structural label values; target data annotation information corresponding to the target sample data is determined based on the aggregation results.

[0125] Specifically, average aggregation refers to the operation of averaging the target numerical label values. Union aggregation refers to the operation of unioning the target list-type label values. Priority aggregation refers to the operation of selecting the label values ​​with higher priority.

[0126] Based on this, the fusion of at least two sets of data annotation information can employ different methods for label values ​​of different dimensions. Specifically, from the at least two sets of data annotation information corresponding to the target sample data, at least two target numerical label values, at least two target list label values, and at least two target structural label values ​​can be determined. Then, average aggregation can be performed on the at least two target numerical label values, union aggregation can be performed on the at least two target list label values, and priority aggregation can be performed on the at least two target structural label values; thus, the target data annotation information corresponding to the target sample data can be determined based on the aggregation results.

[0127] In practical applications, for samples that pass the consistency check, an adaptive aggregation strategy can be used to generate the final annotation results. This strategy employs different aggregation methods based on the field type and the degree of consistency. For example, for numerical label values, an arithmetic mean aggregation can be used: final_score = (s1 + s2) / 2. This method can effectively reduce the impact of random fluctuations and improve the stability of the annotation results.

[0128] For list-type label values, a union aggregation strategy can be used: final_list = L1∪L2. This strategy can fully utilize the information from both annotations and reduce the risk of omissions. For the R_Type field, confidence-weighted aggregation can be used: for the same inference type, the final confidence is the average of the confidence scores of the two annotations; for a type that appears only in one annotation, if the confidence score is higher than 0.7, the type is retained and the confidence score is adjusted to 0.5.

[0129] For string-type label values ​​(such as Scene), a priority aggregation strategy can be used: first check if the two labels are consistent; if they are consistent, use the result of the first label; if they are inconsistent, use the result of the first label. This strategy is based on the assumption that the first label is usually more cautious, and can provide stable results even in cases of inconsistency.

[0130] Following the previous example, for any candidate text among p candidate texts, we can determine its corresponding two sets of labeling information, A and B. At this time, for numerical label values, we can calculate its corresponding target label value by arithmetic average; for list-type fields, we can calculate its corresponding target label value by union; and for string-type fields, we can select the label value in the first labeling result as the target label value. After completing the fusion of label values ​​of various types, we can obtain a set of labeling information corresponding to each candidate text, which contains the target label values ​​corresponding to 12 labels.

[0131] In summary, this innovative approach utilizes numerical averaging, list union, and structure priority aggregation strategies to intelligently fuse multiple sets of annotation information to generate unified target annotations. This effectively resolves label redundancy and conflict issues, significantly improving the standardization and ease of operation of data annotation. It provides concise and accurate input data for AI model training, significantly optimizing the execution efficiency and reliability of analysis tasks, and supporting high-precision decision support in complex scenarios.

[0132] Step S108: Construct target sample pairs based on the target sample data and the target data annotation information, and store the target sample pairs in the target sample set.

[0133] Specifically, after obtaining the target sample data and its corresponding target data annotation information, target sample pairs can be constructed by combining the target sample data and target data annotation information, and the target sample pairs can be stored in the target sample set. By doing so, a large number of high-quality sample pairs can be stored in the target sample set for use in training any classifier.

[0134] Furthermore, when constructing target sample pairs, they can be evaluated again, and high-quality sample pairs can be selected and stored in the set. In this embodiment, constructing target sample pairs based on the target sample data and the target data annotation information, and storing the target sample pairs in the target sample set, includes:

[0135] An initial sample pair is constructed based on the target sample data and the target data annotation information, and a quality assessment is performed on the initial sample pair. If the quality assessment result meets the assessment conditions, the initial sample pair is used as a target sample pair and stored in the target sample set. The sample distribution characteristics corresponding to the target sample set are used to update the verification conditions of the multi-dimensional verification.

[0136] Specifically, the initial sample pair refers to the sample pair that has not yet undergone quality assessment. Correspondingly, the quality assessment refers to the evaluation process that detects whether the quality of the sample pair meets the set conditions in various dimensions. Correspondingly, the sample distribution characteristics refer to the representation of the distribution of sample pairs in the target sample set. Based on this representation, the verification conditions of multi-dimensional verification can be updated, which can enable upstream processing to screen out sample pairs of different types or different distributions, thereby improving the richness of the target sample set.

[0137] Based on this, a quality assessment can be performed before storing sample pairs into the set. At this time, initial sample pairs can be constructed based on target sample data and target data annotation information, and then a quality assessment can be performed on the initial sample pairs. If the quality assessment results meet the assessment conditions, the initial sample pairs can be used as target sample pairs and stored in the target sample set. Furthermore, the sample distribution characteristics corresponding to the target sample set can be used to update the verification conditions of multi-dimensional verification, thereby making the sample construction stage faster and more accurate.

[0138] In practical applications, a multi-layered quality assessment mechanism can be set up to monitor annotation quality in real time and identify anomalies. At the sample level, a consistency score can be calculated for each sample, which comprehensively considers the consistency degree across various dimensions. At the batch level, metrics such as pass rate, average consistency score, and divergence distribution across various dimensions can be statistically analyzed. When the pass rate falls below a preset threshold (e.g., 70%), an anomaly warning can be triggered.

[0139] Furthermore, to enhance the richness of the sample set, a dynamic threshold adjustment mechanism can be implemented to automatically adjust the consistency test threshold based on the distribution characteristics of historical data. This mechanism, based on sliding window statistics, can adapt to the varying annotation difficulty of different text types. When a system detects low consistency in the annotation of a certain type of text, the system will appropriately relax the consistency requirements for that type of text, balancing annotation quality and data output efficiency.

[0140] Following the previous example, after obtaining the target text and its corresponding annotation information, the quality of each sample pair consisting of the target text and its annotation information can be evaluated, and high-quality sample pairs can be stored in the target sample set. Then, based on the distribution of the target sample set, the threshold set during multi-dimensional verification can be dynamically adjusted, thereby improving the uniformity of the sample distribution in the target sample set. Subsequently, the sample pairs contained in the target sample set can be used to train a classifier for any scenario.

[0141] In summary, this approach intelligently selects high-quality sample pairs through a quality assessment mechanism and dynamically optimizes verification conditions based on sample distribution characteristics, achieving closed-loop iteration of data quality and adaptive system upgrades. This effectively improves the reliability and task matching of sample pairs and significantly reduces reliance on manual calibration. It provides a continuously evolving, high-quality data foundation for AI training, enhancing the system's accurate decision-making capabilities and long-term operational stability in dynamic data environments.

[0142] Furthermore, after obtaining the target sample pairs, considering that all target sample pairs in the set contain multiple label values, the classifier training can meet the training requirements of multivariate classifiers or binary classifiers. In this embodiment, after the step of storing the target sample pairs into the target sample set is performed, the method further includes:

[0143] The target sample pairs contained in the target sample set are preprocessed, and a multidimensional feature space is constructed based on the preprocessing results. A binary classifier is trained based on the multidimensional feature space. Alternatively, a multivariate classifier containing encoding and decoding units is determined, and the multivariate classifier is trained using the target sample pairs contained in the target sample set. The input of the trained binary classifier is the data to be processed, and the output is either a reasoning label or a non-reasoning label. The input of the trained multivariate classifier is the data to be processed, and the output is labels of various data types.

[0144] Specifically, a binary classifier is a classifier that can process any data and output either a reasoned label or a non-reasoned label, while a multivariate classifier is a classifier that can process any data and output an accurate label.

[0145] Therefore, when training a classifier, considering that sample pairs contain multi-dimensional label values, a binary classifier or a multi-variable classifier can be trained according to requirements. For a binary classifier, the target sample pairs in the target sample set can be preprocessed, and a multi-dimensional feature space can be constructed based on the preprocessing results. The binary classifier can then be trained based on this multi-dimensional feature space. The input of the trained binary classifier is the data to be processed, and the output is either an inferred label or a non-inferred label. For a multi-variable classifier, a multi-variable classifier containing encoding and decoding units can be determined. The multi-variable classifier can then be trained using the target sample pairs in the target sample set. The input of the trained multi-variable classifier is the data to be processed, and the output is labels of various data types.

[0146] In practical applications, when training a binary classifier based on target sample pairs contained in the target sample set, a method combining feature engineering and deep learning can be used to construct the binary classifier through multi-indicator fusion. This method first performs in-depth statistical analysis on the labeled data, using exploratory data analysis (EDA) techniques to identify the distribution characteristics, correlations, and discriminative abilities of each dimension of the indicators. The system calculates the distributional differences of each indicator in the inference and non-inference data, and uses statistical methods such as the Kolmogorov-Smirnov test and the Mann-Whitney U test to evaluate the discriminative ability of the indicators.

[0147] Based on statistical analysis results, a feature engineering strategy that integrates multiple indicators can be designed. This strategy considers not only the absolute values ​​of each indicator but also the interactions and combined effects between them. Specifically, a multi-dimensional feature space can be constructed, including the original indicator values, the ratios between indicators, the second-order interaction terms of the indicators, and composite indicators based on domain knowledge. For example, the system might calculate the product of Chain_Depth and Goal_Reasoning_Relevance as a composite indicator of reasoning complexity.

[0148] The recall boundary is determined using a threshold optimization method based on ROC curve analysis. The system calculates precision, recall, and F1 score for different threshold combinations on the validation set, and finds the optimal threshold combination through grid search. The threshold optimization process considers trade-offs in business requirements, and the weights of precision and recall can be adjusted according to the actual application scenario.

[0149] The classifier can be trained using qwen2.5-0.5b-instruct as the base model, with task-related classification heads added on top. The training process employs autoregressive language modeling, transforming the classification task into a text generation task. Specifically, the system represents the label of inference data as "1" and the label of non-inference data as "0," and the model outputs the classification result by predicting the next token.

[0150] The loss function uses standard cross-entropy loss, but incorporates a class weight balancing mechanism to handle data imbalance. During training, regularization techniques such as learning rate decay, gradient clipping, and early stopping are employed to prevent overfitting. The system uses k-fold cross-validation to evaluate model performance, ensuring the reliability of the results.

[0151] This approach improves training efficiency, reduces model complexity, and maintains inference speed. Furthermore, because binary classification tasks are relatively simple, the model converges quickly, significantly shortening training time. Simultaneously, due to the well-defined task, the model's accuracy is typically high, particularly excelling in false positive control. The classifier obtained through this training method is particularly suitable for applications with high real-time requirements and limited computational resources.

[0152] For training multi-dimensional classifiers, a multi-task learning framework can be employed, constructing an independent prediction task for each key dimension indicator. This method decomposes the inference data recognition problem into multiple interconnected sub-tasks, each responsible for predicting a specific dimension indicator. This decomposition strategy not only obtains richer prediction information but also leverages the correlation between tasks to improve overall prediction performance.

[0153] A neural network architecture consisting of a shared encoder and multiple task-specific decoders is constructed. The shared encoder, based on the qwen2.5-0.5b-instruct model, is responsible for extracting a general semantic representation of the text. Each task-specific decoder is customized for predicting a single-dimensional metric. For numerical metrics (such as Goal_Reasoning_Relevance, Chain_Depth, etc.), the decoder employs a multi-classification architecture, transforming the regression problem into a classification problem. For structured metrics (such as R_Type), the decoder uses a multi-label classification architecture, capable of simultaneously predicting multiple inference types and their confidence levels.

[0154] The multi-task learning training strategy employs a joint optimization method, designing a comprehensive loss function that is a weighted sum of the losses from each task. The weights are set considering the importance, difficulty, and relevance of the tasks. A dynamic weight adjustment strategy is used, automatically adjusting the weights based on the convergence of each task during training to ensure that all tasks are adequately trained.

[0155] To handle the correlation between tasks, a task-specific regularization term is introduced to encourage related tasks to learn similar feature representations. For example, the Chain_Depth and Explicit_Chain_Depth tasks are strongly correlated, so a correlation constraint is added to the loss function to encourage these two tasks to learn consistent feature representations.

[0156] The training process employs a course-based learning strategy, first training on relatively simple tasks (such as Goal_Type prediction), and then gradually introducing more complex tasks (such as R_Type prediction). This strategy helps the model converge better and avoids the negative impact of complex tasks on simple tasks.

[0157] During the inference data recall phase, predictors for all tasks run in parallel to obtain detailed prediction results for each dimension. Then, a configurable fusion strategy is used to combine multiple prediction results into a final recall decision. The fusion strategy can be a simple combination of thresholds or a pre-trained fusion model. Crucially, when the recall boundary needs adjustment, only the parameters of the fusion strategy need to be modified, without retraining the underlying prediction model.

[0158] Training a multi-dimensional classifier ensures rich output, high system flexibility, and strong interpretability. Through multi-task learning, detailed predictive information for each dimension can be obtained, which is of significant value for data analysis, quality control, and business decision-making. The system is highly adaptable to changes in recall requirements, enabling rapid adjustments without retraining. Furthermore, the multi-dimensional prediction results provide good interpretability, facilitating understanding and debugging.

[0159] In summary, by constructing a multi-dimensional feature space and training customized binary / multi-class classifiers, automated and accurate identification of inference labels and multiple data types is achieved. This significantly improves classification accuracy and real-time processing capabilities, while reducing the risk of misjudgment and computational resource consumption. It provides a lightweight and robust classification engine for intelligent systems, effectively supporting dynamic data processing needs in high-concurrency scenarios and optimizing decision-making efficiency and long-term operational stability.

[0160] Based on this, once the trained classifier is capable of classifying both reasoning data and non-reasoning data, it can be used to filter reasoning data. Training the model in this way can enhance its reasoning ability. This embodiment also includes:

[0161] The trained binary or multi-class classifier classifies the data to be processed in the dataset, and determines the inference data in the dataset based on the classification results. The inference data is then used to train the business model until the target business model that meets the training stopping condition is obtained.

[0162] Specifically, the dataset to be processed refers to a collection containing a large amount of data that is not clearly defined as either inference data or non-inference data. Inference data specifically refers to data in the dataset to be processed that contains inference logic. The business model specifically refers to the large language model that needs to be deployed in actual business scenarios, such as question-answering models, customer service interaction models, text processing models, recommendation models, etc. This embodiment does not impose any limitations. Correspondingly, the training stopping condition specifically refers to the conditions for stopping the training of the business model, including but not limited to loss value comparison conditions, validation set verification conditions, or iteration count conditions.

[0163] Based on this, the trained classifier can effectively distinguish between inference data and non-inference data. Furthermore, when the classifier is a multi-classifier, it can accurately distinguish the correct label corresponding to the inference data. Therefore, after deploying it to a business scenario, the unprocessed data in the dataset to be processed can be classified based on the trained binary or multi-classifier. This allows the inference data to be determined from the dataset based on the classification results. Subsequently, the business model can be trained using the inference data until the target business model that meets the training stopping condition is obtained. This enables the deployment of models with higher prediction accuracy to complete business services in business scenarios.

[0164] For example, when a content application platform needs to train a model that can interact with users through question and answer, the trained classifier can be used to classify the collected text to filter out inference text containing reasoning logic from multiple texts. Inference text can enhance the model's reasoning ability. Therefore, inference text can be used to train the question and answer model. After the trained text model is deployed to the business scenario, it can quickly provide accurate answers to the questions raised by users to meet their question and answer needs.

[0165] In summary, by driving a closed loop of precise data selection and adaptive training of business models through an intelligent classifier, a fully automated process from data processing to model deployment is achieved. This effectively improves training efficiency and model generalization ability, significantly shortens the development cycle, and reduces resource consumption. It ensures continuous optimization of the business model to match dynamic business scenarios, provides high-precision, low-latency decision support, and greatly enhances the system's real-time response capability and long-term application value in complex environments.

[0166] The data processing method provided in this embodiment can support an automated inference data identification and recall system. First, based on the essential characteristics of inference, a theoretically complete and practical inference labeling system is constructed. This system needs to be clear, reasonable, and API-friendly, effectively distinguishing different types and intensities of inference patterns. Second, a high-quality automated labeling method is developed, ensuring the reliability and consistency of API labeling through a quality control mechanism combining dual labeling and consistency verification. Third, based on high-quality labeled data, a lightweight classifier capable of accurately identifying inference data is trained. This classifier needs to achieve high recall while maintaining high accuracy. Finally, large-scale inference data recall is achieved, efficiently retrieving inference data from massive amounts of general web page text, providing effective support for improving the inference capabilities of pre-trained models.

[0167] This significantly reduces the cost of inference data retrieval while greatly improving data quality and retrieval effectiveness. More importantly, the system has good scalability and maintainability, enabling it to adapt to inference data retrieval needs of different scales and domains, providing reliable technical support for improving the inference capabilities of large models.

[0168] The data processing method provided in this embodiment, in order to reduce the cost of sample construction and improve the scalability of sample construction, can first label the candidate sample data in the candidate sample data set according to the data labeling strategy after determining the candidate sample data set. At this time, at least two sets of data labeling information corresponding to each candidate sample data can be determined. Each set of data labeling information contains multiple label values, and each label value represents a different type. On this basis, multi-dimensional verification can be performed on the at least two sets of data labeling information corresponding to each candidate sample data, so as to determine the target sample data that passes the verification in the candidate sample data set based on the multi-dimensional verification results. Since each sample data corresponds to at least two sets of data labeling information, the at least two sets of data labeling information corresponding to the target sample data can be fused to obtain the target data labeling information corresponding to the target sample data. Based on this, target sample pairs can be constructed based on the target sample data and target data labeling information, and the target sample pairs can be stored in the target sample set. This allows the high-quality target sample set to be used for model training. This data processing method can not only reduce the cost of data labeling, but also ensure the quality of data labeling, adapt to different labeling scenarios, and thus improve the efficiency of training the model using the target sample set.

[0169] See Figure 2 , Figure 2 A flowchart of a classifier training method according to an embodiment of this specification is shown, which specifically includes the following steps.

[0170] Step S202: Label the candidate sample data in the candidate sample data set according to the data labeling strategy, and obtain at least two sets of data labeling information corresponding to each candidate sample data, wherein each set of data labeling information contains multiple label values.

[0171] Step S204: Perform multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results.

[0172] Step S206: At least two sets of data annotation information corresponding to the target sample data are fused to obtain the target data annotation information corresponding to the target sample data.

[0173] Step S208: Construct target sample pairs based on the target sample data and the target data annotation information, and store the target sample pairs in the target sample set.

[0174] Step S210: Train the classifier using the target sample pairs contained in the target sample set to obtain the target classifier.

[0175] The classifier training method provided in this embodiment, and any content not described in detail herein, can be referred to the descriptions in the above embodiments. This embodiment will not elaborate further here.

[0176] Specifically, a classifier refers to a binary classifier or a multi-class classifier. A binary classifier is a classifier that can process any data and output either a reasoned label or a non-reasoned label. A multi-class classifier is a classifier that can process any data and output an accurate label.

[0177] Based on this, after obtaining the target sample set, considering that the sample pairs contain multi-dimensional label values, a binary classifier or a multi-class classifier can be trained according to requirements. For a binary classifier, the target sample pairs in the target sample set can be preprocessed, and a multi-dimensional feature space can be constructed based on the preprocessing results. The binary classifier can then be trained based on this multi-dimensional feature space. The input of the trained binary classifier is the data to be processed, and the output is either an inferred label or a non-inferred label. For a multi-class classifier, a multi-classifier containing encoding and decoding units can be determined. The multi-classifier can then be trained using the target sample pairs in the target sample set. The input of the trained multi-class classifier is the data to be processed, and the output is labels of various data types.

[0178] In practical applications, when training a binary classifier based on target sample pairs contained in the target sample set, a method combining feature engineering and deep learning can be used to construct the binary classifier through multi-indicator fusion. This method first performs in-depth statistical analysis on the labeled data, using exploratory data analysis (EDA) techniques to identify the distribution characteristics, correlations, and discriminative abilities of each dimension of the indicators. The system calculates the distributional differences of each indicator in the inference and non-inference data, and uses statistical methods such as the Kolmogorov-Smirnov test and the Mann-Whitney U test to evaluate the discriminative ability of the indicators.

[0179] Based on statistical analysis results, a feature engineering strategy that integrates multiple indicators can be designed. This strategy considers not only the absolute values ​​of each indicator but also the interactions and combined effects between them. Specifically, a multi-dimensional feature space can be constructed, including the original indicator values, the ratios between indicators, the second-order interaction terms of the indicators, and composite indicators based on domain knowledge. For example, the system might calculate the product of Chain_Depth and Goal_Reasoning_Relevance as a composite indicator of reasoning complexity.

[0180] The recall boundary is determined using a threshold optimization method based on ROC curve analysis. The system calculates precision, recall, and F1 score for different threshold combinations on the validation set, and finds the optimal threshold combination through grid search. The threshold optimization process considers trade-offs in business requirements, and the weights of precision and recall can be adjusted according to the actual application scenario.

[0181] The classifier can be trained using qwen2.5-0.5b-instruct as the base model, with task-related classification heads added on top. The training process employs autoregressive language modeling, transforming the classification task into a text generation task. Specifically, the system represents the label of inference data as "1" and the label of non-inference data as "0," and the model outputs the classification result by predicting the next token.

[0182] The loss function uses standard cross-entropy loss, but incorporates a class weight balancing mechanism to handle data imbalance. During training, regularization techniques such as learning rate decay, gradient clipping, and early stopping are employed to prevent overfitting. The system uses k-fold cross-validation to evaluate model performance, ensuring the reliability of the results.

[0183] This approach improves training efficiency, reduces model complexity, and maintains inference speed. Furthermore, because binary classification tasks are relatively simple, the model converges quickly, significantly shortening training time. Simultaneously, due to the well-defined task, the model's accuracy is typically high, particularly excelling in false positive control. The classifier obtained through this training method is particularly suitable for applications with high real-time requirements and limited computational resources.

[0184] For training multi-dimensional classifiers, a multi-task learning framework can be employed, constructing an independent prediction task for each key dimension indicator. This method decomposes the inference data recognition problem into multiple interconnected sub-tasks, each responsible for predicting a specific dimension indicator. This decomposition strategy not only obtains richer prediction information but also leverages the correlation between tasks to improve overall prediction performance.

[0185] A neural network architecture consisting of a shared encoder and multiple task-specific decoders is constructed. The shared encoder, based on the qwen2.5-0.5b-instruct model, is responsible for extracting a general semantic representation of the text. Each task-specific decoder is customized for predicting a single-dimensional metric. For numerical metrics (such as Goal_Reasoning_Relevance, Chain_Depth, etc.), the decoder employs a multi-classification architecture, transforming the regression problem into a classification problem. For structured metrics (such as R_Type), the decoder uses a multi-label classification architecture, capable of simultaneously predicting multiple inference types and their confidence levels.

[0186] The multi-task learning training strategy employs a joint optimization method, designing a comprehensive loss function that is a weighted sum of the losses from each task. The weights are set considering the importance, difficulty, and relevance of the tasks. A dynamic weight adjustment strategy is used, automatically adjusting the weights based on the convergence of each task during training to ensure that all tasks are adequately trained.

[0187] To handle the correlation between tasks, a task-specific regularization term is introduced to encourage related tasks to learn similar feature representations. For example, the Chain_Depth and Explicit_Chain_Depth tasks are strongly correlated, so a correlation constraint is added to the loss function to encourage these two tasks to learn consistent feature representations.

[0188] The training process employs a course-based learning strategy, first training on relatively simple tasks (such as Goal_Type prediction), and then gradually introducing more complex tasks (such as R_Type prediction). This strategy helps the model converge better and avoids the negative impact of complex tasks on simple tasks.

[0189] During the inference data recall phase, predictors for all tasks run in parallel to obtain detailed prediction results for each dimension. Then, a configurable fusion strategy is used to combine multiple prediction results into a final recall decision. The fusion strategy can be a simple combination of thresholds or a pre-trained fusion model. Crucially, when the recall boundary needs adjustment, only the parameters of the fusion strategy need to be modified, without retraining the underlying prediction model.

[0190] Training a multi-dimensional classifier ensures rich output, high system flexibility, and strong interpretability. Through multi-task learning, detailed predictive information for each dimension can be obtained, which is of significant value for data analysis, quality control, and business decision-making. The system is highly adaptable to changes in recall requirements, enabling rapid adjustments without retraining. Furthermore, the multi-dimensional prediction results provide good interpretability, facilitating understanding and debugging.

[0191] In summary, by constructing a multi-dimensional feature space and training customized binary / multi-class classifiers, automated and accurate identification of inference labels and multiple data types is achieved. This significantly improves classification accuracy and real-time processing capabilities, while reducing the risk of misjudgment and computational resource consumption. It provides a lightweight and robust classification engine for intelligent systems, effectively supporting dynamic data processing needs in high-concurrency scenarios and optimizing decision-making efficiency and long-term operational stability.

[0192] See Figure 3 , Figure 3 A flowchart of another data processing method provided according to an embodiment of this specification is shown, which specifically includes the following steps.

[0193] Step S302: Obtain the data set to be processed.

[0194] Step S304: Input the data to be processed contained in the data set to be processed into the target classifier for classification processing, and determine the inference data in the data set to be processed according to the classification processing result. The target classifier is obtained by the above method.

[0195] Step S306: Use the inference data to train the business model until a target business model that meets the training stopping condition is obtained.

[0196] This embodiment provides another data processing method. For any content not described in detail, please refer to the description in the above embodiments. This embodiment will not elaborate further here.

[0197] Specifically, the dataset to be processed refers to a collection containing a large amount of data that is not clearly defined as either inference data or non-inference data. Inference data specifically refers to data in the dataset to be processed that contains inference logic. The business model specifically refers to the large language model that needs to be deployed in actual business scenarios, such as question-answering models, customer service interaction models, text processing models, recommendation models, etc. This embodiment does not impose any limitations. Correspondingly, the training stopping condition specifically refers to the conditions for stopping the training of the business model, including but not limited to loss value comparison conditions, validation set verification conditions, or iteration count conditions.

[0198] Based on this, the trained classifier can effectively distinguish between inference data and non-inference data. Furthermore, when the classifier is a multi-classifier, it can accurately distinguish the correct label corresponding to the inference data. Therefore, after deploying it to a business scenario, the unprocessed data in the dataset to be processed can be classified based on the trained binary or multi-classifier. This allows the inference data to be determined from the dataset based on the classification results. Subsequently, the business model can be trained using the inference data until the target business model that meets the training stopping condition is obtained. This enables the deployment of models with higher prediction accuracy to complete business services in business scenarios.

[0199] For example, when a content application platform needs to train a model that can interact with users through question and answer, the trained classifier can be used to classify the collected text to filter out inference text containing reasoning logic from multiple texts. Inference text can enhance the model's reasoning ability. Therefore, inference text can be used to train the question and answer model. After the trained text model is deployed to the business scenario, it can quickly provide accurate answers to the questions raised by users to meet their question and answer needs.

[0200] In summary, by driving a closed loop of precise data selection and adaptive training of business models through an intelligent classifier, a fully automated process from data processing to model deployment is achieved. This effectively improves training efficiency and model generalization ability, significantly shortens the development cycle, and reduces resource consumption. It ensures continuous optimization of the business model to match dynamic business scenarios, provides high-precision, low-latency decision support, and greatly enhances the system's real-time response capability and long-term application value in complex environments.

[0201] The following is in conjunction with the appendix Figure 4 Taking the application of the data processing method provided in this specification in a question-and-answer interaction scenario as an example, the data processing method will be further explained. Figure 4 A flow chart of a data processing method provided in one embodiment of this specification is shown.

[0202] Data processing methods are applied to content application platforms, which include client and server sides. The client is the terminal device held by the user to interact with the AI ​​model in question-and-answer sessions, while the server specifically refers to the platform that provides question-and-answer services to the user. For example... Figure 4 As shown:

[0203] On the server side, the candidate sample data in the candidate sample dataset is labeled according to the data labeling strategy, and at least two sets of data labeling information are obtained for each candidate sample data. Each set of data labeling information contains multiple label values.

[0204] On the server side, multi-dimensional verification is performed on at least two sets of data annotation information corresponding to each candidate sample data, and the target sample data is determined from the candidate sample data set based on the multi-dimensional verification results.

[0205] On the server side, from at least two sets of data annotation information corresponding to the target sample data, at least two target numerical label values, at least two target list label values, and at least two target structural label values ​​are determined for the target sample data.

[0206] On the server side, average aggregation is performed on at least two target numerical tag values, union aggregation is performed on at least two target list tag values, and priority aggregation is performed on at least two target structure tag values.

[0207] On the server side, the target data annotation information corresponding to the target sample data is determined based on the aggregation results.

[0208] On the server side, initial sample pairs are constructed based on the target sample data and target data annotation information, and the quality of the initial sample pairs is evaluated.

[0209] On the server side, if the quality assessment results meet the assessment conditions, the initial sample pair is used as the target sample pair and stored in the target sample set.

[0210] On the server side, the target sample pairs contained in the target sample set are used to train the classifier to obtain the target classifier.

[0211] On the server side, the data to be processed contained in the dataset to be processed is input into the target classifier for classification processing, and the inference data is determined from the dataset to be processed based on the classification processing results.

[0212] On the server side, the question-answering model is trained using inference data until the target question-answering model that meets the training stopping condition is obtained.

[0213] The client sends question text to the server through a question-and-answer interaction page.

[0214] On the server side, the question text is input into the target question-answering model for processing, the answer text is obtained, and the answer text is sent to the client.

[0215] The client displays the answer text through a dialog box on the question-and-answer interaction page.

[0216] Furthermore, to ensure the uniqueness of candidate sample data within the candidate sample dataset and avoid duplicate sample construction, the candidate sample dataset can be constructed by calculating hash values. In this embodiment, before the step of labeling the candidate sample data in the candidate sample dataset according to the data labeling strategy to obtain at least two sets of data labeling information corresponding to each candidate sample data, the following steps are also included:

[0217] Initial sample data is acquired and preprocessed to obtain a word sequence. If the number of words in the word sequence exceeds a set threshold, the word sequence is truncated to obtain a target word sequence. The target word sequence is processed using a hash algorithm to obtain sequence identifier information, and the initial sample data is stored in a candidate sample data set according to the sequence identifier information. The sequence identifier information is used to deduplicate candidate sample data in the candidate sample data set.

[0218] Furthermore, the annotation process can be quickly completed using a model. In this embodiment, the annotation of any candidate sample data in the candidate sample data set includes:

[0219] A data annotation model is determined according to a data annotation strategy, and the prompt words corresponding to the first candidate sample data are determined. The data annotation model is used to annotate the first candidate sample data at least twice according to the prompt words to obtain at least two sets of data annotation information corresponding to the first candidate sample data. Among them, the multiple label values ​​contained in each set of data annotation information are associated with the target analysis category, the reasoning behavior category and / or the context analysis category, and the target analysis category, the reasoning behavior category and / or the context analysis category correspond to at least one of the multiple label values.

[0220] Furthermore, in order to enable the subsequent construction of sample pairs based on samples containing inference data, sample filtering can be performed in advance based on inference requirement labels. In this embodiment, before the multi-dimensional verification step of at least two sets of data annotation information corresponding to each candidate sample data is executed, the following steps are also included:

[0221] In each candidate sample data set, at least two inference requirement label values ​​are extracted from at least two sets of data annotation information. These inference requirement label values ​​are then fused. Candidate sample data with the first field value associated with the fused inference requirement label value are deleted from the candidate sample data set. A target candidate sample data set is determined based on the deletion result. The step of performing multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data set, and determining the target sample data in the candidate sample data set based on the multi-dimensional verification results, includes: performing multi-dimensional verification on at least two sets of data annotation information corresponding to the candidate sample data included in the target candidate sample data set, and determining the target sample data in the target candidate sample data set based on the multi-dimensional verification results.

[0222] Furthermore, when performing multi-dimensional validation for each candidate sample data, tolerance dimension, matching dimension, and confidence dimension can be combined to validate different types of label values. In this embodiment, the multi-dimensional validation of any candidate sample data includes:

[0223] From at least two sets of data annotation information corresponding to the second candidate sample data, at least two numerical label values, at least two list label values, and at least two structural label values ​​corresponding to the second candidate sample data are determined; the at least two numerical label values ​​are validated in the tolerance dimension, the at least two list label values ​​are validated in the matching dimension, and the at least two structural label values ​​are validated in the confidence dimension; if the validation results of the tolerance dimension, the matching dimension, and the confidence dimension all meet the validation conditions, the second candidate sample data is used as the target sample data.

[0224] Based on this, the verification of the at least two numerical label values ​​in the tolerance dimension includes: constructing statistical label values ​​based on the at least two numerical label values, comparing the statistical label values ​​with a set statistical threshold, and determining the verification result of the tolerance dimension based on the comparison result;

[0225] The step of validating the at least two list-type label values ​​in the matching dimension includes: when the at least two list-type label values ​​are associated with a first matching strategy, detecting whether the at least two list-type label values ​​are equal based on the first matching strategy; when the at least two list-type label values ​​are associated with a second matching strategy, detecting whether the intersection of the at least two list-type label values ​​is an empty set based on the second matching strategy; and determining the validation result of the matching dimension based on the detection result.

[0226] The step of verifying the confidence dimension for the at least two structural label values ​​includes: determining the confidence level corresponding to each of the at least two structural label values; fusing the confidence levels corresponding to the at least two structural label values ​​to obtain an average confidence level; comparing the average confidence level with a preset confidence threshold; and determining the verification result of the confidence dimension based on the comparison result.

[0227] Furthermore, when fusing at least two sets of data annotation information, different methods can be used to fuse different types of label values, thereby ensuring the accuracy of the fusion result. In this embodiment, fusing at least two sets of data annotation information corresponding to the target sample data to obtain the target data annotation information corresponding to the target sample data includes:

[0228] In the at least two sets of data annotation information corresponding to the target sample data, at least two target numerical label values, at least two target list label values, and at least two target structural label values ​​corresponding to the target sample data are determined; average aggregation is performed on the at least two target numerical label values, union aggregation is performed on the at least two target list label values, and priority aggregation is performed on the at least two target structural label values; target data annotation information corresponding to the target sample data is determined based on the aggregation results.

[0229] Furthermore, when constructing target sample pairs, they can be evaluated again, and high-quality sample pairs can be selected and stored in the set. In this embodiment, constructing target sample pairs based on the target sample data and the target data annotation information, and storing the target sample pairs in the target sample set, includes:

[0230] An initial sample pair is constructed based on the target sample data and the target data annotation information, and a quality assessment is performed on the initial sample pair. If the quality assessment result meets the assessment conditions, the initial sample pair is used as a target sample pair and stored in the target sample set. The sample distribution characteristics corresponding to the target sample set are used to update the verification conditions of the multi-dimensional verification.

[0231] Furthermore, after obtaining the target sample pairs, considering that all target sample pairs in the set contain multiple label values, the classifier training can meet the training requirements of multivariate classifiers or binary classifiers. In this embodiment, after the step of storing the target sample pairs into the target sample set is performed, the method further includes:

[0232] The target sample pairs contained in the target sample set are preprocessed, and a multidimensional feature space is constructed based on the preprocessing results. A binary classifier is trained based on the multidimensional feature space. Alternatively, a multivariate classifier containing encoding and decoding units is determined, and the multivariate classifier is trained using the target sample pairs contained in the target sample set. The input of the trained binary classifier is the data to be processed, and the output is either a reasoning label or a non-reasoning label. The input of the trained multivariate classifier is the data to be processed, and the output is labels of various data types.

[0233] Based on this, once the trained classifier is capable of classifying both reasoning data and non-reasoning data, it can be used to filter reasoning data. Training the model in this way can enhance its reasoning ability. This embodiment also includes:

[0234] The trained binary or multi-class classifier classifies the data to be processed in the dataset, and determines the inference data in the dataset based on the classification results. The inference data is then used to train the business model until the target business model that meets the training stopping condition is obtained.

[0235] In summary, to reduce sample construction costs and improve scalability, after determining the candidate sample dataset, the candidate sample data in the dataset can be labeled according to a data labeling strategy. At this point, at least two sets of data labeling information can be determined for each candidate sample data. Each set of data labeling information contains multiple label values, and each label value represents a different type. Based on this, multi-dimensional verification can be performed on the at least two sets of data labeling information corresponding to each candidate sample data. Based on the multi-dimensional verification results, the verified target sample data can be determined from the candidate sample dataset. Since each sample data corresponds to at least two sets of data labeling information, the at least two sets of data labeling information corresponding to the target sample data can be fused to obtain the target data labeling information corresponding to the target sample data. Based on this, target sample pairs can be constructed based on the target sample data and target data labeling information, and the target sample pairs can be stored in the target sample set. This allows for the subsequent use of a high-quality target sample set for model training. This data processing method not only reduces data labeling costs but also ensures data labeling quality, adapting to different labeling scenarios and thus improving the efficiency of training the model using the target sample set.

[0236] Corresponding to the above method embodiments, this specification also provides data processing apparatus embodiments. Figure 5 A schematic diagram of the structure of a data processing apparatus according to one embodiment of this specification is shown. Figure 5 As shown, the device includes:

[0237] The annotation module 502 is configured to annotate the candidate sample data in the candidate sample dataset according to the data annotation strategy, and obtain at least two sets of data annotation information corresponding to each candidate sample data, wherein each set of data annotation information contains multiple label values;

[0238] The verification module 504 is configured to perform multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results.

[0239] The fusion module 506 is configured to fuse at least two sets of data annotation information corresponding to the target sample data to obtain target data annotation information corresponding to the target sample data.

[0240] The construction module 508 is configured to construct target sample pairs based on the target sample data and the target data annotation information, and store the target sample pairs in the target sample set.

[0241] In an optional embodiment, before the step of annotating the candidate sample data in the candidate sample dataset according to the data annotation strategy to obtain at least two sets of data annotation information corresponding to each candidate sample data is performed, the method further includes:

[0242] Initial sample data is acquired and preprocessed to obtain a word sequence. If the number of words in the word sequence exceeds a set threshold, the word sequence is truncated to obtain a target word sequence. The target word sequence is processed using a hash algorithm to obtain sequence identifier information, and the initial sample data is stored in a candidate sample data set according to the sequence identifier information. The sequence identifier information is used to deduplicate candidate sample data in the candidate sample data set.

[0243] In an optional embodiment, the annotation of any candidate sample data in the candidate sample data set includes:

[0244] A data annotation model is determined according to a data annotation strategy, and the prompt words corresponding to the first candidate sample data are determined. The data annotation model is used to annotate the first candidate sample data at least twice according to the prompt words to obtain at least two sets of data annotation information corresponding to the first candidate sample data. Among them, the multiple label values ​​contained in each set of data annotation information are associated with the target analysis category, the reasoning behavior category and / or the context analysis category, and the target analysis category, the reasoning behavior category and / or the context analysis category correspond to at least one of the multiple label values.

[0245] In an optional embodiment, before performing the multi-dimensional verification step for at least two sets of data annotation information corresponding to each candidate sample data, the method further includes:

[0246] In each candidate sample data set, at least two inference requirement label values ​​are extracted from at least two sets of data annotation information. These inference requirement label values ​​are then fused. Candidate sample data with the first field value associated with the fused inference requirement label value are deleted from the candidate sample data set. A target candidate sample data set is determined based on the deletion result. The step of performing multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data set, and determining the target sample data in the candidate sample data set based on the multi-dimensional verification results, includes: performing multi-dimensional verification on at least two sets of data annotation information corresponding to the candidate sample data included in the target candidate sample data set, and determining the target sample data in the target candidate sample data set based on the multi-dimensional verification results.

[0247] In one optional embodiment, multi-dimensional verification of any candidate sample data includes:

[0248] From at least two sets of data annotation information corresponding to the second candidate sample data, at least two numerical label values, at least two list label values, and at least two structural label values ​​corresponding to the second candidate sample data are determined; the at least two numerical label values ​​are validated in the tolerance dimension, the at least two list label values ​​are validated in the matching dimension, and the at least two structural label values ​​are validated in the confidence dimension; if the validation results of the tolerance dimension, the matching dimension, and the confidence dimension all meet the validation conditions, the second candidate sample data is used as the target sample data.

[0249] In an optional embodiment, the validation of the at least two numerical label values ​​in the tolerance dimension includes:

[0250] Statistical label values ​​are constructed based on the at least two numerical label values. These statistical label values ​​are compared with a set statistical threshold, and the verification result for the tolerance dimension is determined based on the comparison result. Specifically, the verification of the at least two list-type label values ​​in the matching dimension includes: when the at least two list-type label values ​​are associated with a first matching strategy, detecting whether the at least two list-type label values ​​are equal based on the first matching strategy; when the at least two list-type label values ​​are associated with a second matching strategy, detecting whether the intersection of the at least two list-type label values ​​is an empty set based on the second matching strategy; and determining the verification result for the matching dimension based on the detection result. The verification of the at least two structural label values ​​in the confidence dimension includes: determining the confidence level corresponding to each of the at least two structural label values; fusing the confidence levels corresponding to each of the at least two structural label values ​​to obtain an average confidence level; comparing the average confidence level with a preset confidence threshold, and determining the verification result for the confidence dimension based on the comparison result.

[0251] In an optional embodiment, fusing at least two sets of data annotation information corresponding to the target sample data to obtain target data annotation information corresponding to the target sample data includes:

[0252] In the at least two sets of data annotation information corresponding to the target sample data, at least two target numerical label values, at least two target list label values, and at least two target structural label values ​​corresponding to the target sample data are determined; average aggregation is performed on the at least two target numerical label values, union aggregation is performed on the at least two target list label values, and priority aggregation is performed on the at least two target structural label values; target data annotation information corresponding to the target sample data is determined based on the aggregation results.

[0253] In an optional embodiment, the step of constructing target sample pairs based on the target sample data and the target data annotation information, and storing the target sample pairs in the target sample set, includes:

[0254] An initial sample pair is constructed based on the target sample data and the target data annotation information, and a quality assessment is performed on the initial sample pair. If the quality assessment result meets the assessment conditions, the initial sample pair is used as a target sample pair and stored in the target sample set. The sample distribution characteristics corresponding to the target sample set are used to update the verification conditions of the multi-dimensional verification.

[0255] In an optional embodiment, after the step of storing the target sample pair into the target sample set is performed, the method further includes:

[0256] The target sample pairs contained in the target sample set are preprocessed, and a multidimensional feature space is constructed based on the preprocessing results. A binary classifier is trained based on the multidimensional feature space. Alternatively, a multivariate classifier containing encoding and decoding units is determined, and the multivariate classifier is trained using the target sample pairs contained in the target sample set. The input of the trained binary classifier is the data to be processed, and the output is either a reasoning label or a non-reasoning label. The input of the trained multivariate classifier is the data to be processed, and the output is labels of various data types.

[0257] In an optional embodiment, it further includes:

[0258] The trained binary or multi-class classifier classifies the data to be processed in the dataset, and determines the inference data in the dataset based on the classification results. The inference data is then used to train the business model until the target business model that meets the training stopping condition is obtained.

[0259] The data processing apparatus provided in this embodiment, in order to reduce the cost of sample construction and improve the scalability of sample construction, can first label the candidate sample data in the candidate sample data set according to the data labeling strategy after determining the candidate sample data set. At this time, at least two sets of data labeling information corresponding to each candidate sample data can be determined. Each set of data labeling information contains multiple label values, and each label value represents a different type. On this basis, multi-dimensional verification can be performed on the at least two sets of data labeling information corresponding to each candidate sample data, so as to determine the target sample data that has passed the verification in the candidate sample data set based on the multi-dimensional verification results. Since each sample data corresponds to at least two sets of data labeling information, the at least two sets of data labeling information corresponding to the target sample data can be fused to obtain the target data labeling information corresponding to the target sample data. Based on this, target sample pairs can be constructed based on the target sample data and target data labeling information, and the target sample pairs can be stored in the target sample set. This allows the high-quality target sample set to be used for model training. This data processing method can not only reduce the cost of data labeling, but also ensure the quality of data labeling, adapt to different labeling scenarios, and thus improve the efficiency of training the model using the target sample set.

[0260] The above is an illustrative scheme of a data processing apparatus according to this embodiment. It should be noted that the technical solution of this data processing apparatus and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing apparatus, please refer to the description of the technical solution of the data processing method described above.

[0261] Corresponding to the above method embodiments, this specification also provides embodiments of a classifier training device. Figure 6 A schematic diagram of a classifier training apparatus according to one embodiment of this specification is shown. Figure 6 As shown, the device includes:

[0262] The annotation module 602 is configured to annotate the candidate sample data in the candidate sample dataset according to the data annotation strategy, and obtain at least two sets of data annotation information corresponding to each candidate sample data, wherein each set of data annotation information contains multiple label values;

[0263] The verification module 604 is configured to perform multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results.

[0264] The fusion module 606 is configured to fuse at least two sets of data annotation information corresponding to the target sample data to obtain target data annotation information corresponding to the target sample data.

[0265] The construction module 608 is configured to construct target sample pairs based on the target sample data and the target data annotation information, and store the target sample pairs in the target sample set;

[0266] Training module 610 is configured to train a classifier using target sample pairs contained in the target sample set to obtain a target classifier.

[0267] The above is a schematic scheme of a classifier training device according to this embodiment. It should be noted that the technical solution of this classifier training device and the technical solution of the classifier training method described above belong to the same concept. For details not described in detail in the technical solution of the classifier training device, please refer to the description of the technical solution of the classifier training method described above.

[0268] Corresponding to the above method embodiments, this specification also provides another data processing apparatus embodiment. Figure 7 A schematic diagram of another data processing apparatus provided in one embodiment of this specification is shown. Figure 7 As shown, the device includes:

[0269] The acquisition module 702 is configured to acquire a set of data to be processed.

[0270] The processing module 704 is configured to input the data to be processed contained in the data set to be processed into the target classifier for classification processing, and determine the inference data in the data set to be processed based on the classification processing result, wherein the target classifier is obtained by the above method;

[0271] Training module 706 is configured to train a business model using the inference data until a target business model that meets the training stopping condition is obtained.

[0272] The above is an illustrative scheme of another data processing device according to this embodiment. It should be noted that the technical solution of this data processing device and the technical solution of the data processing method described above belong to the same concept. For details not described in detail in the technical solution of the data processing device, please refer to the description of the technical solution of the data processing method described above.

[0273] Figure 8 A structural block diagram of a computing device 800 according to one embodiment of this specification is shown. The components of the computing device 800 include, but are not limited to, a memory 810 and a processor 820. The processor 820 is connected to the memory 810 via a bus 830, and a database 850 is used to store data.

[0274] The computing device 800 also includes an access device 840, which enables the computing device 800 to communicate via one or more networks 860. Examples of these networks include Public Switched Telephone Network (PSTN), Local Area Network (LAN), Wide Area Network (WAN), Personal Area Network (PAN), or combinations of communication networks such as the Internet. The access device 840 may include one or more of any type of wired or wireless network interface (e.g., a network interface card (NIC)), such as an IEEE 802.11 Wireless Local Area Network (WLAN) wireless interface, a Wi-MAX (Worldwide Interoperability for Microwave Access) interface, an Ethernet interface, a Universal Serial Bus (USB) interface, a cellular network interface, a Bluetooth interface, or a Near Field Communication (NFC) interface.

[0275] In one embodiment of this specification, the above-described components of the computing device 800 and Figure 8 Other components, not shown, can also be connected to each other, for example, via a bus. It should be understood that... Figure 8 The block diagram of the computing device shown is for illustrative purposes only and is not intended to limit the scope of this specification. Those skilled in the art can add or replace other components as needed.

[0276] The computing device 800 can be any type of stationary or mobile computing device, including mobile computers or mobile computing devices (e.g., tablet computers, personal digital assistants, laptop computers, notebook computers, netbooks, etc.), mobile phones (e.g., smartphones), wearable computing devices (e.g., smartwatches, smart glasses, etc.) or other types of mobile devices, or stationary computing devices such as desktop computers or personal computers (PCs). The computing device 800 can also be a mobile or stationary server.

[0277] The processor 820 is used to execute the following computer-executable instructions, which, when executed by the processor, implement the steps of the above-described data processing method or classifier training method.

[0278] The above is an illustrative scheme of a computing device according to this embodiment. It should be noted that the technical solution of this computing device belongs to the same concept as the technical solution of the data processing method or classifier training method described above. For details not described in detail in the technical solution of the computing device, please refer to the description of the technical solution of the data processing method or classifier training method described above.

[0279] An embodiment of this specification also provides a computer-readable storage medium storing computer-executable instructions that, when executed by a processor, implement the steps of the above-described data processing method or classifier training method.

[0280] The above is an illustrative scheme of a computer-readable storage medium according to this embodiment. It should be noted that the technical solution of this storage medium belongs to the same concept as the technical solution of the data processing method or classifier training method described above. For details not described in detail in the technical solution of the storage medium, please refer to the description of the technical solution of the data processing method or classifier training method described above.

[0281] An embodiment of this specification also provides a computer program product, including a computer program or instructions that, when executed by a processor, implement the steps of the above-described data processing method or classifier training method.

[0282] The above is an illustrative scheme of a computer program product according to this embodiment. It should be noted that the technical solution of this computer program product belongs to the same concept as the technical solution of the data processing method or classifier training method described above. For details not described in detail in the technical solution of the computer program product, please refer to the description of the technical solution of the data processing method or classifier training method described above.

[0283] The foregoing describes specific embodiments of this specification. Other embodiments are within the scope of the appended claims. In some cases, the actions or steps described in the claims may be performed in a different order than those shown in the embodiments and may still achieve the desired results. Furthermore, the processes depicted in the drawings do not necessarily require a specific or sequential order to achieve the desired results. In some embodiments, multitasking and parallel processing are possible or may be advantageous. The computer instructions include computer program code, which may be in the form of source code, object code, executable files, or certain intermediate forms. The computer-readable medium may include any entity or device capable of carrying the computer program code, recording media, USB flash drives, portable hard drives, magnetic disks, optical disks, computer memory, read-only memory (ROM), random access memory (RAM), electrical carrier signals, telecommunication signals, and software distribution media, etc. It should be noted that the content included in the computer-readable medium may be appropriately added or removed according to the requirements of patent practice. For example, in some regions, according to patent practice, computer-readable media may not include electrical carrier signals and telecommunication signals.

[0284] It should be noted that, for the sake of simplicity, the foregoing method embodiments are all described as a series of actions. However, those skilled in the art should understand that the embodiments of this specification are not limited to the described order of actions, because according to the embodiments of this specification, some steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also understand that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the embodiments of this specification. In the above embodiments, the descriptions of each embodiment have different focuses; for parts not described in detail in a certain embodiment, refer to the relevant descriptions of other embodiments. The preferred embodiments disclosed above are only used to help explain this specification. Optional embodiments do not describe all details exhaustively, nor do they limit the invention to the specific implementation described. Obviously, many modifications and variations can be made based on the content of the embodiments of this specification. This specification selects and specifically describes these embodiments to better explain the principles and practical applications of the embodiments of this specification, so that those skilled in the art can better understand and utilize this specification.

Claims

1. A data processing method, characterized in that, include: The candidate sample data in the candidate sample dataset is labeled according to the data labeling strategy, and at least two sets of data labeling information are obtained for each candidate sample data, wherein each set of data labeling information contains multiple label values. For each candidate sample data, perform multi-dimensional verification on at least two sets of data annotation information corresponding to the candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results. At least two sets of data annotation information corresponding to the target sample data are fused to obtain the target data annotation information corresponding to the target sample data. Target sample pairs are constructed based on the target sample data and the target data annotation information, and the target sample pairs are stored in the target sample set.

2. The data processing method according to claim 1, characterized in that, Before the step of labeling the candidate sample data in the candidate sample dataset according to the data labeling strategy to obtain at least two sets of data labeling information corresponding to each candidate sample data is executed, the following steps are also included: Obtain initial sample data and preprocess the initial sample data to obtain a word sequence; If the number of lexical elements contained in the lexical sequence exceeds a set threshold, the lexical sequence is truncated to obtain the target lexical sequence. The target word sequence is processed using a hash algorithm to obtain sequence identifier information, and the initial sample data is stored in the candidate sample data set according to the sequence identifier information. The sequence identifier information is used to deduplicate candidate sample data in the candidate sample data set.

3. The data processing method according to claim 1, characterized in that, The annotation of any candidate sample data in the candidate sample data set includes: The data annotation model is determined according to the data annotation strategy, and the prompt words corresponding to the first candidate sample data are determined. Using a data annotation model, the first candidate sample data is annotated at least twice according to the prompt words to obtain at least two sets of data annotation information corresponding to the first candidate sample data. In each set of data annotation information, multiple label values ​​are associated with target analysis category, reasoning behavior category and / or context analysis category, and the target analysis category, the reasoning behavior category and / or the context analysis category correspond to at least one of the multiple label values.

4. The data processing method according to claim 1, characterized in that, Before performing the multi-dimensional verification step for at least two sets of data annotation information corresponding to each candidate sample data, the following steps are also included: Extract at least two inference requirement label values ​​corresponding to each candidate sample data from at least two sets of data annotation information corresponding to each candidate sample data; Fuse at least two inference requirement label values ​​corresponding to each candidate sample data; In the candidate sample data set, delete the candidate sample data whose fused inference requirement label value is associated with the first field value, and determine the target candidate sample data set based on the deletion result; The step of performing multi-dimensional verification on at least two sets of data annotation information corresponding to each candidate sample data, and determining the target sample data in the candidate sample data set based on the multi-dimensional verification results, includes: Multi-dimensional verification is performed on at least two sets of data annotation information corresponding to the candidate sample data contained in the target candidate sample data set, and the target sample data is determined in the target candidate sample data set based on the multi-dimensional verification results.

5. The data processing method according to claim 1 or 4, characterized in that, Multi-dimensional validation of any candidate sample data, including: In the at least two sets of data annotation information corresponding to the second candidate sample data, determine at least two numerical label values, at least two list label values ​​and at least two structural label values ​​corresponding to the second candidate sample data; The at least two numerical label values ​​are validated in the tolerance dimension, the at least two list-type label values ​​are validated in the matching dimension, and the at least two structured label values ​​are validated in the confidence dimension. If the verification results of the tolerance dimension, the matching dimension, and the confidence dimension all meet the verification conditions, the second candidate sample data is used as the target sample data.

6. The data processing method according to claim 5, characterized in that, The validation of the at least two numerical label values ​​in the tolerance dimension includes: Statistical label values ​​are constructed based on the at least two numerical label values. The statistical label values ​​are compared with a set statistical threshold, and the verification result of the tolerance dimension is determined based on the comparison result. The validation of the at least two list-type label values ​​in the matching dimension includes: In the case where at least two list-type label values ​​are associated with a first matching strategy, the system detects whether the at least two list-type label values ​​are equal based on the first matching strategy. In the case where the at least two list-type label values ​​are associated with the second matching strategy, the intersection of the at least two list-type label values ​​is detected as an empty set based on the second matching strategy. The verification results for the matching dimension are determined based on the detection results; The verification of the at least two structured label values ​​on the confidence dimension includes: Determine the confidence level corresponding to each of the at least two structural label values, and fuse the confidence levels corresponding to the at least two structural label values ​​to obtain the average confidence level; The average confidence level is compared with a preset confidence threshold, and the verification result of the confidence dimension is determined based on the comparison result.

7. The data processing method according to claim 5, characterized in that, The step of fusing at least two sets of data annotation information corresponding to the target sample data to obtain target data annotation information corresponding to the target sample data includes: In the at least two sets of data annotation information corresponding to the target sample data, at least two target numerical label values, at least two target list label values, and at least two target structural label values ​​corresponding to the target sample data are determined. The system performs average aggregation on the at least two target numerical label values, union aggregation on the at least two target list label values, and priority aggregation on the at least two target structure label values. The target data annotation information corresponding to the target sample data is determined based on the aggregation results.

8. The data processing method according to claim 1, characterized in that, The step of constructing target sample pairs based on the target sample data and the target data annotation information, and storing the target sample pairs in the target sample set, includes: Initial sample pairs are constructed based on the target sample data and the target data annotation information, and the quality of the initial sample pairs is evaluated. If the quality assessment results meet the assessment conditions, the initial sample pair is used as the target sample pair and stored in the target sample set; The sample distribution characteristics corresponding to the target sample set are used to update the verification conditions of the multi-dimensional verification.

9. The data processing method according to claim 1, characterized in that, After the step of storing the target sample pairs into the target sample set is performed, the method further includes: The target sample pairs contained in the target sample set are preprocessed, a multidimensional feature space is constructed based on the preprocessing results, and a binary classifier is trained based on the multidimensional feature space; or, A multivariate classifier containing an encoding unit and a decoding unit is determined, and the multivariate classifier is trained using the target sample pairs contained in the target sample set; The trained binary classifier takes the data to be processed as input and outputs either a reasoned label or a non-reasoned label as output. The trained multivariate classifier takes the data to be processed as input and outputs labels of various data types as output.

10. The data processing method according to claim 9, characterized in that, Also includes: The trained binary or multi-class classifier is used to classify the data to be processed in the dataset to be processed, and the inference data is determined in the dataset to be processed based on the classification results. The business model is trained using the inference data until a target business model that meets the training stopping condition is obtained.

11. A classifier training method, characterized in that, include: The candidate sample data in the candidate sample dataset is labeled according to the data labeling strategy, and at least two sets of data labeling information are obtained for each candidate sample data, wherein each set of data labeling information contains multiple label values. For each candidate sample data, perform multi-dimensional verification on at least two sets of data annotation information corresponding to the candidate sample data, and determine the target sample data in the candidate sample data set based on the multi-dimensional verification results. At least two sets of data annotation information corresponding to the target sample data are fused to obtain the target data annotation information corresponding to the target sample data. Target sample pairs are constructed based on the target sample data and the target data annotation information, and the target sample pairs are stored in the target sample set; The target classifier is obtained by training the classifier using the target sample pairs contained in the target sample set.

12. A data processing method, characterized in that, include: Obtain the set of data to be processed; The data to be processed contained in the dataset to be processed is input into the target classifier for classification processing. Based on the classification processing result, inference data is determined in the dataset to be processed. The target classifier is obtained by the method described in claim 11. The business model is trained using the inference data until a target business model that meets the training stopping condition is obtained.

13. A computing device, characterized in that, include: Memory and processor; The memory is used to store computer-executable instructions, and the processor is used to execute the computer-executable instructions, which, when executed by the processor, implement the steps of the method according to any one of claims 1 to 12.

14. A computer-readable storage medium, characterized in that, It stores computer-executable instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

15. A computer program product, characterized in that, It includes a computer program or instructions that, when executed by a processor, implement the steps of the method according to any one of claims 1 to 12.

Images