Data labeling and processing method and system based on natural language model

Abstract

The application provides a data labeling and processing method and system based on a natural language model, and relates to the technical field of data processing.The method comprises the following steps: step 1, a dynamic label description library is constructed, and task names, label names, label definitions and task description texts of a plurality of labeling tasks are stored to form an extensible labeling knowledge base; step 2, based on the extensible labeling knowledge base, structured prompt information including complete task descriptions, detailed label definitions and strict output format requirements is generated; and step 3, the structured prompt information and a text to be labeled are input into a pre-trained natural language model, and through semantic understanding and reasoning of the natural language model, natural language model response content including a preliminary labeling result and a judgment basis is obtained.The application improves data labeling efficiency and accuracy, and guarantees labeling result reliability and labeling system continuous adaptability.

Description

Technical Field

[0001] This invention relates to the field of data processing technology, and in particular to a data annotation and processing method and system based on natural language models. Background Technology

[0002] In sentiment annotation of e-commerce user reviews, traditional annotation technologies mostly use fixed sets of manually preset tags with low update frequency, making it difficult to cover new emotional dimensions that emerge as users' expression habits and product types change. For example, an e-commerce platform initially only set positive and negative tags for clothing reviews. When users began to frequently post mixed sentiment reviews such as "the clothes fit is great but the logistics are delayed by 3 days," the static tag library could not provide subdivided tags such as mixed sentiment with positive bias or mixed sentiment with negative bias. Annotators could only forcibly classify them into a single category, resulting in a large deviation between the annotation results and the true sentiment of the reviews.

[0003] Furthermore, traditional techniques mostly rely on manual labeling or simple rule-based initial labeling. They neither require recording the basis for labeling judgments nor have a systematic review mechanism for preliminary results. This makes it difficult to trace and correct labeling errors. For example, when labelers handle a review like "This store's customer service attitude is average, but the return and exchange of goods is very convenient," if they label it as positive based on subjective judgment, the traditional process will not retain the basis for judging why the negative information about the average customer service attitude was ignored. If the rationality of the labeling needs to be verified, the review can only be re-analyzed, which is extremely inefficient. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a data annotation and processing method and system based on natural language models, so as to improve the efficiency and accuracy of data annotation, and ensure the credibility of annotation results and the continuous adaptability of the annotation system.

[0005] To solve the above-mentioned technical problems, the technical solution of the present invention is as follows:

[0006] Firstly, a data annotation and processing method based on a natural language model, the method comprising:

[0007] Step 1: Build a dynamic tag description library to store the task names, tag names, tag definitions, and task description texts of multiple annotation tasks, forming an extensible annotation knowledge base;

[0008] Step 2: Based on a scalable annotation knowledge base, generate structured prompts including a complete task description, detailed label definitions, and strict output format requirements;

[0009] Step 3: Input the structured prompts and the text to be labeled into the pre-trained natural language model. Through the semantic understanding and reasoning of the natural language model, obtain the natural language model response content, including the preliminary labeling results and the basis for judgment.

[0010] Step 4: Parse and verify the consistency of the natural language model response content, compare the preliminary annotation results with the corresponding original text fragments of the text to be annotated, and generate preliminary annotation data;

[0011] Step 5: Input the preliminary labeled data, the text to be labeled, and the label definition back into the natural language model for verification and semantic calibration to generate the final labeled data;

[0012] Step 6: Based on the final labeled data, update the dynamic label description library to achieve continuous adjustment of the labeling system.

[0013] Secondly, data annotation and processing systems based on natural language models include:

[0014] The building module is used to build a dynamic tag description library, which stores the task name, tag name, tag definition and task description text of multiple annotation tasks, forming an extensible annotation knowledge base;

[0015] The generation module is used to generate structured prompts based on a scalable annotation knowledge base, including complete task descriptions, detailed label definitions, and strict output format requirements.

[0016] The acquisition module is used to input structured prompts and text to be labeled into a pre-trained natural language model. Through the semantic understanding and reasoning of the natural language model, the module acquires the response content of the natural language model, including the preliminary labeling results and the basis for judgment.

[0017] The annotation module is used to parse and verify the consistency of the natural language model response content, compare the preliminary annotation results with the corresponding original text fragments of the text to be annotated, and generate preliminary annotation data.

[0018] The verification module is used to re-input the preliminary labeled data, the text to be labeled, and the label definition into the natural language model for verification and semantic calibration, and generate the final labeled data.

[0019] The update module is used to update the dynamic label description library based on the final labeled data, so as to realize the continuous adjustment of the labeling system.

[0020] Thirdly, a computing device, comprising:

[0021] One or more processors;

[0022] A storage device for storing one or more programs that, when executed by one or more processors, cause the one or more processors to implement the method.

[0023] Fourthly, a computer-readable storage medium storing a program that, when executed by a processor, implements the method.

[0024] The above-described solution of the present invention has at least the following beneficial effects:

[0025] By leveraging the semantic understanding and reasoning capabilities of pre-trained natural language models, the initial processing of the annotation process is automated, reducing reliance on manual sentence-by-sentence annotation. Structured prompts guide the natural language model to output standardized results, reducing the workload of manual correction. Simultaneously, the pre-construction of a dynamic tag library avoids redundant processes of repeatedly designing annotation rules, shortening the annotation cycle. Semantic similarity analysis of the dynamic tag library ensures the consistency of the tag system itself, reducing ambiguity. Invalid annotations are eliminated through semantic comparison and positional boundary verification, preventing the natural language model output from deviating from the original text. Secondary verification and calibration further correct fuzzy annotations, ensuring a high degree of match between the results and tag definitions. Compared to the problem of subjective differences that make manual annotation susceptible to errors, this approach improves the accuracy and consistency of annotation results across tasks and batches.

[0026] By quickly matching or adding tags through task identifiers and automatically reconstructing tag groups through semantic analysis, it can adapt to different NLP tasks such as text classification, entity recognition, and sentiment analysis without reconstructing the underlying system. The automatic generation and association of task description text and tag definitions enable the annotation system to flexibly respond to new business needs. Through the closed-loop mechanism of feeding back the final annotation data to the dynamic tag library, it automatically identifies problems such as ambiguous tag definitions and task descriptions that occur during the annotation process, generates targeted update strategies, and dynamically adjusts the tag set and task description. Attached Figure Description

[0027] Figure 1 This is a flowchart illustrating the data annotation and processing method based on a natural language model provided in an embodiment of the present invention.

[0028] Figure 2 This is a schematic diagram of a data annotation and processing system based on a natural language model provided in an embodiment of the present invention. Detailed Implementation

[0029] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0030] like Figure 1As shown, embodiments of the present invention propose a data annotation and processing method based on a natural language model, the method comprising the following steps:

[0031] Step 1: Build a dynamic tag description library to store the task names, tag names, tag definitions, and task description texts of multiple annotation tasks, forming an extensible annotation knowledge base;

[0032] Step 2: Based on a scalable annotation knowledge base, generate structured prompts including a complete task description, detailed label definitions, and strict output format requirements;

[0033] Step 3: Input the structured prompts and the text to be labeled into the pre-trained natural language model. Through the semantic understanding and reasoning of the natural language model, obtain the natural language model response content, including the preliminary labeling results and the basis for judgment.

[0034] Step 4: Parse and verify the consistency of the natural language model response content, compare the preliminary annotation results with the corresponding original text fragments of the text to be annotated, and generate preliminary annotation data;

[0035] Step 5: Input the preliminary labeled data, the text to be labeled, and the label definition back into the natural language model for verification and semantic calibration to generate the final labeled data;

[0036] Step 6: Based on the final labeled data, update the dynamic label description library to achieve continuous adjustment of the labeling system.

[0037] In this embodiment of the invention, the semantic understanding and reasoning capabilities of a pre-trained natural language model are leveraged to automate the initial processing of the annotation process, reducing reliance on manual sentence-by-sentence annotation. Structured prompts guide the natural language model to output standardized results, reducing the workload of manual correction. Simultaneously, the pre-construction of a dynamic tag library avoids redundant processes of repeatedly designing annotation rules, shortening the annotation cycle. Semantic similarity analysis of the dynamic tag library ensures the consistency of the tag system itself, reducing ambiguity. Invalid annotations are eliminated through semantic comparison and positional boundary verification, preventing the output of the natural language model from deviating from the original text. Secondary verification and calibration further correct fuzzy annotations, ensuring that the results highly match the tag definitions. Compared with the problem of manual annotation being easily affected by subjective differences, this improves the accuracy and consistency of cross-task and cross-batch annotation results.

[0038] By quickly matching or adding tags through task identifiers and automatically reconstructing tag groups through semantic analysis, it can adapt to different NLP tasks such as text classification, entity recognition, and sentiment analysis without reconstructing the underlying system. The automatic generation and association of task description text and tag definitions enable the annotation system to flexibly respond to new business needs. Through the closed-loop mechanism of feeding back the final annotation data to the dynamic tag library, it automatically identifies problems such as ambiguous tag definitions and task descriptions that occur during the annotation process, generates targeted update strategies, and dynamically adjusts the tag set and task description.

[0039] In a preferred embodiment of the present invention, step 1 above, constructing a dynamic tag description library to store the task names, tag names, tag definitions, and task description texts of multiple annotation tasks, forming an extensible annotation knowledge base, may include:

[0040] Step 110: Receive the annotation task creation instruction. Based on the multiple task identifiers included in the instruction, generate and store the corresponding task name set. Specifically, this includes: continuously monitoring for input of annotation task creation instructions. When an operator issues the instruction through the interface, the system immediately receives it. The instruction contains multiple task identifiers, which are specific information used to clearly distinguish different annotation tasks. For example, for sentiment annotation of reviews of different product categories, the task identifiers might be textual information such as sentiment analysis of clothing reviews, sentiment judgment of home appliances reviews, and sentiment recognition of beauty reviews. The system parses these task identifiers one by one, extracts the core content of each identifier, and, combined with the characteristics of e-commerce user review sentiment annotation scenarios, transforms the identifiers into more standardized task names. For example, the identifier "sentiment analysis of clothing reviews" is transformed into "sentiment annotation of clothing product user reviews," and "sentiment judgment of home appliances reviews" is transformed into "sentiment annotation of home appliance product user reviews." All generated task names are arranged in order to form a complete task name set, which is then stored in the system's basic data storage unit. An index is created for this set during storage.

[0041] Step 111: Based on the task name set, match one or more associated tag items for each task name to construct a structured task tag mapping table; calculate the semantic similarity between each tag item in the task tag mapping table, and construct a semantically consistent tag grouping structure based on the semantic similarity analysis results; specifically, this includes: retrieving the task name set from the storage unit, reading each task name one by one, and for each task name, referring to the common sentiment expression forms of user reviews of this type of product in the e-commerce field, matching the associated tag items. Taking the task name of sentiment labeling of user reviews of clothing products as an example, in addition to the basic positive sentiment and negative sentiment tag items, it will also match mixed sentiment, positive bias mixed sentiment, negative bias mixed sentiment, etc., because clothing reviews often show mixed sentiment expressions such as satisfaction with the style but dissatisfaction with the size.

[0042] Each task name is mapped to all its matching tags, arranged in tabular form with the task name preceding the tags. Each row in the table corresponds to a task name and all its associated tags. The system then performs pairwise comparisons of all tags in the task tag mapping table, calculating the semantic similarity between each pair. During the calculation, the core semantic meaning of each tag is first determined. For example, the core semantic meaning of positive sentiment is an affirmative attitude towards a product or service, while the core semantic meaning of negative sentiment is a negative attitude towards a product or service. The core semantic meaning of positive-leaning mixed sentiment is the simultaneous presence of both affirmative and negative attitudes, with a higher proportion of affirmative attitude. Then, the overlap and correlation of the core semantic meanings of each pair of tags are compared. The greater the overlap, the higher the correlation, and the greater the semantic similarity value. For example, the semantic similarity between positive sentiment and positive-leaning mixed sentiment is greater than that between positive sentiment and negative sentiment. Based on the semantic similarity calculation results of all tags, tags with high semantic similarity are grouped together, forming a tag grouping structure.

[0043] Step 112: Reconstruct the task tag mapping table based on the tag grouping structure to form a reorganized task tag mapping table; according to the tag item organization structure in the reorganized task tag mapping table, create and store the corresponding detailed tag definition for each tag item to form a tag definition dictionary; specifically, according to the obtained tag grouping structure, rearrange the original task tag mapping table. During the reorganization process, the tag items corresponding to each task name are no longer arranged randomly, but are presented sequentially according to their tag groups. Tag items in the same group are arranged together, and there are clear distinguishing marks between different groups, thus forming the reorganized task tag mapping table.

[0044] This table provides a clear overview of the tags grouped by different sentiment tendencies under each task name. Next, based on the specific meaning of each tag in the reorganized task tag mapping table and its practical application in e-commerce user review sentiment annotation, the system creates detailed tag definitions for each tag. For example, a positive sentiment is defined as a review containing only positive evaluations of the clothing item, such as good material, novel style, and comfortable wear, without any negative evaluations; a negative sentiment is defined as a review containing only negative evaluations of the clothing item, such as rough workmanship, incorrect size, and poor fabric, without any positive evaluations; and a mixed sentiment is defined as a review containing both positive and negative evaluations of the clothing item. Positive and negative reviews are defined as having roughly equal intensity and no obvious bias. A positive-biased mixed sentiment review is defined as a review containing both positive and negative reviews, but with more specific and numerous positive reviews or stronger emotional intensity, resulting in an overall positive bias. A negative-biased mixed sentiment review is defined as a review containing both positive and negative reviews, but with more specific and numerous negative reviews or stronger emotional intensity, resulting in an overall negative bias. All tags and their corresponding detailed definitions are compiled into a dictionary, with each tag as the key and its definition as the value, forming a tag definition dictionary, which is then stored in a designated storage area.

[0045] Step 113: Based on the task name set and task tag mapping table, automatically generate corresponding task description text for each task name, constructing a task description text library. Specifically, this includes: extracting a task name from the task name set, finding all tags corresponding to that task name and their respective groups from the reorganized task tag mapping table, and combining this with the overall goal of e-commerce user review sentiment annotation—accurately identifying the sentiment tendency in reviews—integrating the task name tag group information and the core features of each tag into the description text, automatically generating the task description text for that task name. Taking sentiment annotation of apparel product user reviews as an example, the generated task description text could be: the goal of this task is to... To assign sentiment to published reviews of clothing products, it's necessary to determine whether the review's sentiment leans towards a positive, negative, or mixed sentiment group based on the specific wording of the review. The positive sentiment group includes positive sentiment with a positive bias and mixed sentiment; the negative sentiment group includes negative sentiment with a negative bias and mixed sentiment; and the mixed sentiment group is simply a collection of mixed sentiments. Specific criteria for each tag can be found in the tag definition dictionary. During tagging, it's crucial to ensure that each review is matched with the most appropriate tag. Similarly, generate corresponding task description text for each task name in the task name set, and store all generated texts categorized by task name to form a task description text library.

[0046] Step 114 integrates and associates the task name set, task tag mapping table, tag definition dictionary, and task description text library, storing them in a dynamically updated structured database to ultimately form an extensible annotation knowledge base. Specifically, this includes four parts: the task name set stored via system calls, the reorganized task tag mapping table, the tag definition dictionary, and the task description text library. Interrelationships between these parts are achieved by establishing association fields. For example, using task name as an association field, each task name can be mapped to both its associated tag item in the task tag mapping table and its corresponding task description text in the task description text library. Similarly, using tag item as an association field, each tag item can be mapped to both the task name containing it in the task tag mapping table and its detailed tag definition in the tag definition dictionary. After integration and association, an organic whole is formed and stored in a structured database that supports dynamic updates. This database allows operators to add new task names, add or modify tags, update tag definitions, and adjust task description text according to actual needs. When new product categories appear on the e-commerce platform or users begin to frequently use new forms of emotional expression, the database update function can promptly supplement the corresponding content, and the resulting labeled knowledge base is scalable.

[0047] In this embodiment, when users begin to frequently post mixed sentiment comments, the mechanism can be used to promptly add mixed sentiment and related sub-tags and clarify their definitions, avoiding annotation bias caused by the static tag library's inability to cover new sentiment dimensions. The structured task tag mapping table makes the correspondence between tasks and tags clearer, allowing annotators to quickly find the tags required for each task. The detailed tag definition dictionary unifies the tag judgment criteria and reduces annotation errors caused by inconsistent tag understanding.

[0048] In a preferred embodiment of the present invention, step 2 above, based on a scalable annotation knowledge base, generates structured prompt information including a complete task description, detailed label definitions, and strict output format requirements, and may include:

[0049] Step 210: Receive processing requests for annotation tasks and analyze the task identifier in the processing request to generate a task query instruction. Specifically, this includes: real-time monitoring of annotation task processing requests from the operating terminal. When an operator submits a processing request for, for example, mixed sentiment annotation of user reviews of women's clothing products in the fourth quarter of 2025, the system immediately receives the request. The task identifier in the request contains key information such as the time range of the fourth quarter of 2025, the product category of women's clothing products, the annotation target of user reviews, and the annotation focus of mixed sentiment. The task identifier is broken down word by word. First, the core product category of women's clothing products is separated out, and then the core task type of mixed sentiment annotation is extracted. The annotation target is confirmed to be user reviews, and the time range is used as an auxiliary filtering condition. Based on this extracted information, the system generates a task query instruction according to the retrieval rules of the dynamic tag description library.

[0050] Step 211: Based on the task query instruction, retrieve the corresponding task description text from the dynamic tag description library using the task identifier, and obtain a set of tag names associated with the task; specifically, according to the generated task query instruction, use women's clothing products and mixed sentiment annotations in the task identifier as core search terms, and perform matching searches in the task name set of the dynamic tag description library. By comparing the product categories and annotation types contained in the task names, find the task name "women's clothing products user reviews mixed sentiment annotations" that matches perfectly; subsequently, retrieve the detailed description text corresponding to the task name from the task description text library. The text content is the task description text. Sentiment labeling was performed on user reviews of women's apparel products on the platform in the fourth quarter of 2025. The focus was on identifying the presence of mixed sentiment in the reviews and the degree of bias between positive and negative sentiment within these mixed sentiments. The system needed to accurately match the corresponding sentiment tags to the review content, and the labeling results needed to reflect the true sentiment tendency of the reviews. Simultaneously, based on the association between the task name and tag items in the task tag mapping table, the system extracted all relevant tag item names, including purely positive sentiment, purely negative sentiment, balanced mixed sentiment, positive-biased mixed sentiment, and negative-biased mixed sentiment. These tag names were then arranged in sentiment logical order to form a tag name set.

[0051] Step 212: Based on the tag name set, extract the corresponding detailed tag definitions from the tag definition dictionary to form a complete tag definition set for the target annotation task. Specifically, this includes: sequentially reading each tag name from the obtained tag name set, and performing a precise matching query in the tag definition dictionary for each tag name. For example, when querying purely positive sentiment, the extracted definition is that the comment content only contains positive evaluations of women's clothing products, such as soft fabric, slimming fit, accurate color, and high cost-performance ratio, without any negative evaluation words or expressions, and the overall sentiment is positive and affirmative. When querying balanced mixed sentiment, the extracted definition is that the comment content contains both positive and negative evaluations, with roughly equal numbers of statements for both types of evaluations and similar emotional intensity. For example, "The dress style is very nice, but the fabric is a bit stiff." Positive evaluations of the beautiful style and negative evaluations of the stiff fabric are equally direct in expression, without obvious bias. When querying for a mixed positive sentiment, the definition is that the comment content contains both positive and negative evaluations, with more positive evaluations than negative ones, or the positive evaluations are expressed with stronger emotions. For example, "This coat has a unique design; although it's a bit expensive, it's worth buying." The positive evaluation emphasizes the unique design and the fact that it's worth buying, indicating an overall positive bias. When querying for a mixed negative sentiment, the definition is that the comment content contains both positive and negative evaluations, with more negative evaluations than positive ones, or the negative evaluations are expressed with stronger emotions. For example, "The color of the clothes is nice, but the workmanship is rough, there are many loose threads, and it pills after one wear." The negative evaluation is more specific in its description of rough workmanship, loose threads, and pilling, indicating an overall negative bias. These tag names and their corresponding detailed definitions are organized in the order of the tag name set, with each tag definition forming a separate paragraph, creating a complete tag definition set.

[0052] Step 213 involves integrating the task description text with the complete tag definition set to construct the core semantic content of the annotation task. Specifically, this includes: first, extracting the complete task description text as the opening section of the core semantic content; presenting the time frame of the annotation task word by word, such as the fourth quarter of 2025, clearly defining the comment posting period targeted by the annotation work; listing the product categories in detail, such as women's clothing, covering specific clothing types like dresses, tops, and pants; clearly indicating that the annotation object is the original text of women's clothing comments posted by users on the platform, including text comments and short sentences containing emotional inclinations; and highlighting the core objective, namely, accurately identifying the emotional type in the comments, especially mixed emotions and their positive and negative biases, ensuring that the annotation results are consistent with the users' true emotions.

[0053] After the task description text ends, the system adds a special transition statement. This statement not only serves as a link but also further emphasizes the importance of the tag definition. For example, to ensure the accuracy of the judgment of various emotions during the tagging process, the specific definitions and judgment criteria of each emotion tag are strictly followed. Then, the complete set of tag definitions is logically connected to the transition statement one by one. The order starts with simple emotion tags, i.e., simple positive emotion, simple negative emotion, and then mixed emotion tags, balancing mixed emotions, positive-biased mixed emotions, and negative-biased mixed emotions, so that the presentation of tag definitions matches the progressive complexity of emotional expression in user comments.

[0054] During the integration process, the task description text is compared sentence by sentence with the content of each tag definition to check the logical connection. For example, if the task description mentions the key point of identifying mixed sentiment and the degree of bias, the definition of balanced mixed sentiment will be checked one by one to see if it is clear, whether the judgment criteria for the number of positive and negative evaluation statements are roughly equal and the sentiment intensity is similar, whether the definition of positive biased mixed sentiment details whether the number of positive evaluation statements is greater than that of negative evaluation statements, or whether the positive evaluation uses stronger words, and whether the definition of negative biased mixed sentiment includes the judgment details that the number of negative evaluation statements is greater than that of positive evaluation statements, or whether the negative evaluation description is more specific. This ensures that the definition of each mixed sentiment tag can directly respond to the task description's focus on the degree of bias and form a close echo with the task objective.

[0055] Step 214 involves combining the core semantic content with the preset standardized output format requirements to generate an initial structured prompt framework. This initial framework is then integrated and adjusted to generate the final structured prompt information. Specifically, this includes: calling the preset standardized output format requirements, which detail the presentation format of the annotation results; the content being the original text of the comment to be annotated, a complete excerpt of the user's comment; labeling with tags, filling in matching tag names, such as purely positive sentiment, positive-leaning mixed sentiment, etc.; extracting key sentiment statements, extracting statements reflecting both positive and negative sentiments, or only extracting statements of the corresponding type if it is purely sentimental; defining the criteria, explaining how the extracted statements conform to the tag, including specific reasons such as sentiment intensity comparison and statement quantity comparison; and confirming with the annotator, ensuring the information is accurate after annotation. The core semantic content is placed first, followed by the standardized output format requirements, forming the initial structured prompt framework.

[0056] Next, the initial framework is validated to check whether the tag names in the core semantic content are consistent with the tags involved in the output format requirements. For example, if the core semantics include balanced mixed sentiment, then the key sentiment statement extraction part in the output format needs to show both positive and negative statements. After checking and confirming, it is supplemented that if the tag is balanced mixed sentiment, it is necessary to ensure that the number of positive and negative statements extracted is roughly equal. At the same time, the coherence of the language expression is checked, and the ending sentence of the core semantic content is adjusted to "Please complete the annotation according to the following standard output format" to make the core semantics and output format requirements naturally connected, resulting in the final structured prompt information.

[0057] In this embodiment, the complete task description clarifies the scope and objectives of annotation for annotators, avoiding annotation errors caused by misunderstandings of the task; the detailed tag definitions, especially the clear explanations of mixed sentiment and its sub-types, reduce the problem of inaccurate annotation of mixed sentiment in the static tag library, ensuring that the annotation results are consistent with the true sentiment of the comments.

[0058] In a preferred embodiment of the present invention, step 3 above, which involves inputting the structured prompt information and the text to be labeled into a pre-trained natural language model, and obtaining the natural language model response content, including preliminary labeling results and judgment criteria, through the semantic understanding and reasoning of the natural language model, may include:

[0059] Step 310: Integrate the structured prompts and the text to be labeled according to the preset input format to form a complete input sequence including task description, label definition, output format requirements, and the text to be labeled. Specifically, the preset input format is a fixed four-segment structure. Each segment must be specific to the scene details to ensure that the natural language model can accurately understand the task boundaries. The first segment is the task description, which must clearly define the applicable scenarios, evaluation dimensions, and core requirements for labeling. For example, this labeling is for user reviews under the beauty category on an e-commerce platform. It is necessary to identify the sentiment types in the reviews involving four dimensions: product effect, packaging design, logistics service, and customer service attitude. Specifically, there are four types: positive sentiment with only positive evaluations, negative sentiment with only negative evaluations, mixed sentiment with both positive and negative evaluations and a stronger positive evaluation, and mixed sentiment with both positive and negative evaluations and a stronger negative evaluation. When labeling, it is necessary to strictly distinguish the sentiments of different dimensions and not to forcibly classify multi-dimensional mixed evaluations into a single sentiment type.

[0060] The second paragraph defines the tags, requiring a detailed explanation of the criteria for each tag, including details such as dimension coverage, emotional proportion, and vocabulary strength. For example, positive emotions: all evaluation dimensions are positive statements without any negative words, such as "The lipstick color is perfect, the packaging is exquisite, I received it the next day, and the customer service was very patient." Negative emotions: all evaluation dimensions are negative statements without any positive words, such as "The foundation cakes badly, the packaging is damaged, the delivery took a week, and customer service is unresponsive." Mixed emotions with a positive bias: at least two evaluation dimensions are included, with positive dimensions comprising more than 50% of the word count, or using strong positive words like "amazing" or "perfect," while negative dimensions use only weak negative words like "average" or "okay," such as "The eyeshadow has excellent color payoff, but the packaging is a bit simple, and customer service is a bit slow to respond." Mixed emotions with a negative bias: at least two evaluation dimensions are included, with negative dimensions comprising more than 50% of the word count, or using strong negative words like "terrible" or "rubbish," while positive dimensions use only weak positive words like "okay" or "average," such as "The mask is reasonably priced, but there's very little essence, I had an allergic reaction after using it, and the delivery was delayed."

[0061] The third section outlines the output format requirements, specifying the content limitations and presentation format for each line. For example, the first line can only contain one of the four tags defined in the tag definition, without adding any other text or symbols. The second line must be formatted as positive information, negative information, and the reason for judgment. The third line contains the quality assessment parameters, which are values ​​between 0 and 1, rounded to two decimal places. The fourth section is the text to be labeled, i.e., specific user reviews, such as "This sunscreen has a very good sun protection effect, it's not greasy when applied, but the bottle is a bit small, and the delivery was two days later than expected." After that, the four sections are concatenated in the order described above, with each section separated by two line breaks to ensure clear boundaries between the information parts, forming a complete input sequence that the natural language model can directly read.

[0062] Step 311 involves using the semantic encoding layer of a natural language model to perform multi-level semantic understanding on the complete input sequence, extracting deep feature representations of task semantics and text semantics. Specifically, the semantic encoding layer of the natural language model consists of 12 interconnected Transformer encoding layers, each containing a multi-head attention mechanism and a feedforward neural network. First, the complete input sequence is converted into numerical word vectors, each corresponding to the basic semantics of a word or character. The first encoding layer uses a multi-head attention mechanism to calculate the correlation between each word vector and its surrounding word vectors, for example, calculating the correlation between mixed sentiment positive bias and the more significant influence of positive statements, capturing the semantic connections within the label definition. The feedforward neural network then processes these correlation results... Nonlinear transformation enhances key semantic features; the second encoding layer receives the output of the first layer and further calculates the association of word vectors at greater distances, such as associating appliance reviews in the task description with appliance-related expressions such as cooling speed in the tag definition, establishing a contextual connection between the task and the tag. Subsequent encoding layers repeat this process, deepening the understanding of the text layer by layer. For example, in the 6th encoding layer, the large capacity in the text to be labeled is associated with the definition of the positive sentiment tag across segments. In the 12th encoding layer, all associated information is finally integrated to form a deep feature representation that can simultaneously reflect the task requirements, tag standards, and the semantics of the text to be labeled. These features exist in the form of a numerical matrix, where the value of each element in the matrix represents the strength of the corresponding semantic association.

[0063] Step 312: Based on deep feature representation, the inference decision layer of the natural language model is used for analysis to generate the initial annotation results of the text to be annotated and the corresponding judgment basis of the original response content. Specifically, the inference decision layer consists of 3 fully connected network layers and 1 output layer. The fully connected network is responsible for feature transformation, and the output layer is responsible for generating natural language responses. First, the first fully connected network receives the digital matrix of deep feature representation and converts it into a feature vector with lower dimension through matrix operations, highlighting the matching features of the text to be annotated with each label. For example, for comments with large volume, acceptable noise, but slow cooling, the transformed feature vector will strengthen key features such as large volume (positive), slow cooling (negative), and more positive expressions. The second fully connected network performs classification calculation on the feature vector, compares the feature vector with the four preset sentiment label features, and calculates the matching probability of each label. For example, the matching probability of the comment with mixed sentiment positive bias is calculated to be 85%, and the matching probability of other labels is less than 50%. The third fully connected network determines the initial label as mixed sentiment positive bias based on the highest matching probability and extracts the corresponding comment fragment as a complete sentence. At the same time, the network backtracks the feature comparison process to generate a logical chain of judgment criteria. That is, the comment contains two positive statements about large capacity and acceptable noise, and one negative statement about slow cooling speed. The positive statements are more numerous and have a more significant impact, which meets the definition of mixed sentiment positive bias. Finally, the output layer converts this information into natural language text to form the original response content. For example, if the label result is mixed sentiment positive bias, the corresponding comment fragment is "This refrigerator has a large capacity and acceptable noise, but the cooling speed is a little slower than expected." The judgment criteria are that the positive statements in the comment are more numerous than the negative statements and have a more significant impact.

[0064] Step 313 involves applying preset analysis rules to the original response content for structured parsing, extracting the specific content of the preliminary annotation results and the detailed description of the judgment basis. Specifically, the preset analysis rules contain three parts of identification logic, corresponding to the annotation results, comment fragments, and judgment basis. First, the position of the annotation results in the original response content is located using the sentence structure features of the annotation results in the rules, and the mixed sentiment positive bias is extracted as the tag name. Next, the complete text within quotation marks is located and extracted as the corresponding comment fragment using the sentence structure features of the corresponding comment fragment. Then, the complete description is extracted as the detailed content of the judgment basis using the sentence structure features of the judgment basis. If there are expression deviations in the original response content, such as writing the corresponding comment fragment as the corresponding text, the rules will correct it through keyword matching to ensure that the extracted tag names, comment fragments, and judgment basis accurately correspond. After parsing, three independent structured data sets are formed, recording the specific content of the preliminary annotation results and the detailed description of the judgment basis, respectively.

[0065] Step 314 involves associating the preliminary annotation results with the judgment criteria, and generating quality assessment parameters based on the semantic matching degree between the annotation results and the label definitions. This ultimately forms a complete response content including the annotation results, judgment criteria, and quality assessment parameters. Specifically, this includes: first, establishing an association and matching relationship, binding the label name extracted in Step 313 (mixed sentiment positive bias), the comment fragment ("This refrigerator has a large capacity and the noise is acceptable, but the cooling speed is a little slower than expected") with the judgment criteria ("The number of positive statements in the comment is greater than the number of negative statements, and their impact is more significant") to ensure a one-to-one correspondence and form a set of associated data; then, calculating the semantic matching degree, specifically by combining the semantic features of the comment fragment with the mixed sentiment... The semantic features defined by the positive bias label are compared. First, the label definition is split into two core semantic points: the simultaneous existence of positive and negative expressions and the more significant impact of positive expressions. Then, the judgment criteria for the comment segment are split into two actual features: the existence of two positive expressions and one negative expression, and the greater number and more significant impact of positive expressions. The number of overlaps between the actual features and the core semantic points is counted as 2. The overlap number is divided by the total number of core semantic points of 2 to obtain a ratio of 100%. This ratio is then converted into a value between 0 and 1, 1.0, as a quality assessment parameter. Finally, the labeling results after association matching, the judgment criteria, and the quality assessment parameter 1.0 are integrated into a complete data point to form a complete response content.

[0066] In this embodiment, the natural language model can accurately grasp the scene details and label judgment criteria of sentiment annotation in e-commerce reviews, avoid misjudgment of mixed sentiment due to information ambiguity, reduce the problem that the static label library cannot cover the dynamic sentiment dimension, and retain the judgment basis generated by the inference decision layer completely after structured parsing, recording in detail the specific content of positive and negative information, dimension attribution and bias reasons, thereby improving the accuracy and efficiency of the annotation process.

[0067] In a preferred embodiment of the present invention, step 4 above, which involves parsing and verifying the consistency of the natural language model response content, comparing the preliminary annotation results with the corresponding original text fragments of the text to be annotated, and generating preliminary annotation data, may include:

[0068] Step 410: Perform structured parsing of the response content of the natural language model to extract preliminary annotation results and corresponding judgment criteria. Specifically, the response content of the natural language model has a fixed three-segment structure: the first line records the preliminary annotation results, the second line records the judgment criteria in detail, and the third line records the quality assessment parameters. During parsing, the response content is read line by line to confirm the functional positioning of each line. Taking the response content of an e-commerce review as an example, the first line of content is mixed sentiment with a positive bias, and this line of text is directly extracted as the preliminary annotation result. The second line of content contains positive information: the product returns and exchanges are very convenient, which is negative information; the customer service attitude is average, which is the reason for the judgment. Positive information provides a more specific description of the service and attracts greater user attention. This line of text should be extracted completely and used as the basis for judgment. The third line has a score of 0.95 and will be temporarily retained for future quality reference. During the analysis process, the extracted content must be checked word by word to ensure that the initial annotation results correspond completely with the basis for judgment. The two original text fragments mentioned in the basis for judgment, "the product returns and exchanges are convenient" and "the customer service attitude is average," must be found in the text to be annotated. Furthermore, the initial annotation result, which has a mixed positive sentiment bias, must coexist with the positive and negative information in the basis for judgment, with the positive being more prominent. The description logic must be consistent, and there should be no contradiction between the result and the reason. At the same time, it must be ensured that no original text fragments are omitted.

[0069] Step 411 involves semantically comparing the original text fragments cited in the preliminary annotation results with the corresponding actual original text fragments in the text to be annotated, obtaining a semantic comparison result to characterize the semantic consistency between the two. Specifically, this includes: extracting all cited original text fragments from the extracted judgment criteria. Taking the above judgment criteria as an example, the positive reference fragment "product returns and exchanges are very convenient" and the negative reference fragment "customer service attitude is average" are extracted. Then, the specific content of the text to be annotated is determined. Here, the text to be annotated is "this store's customer service attitude is average, but product returns and exchanges are very convenient." The actual original text fragments corresponding to the cited fragments are searched word by word in this text, and finally, the actual positive fragment "product returns and exchanges are very convenient" and the actual negative fragment "customer service attitude is average" are found.

[0070] Semantic decomposition and comparison were performed on each set of quoted fragments and actual fragments. First, the core words of both were analyzed. The core word of the positive quoted fragment was "convenient product returns and exchanges," and the core word of the actual positive fragment was also "convenient product returns and exchanges," showing complete consistency. Next, the collocation relationship of the core words was analyzed. In the quoted fragment, "returns and exchanges" and "convenient" formed a semantic collocation of "convenient service," and the actual fragment showed the same collocation relationship with "convenient service," with no semantic deviation. The core word of the negative quoted fragment was "average customer service attitude," and the core word of the actual negative fragment was consistent. Furthermore, "attitude" and "average" formed a neutral-to-negative collocation relationship of "service," which was completely identical in both. If a quoted fragment and an actual fragment... In cases where there are word differences, such as quoting "convenient for returns and exchanges" but actually stating "the product is very convenient for returns and exchanges," the system will further determine whether the difference affects the core semantics. Since "product" is a modifier and does not change the core meaning of "convenient for returns and exchanges," the system will still determine semantic consistency. If quoting "poor customer service attitude" but actually stating "average customer service attitude," the opposite of "poor" and "average" will be determined as semantic contradiction. Finally, the system will combine the comparison results of the two sets of fragments to generate a semantic comparison result. For example, if the core word collocation of the positive quotation fragment is completely consistent with the actual positive fragment, and the core word collocation of the negative quotation fragment is completely consistent with the actual negative fragment, the overall semantics will be consistent and there will be no contradiction.

[0071] Step 412: Compare the text position boundaries identified by the preliminary annotation results with the actual position boundaries of the corresponding original text segments in the text to be annotated, generating position comparison results to characterize the degree of position matching between the two. Specifically, this includes: first, determining the position boundaries identified by the preliminary annotation results. The position boundaries are represented by start and end character numbers, with the numbers counting sequentially from the first character of the text to be annotated. Taking the text "This store's customer service attitude is so-so, but product returns and exchanges are very convenient" as an example, the text is first annotated character by character with the following numbers: 1 for "this", 2 for "home". 3 is store, 4 is customer, 5 is service, 6 is attitude, 7 is degree, 8 is one, 9 is generally, 10 is comma, 11 is not, 12 is past, 13 is merchant, 14 is product, 15 is return, 16 is exchange, 17 is goods, 18 is very, 19 is square, 20 is convenient. According to the original text quoted in the judgment, the boundary of the position indicating the customer service attitude is generally from 5 to 11, that is, starting from the 5th character "service" and ending at the 11th character; the boundary of the position indicating that the return and exchange of goods is very convenient is from 15 to 20, that is, starting from the 15th character "return" and ending at the 20th character.

[0072] Next, find the position boundaries of the actual original text fragments in the text to be annotated. The text corresponding to the actual customer service attitude being average is "客服态度一般", and the corresponding serial numbers are from 5 to 9, that is, starting from the 5th character "客" and ending at the 9th character "般"; the text corresponding to the actual convenient return and exchange of goods is "退换货很方便", and the corresponding serial numbers are from 15 to 20, which is exactly the same as the marked boundary. Then calculate the matching data between each group of marked boundaries and the actual boundaries. For "客服态度一般": First, calculate the total number of characters in the marked boundary. Subtract the starting serial number 5 from the ending serial number 11 and then add 1, getting 7 characters; then calculate the total number of characters in the actual boundary. Subtract the starting serial number 5 from the ending serial number 9 and then add 1, getting 5 characters; the overlapping part is from 5 to 9, and the overlapping number of characters is 5. Next, calculate the position matching ratio. Divide the overlapping number of characters 5 by the total number of characters in the marked boundary 7, and the result is approximately 0.71; finally, calculate the deviation number of characters. The marked boundary contains 2 more characters, the 10th character "逗" and the 11th character "不", than the actual boundary, so the deviation number of characters is 2.

[0073] For "退换货很方便"; the marked boundary and the actual boundary completely overlap. First, calculate the total number of characters. Subtract the starting serial number 15 from the ending serial number 20 and then add 1, getting 6 characters; the overlapping number of characters is also 6. The position matching ratio is obtained by dividing the overlapping number of characters 6 by the total number of characters in the marked boundary 6, and the result is 1. The deviation number of characters is 0 because there are no extra or missing characters. Finally, generate the position comparison result by integrating the two groups of data. For example, the marked boundary of the positive fragment "退换货很方便" from 15 to 20 completely overlaps with the actual boundary from 15 to 20, the position matching ratio is 1, and the deviation number of characters is 0; the marked boundary of the negative fragment "客服态度一般" from 5 to 11 partially overlaps with the actual boundary from 5 to 9, the position matching ratio is approximately 0.71, and the deviation number of characters is 2; overall, the position matching degree is good, and only the negative fragment has a slight boundary deviation.

[0074] Step 413: Integrate the semantic comparison results and the positional comparison results, perform consistency verification on the preliminary annotation results, identify and exclude invalid annotations in the preliminary annotation results that have semantic contradictions or positional boundary deviations from the original text, and generate a set of verified annotation results. Specifically, this includes: First, setting dual standards for consistency verification to ensure that the verification logic is feasible. The semantic standard is that the core word collocation relationship of all cited fragments and actual fragments is completely consistent, with no semantic contradictions, and situations such as citing a positive fragment but actually being a negative fragment with opposite core word meanings are not allowed. The positional standard is that the positional matching ratio of each fragment is not less than 0.6, and the number of deviation characters does not exceed 2 characters. If the matching ratio is less than 0.6 or the number of deviation characters exceeds 2 characters, even if the semantics are consistent, it is judged as having too large a positional deviation, which may lead to misplacement of the annotated fragments, such as labeling a fragment about slow logistics in the position of poor customer service. Then, the semantic comparison results and the positional comparison results are compared and verified one by one. Taking the mixed sentiment positive bias annotation as an example, the semantic comparison results show no contradictions and high consistency, meeting the semantic standard. In the positional comparison results, the positive segment matching ratio is 1 with a deviation of 0, and the negative segment matching ratio is 0.71, which is higher than 0.6 and has a deviation of 2 but not exceeding 2, meeting the positional standard. Therefore, the annotation is determined to be valid. If the semantic comparison result of an annotation is a semantic contradiction between the negative reference segment "slow logistics" and the actual segment "fast logistics", it is directly determined to be an invalid annotation. If the positional comparison result of an annotation is a positive segment matching ratio of 0.5, which is lower than 0.6 and has a deviation of 3 characters, which exceeds 2, even if the semantics are consistent, it is also determined to be an invalid annotation. Then, all invalid annotations are recorded separately, explaining the reasons for invalidity, such as semantic contradictions or excessive positional deviations, and are excluded from the preliminary annotation results. Finally, all annotation results that have passed the dual standard verification are collected and organized according to the format of annotation label plus judgment basis plus semantic comparison result plus positional comparison result to form a set of verified annotation results.

[0075] Step 414: Based on the validated annotation result set, construct structured preliminary annotation data. Specifically, this includes: first, determining the fixed fields of the structured preliminary annotation data. Each field corresponds to key information in the annotation process, and the field content must be directly usable in the verification step. The field settings are as follows: The first field is the annotation label, strictly filled in with the validated preliminary annotation result, and must be completely consistent with the name in the label definition, such as "mixed sentiment positive bias," without abbreviation or modification; the second field is the original text to be annotated, a complete copy of the text to be annotated, such as "This store's customer service attitude is so-so, but product returns and exchanges are very convenient," ensuring the original text is free of typos and omissions; the third field is the actual positive segment, filled in with the actual corresponding positive segment in the text to be annotated, and must be completely consistent with the original text, such as "Product returns and exchanges are very convenient"; the fourth field is the actual negative segment, filled in with the actual corresponding negative segment in the text to be annotated, also consistent with the original text, such as "Customer service attitude is so-so"; the fifth field is semantic comparison details, recording each group in detail. The semantic comparison results of the segments are as follows: For example, for positive segments, the core keyword "easy return and exchange of goods" matches the actual situation, and the collocation is consistent; for negative segments, the core keyword "general customer service attitude" matches the actual situation, and the collocation is consistent; there are no semantic contradictions overall. The sixth field is the location comparison details, which records the matching ratio and deviation number of characters for each group of segments. For example, for positive segments, the identifier is 15-20, the actual is 15-20, the matching ratio is 1, and the deviation is 0; for negative segments, the identifier is 5-11, the actual is 5-9, the matching ratio is 0.71, and the deviation is 2. The seventh field is the verification result, which is filled in as valid or invalid, and here it is valid. The eighth field is the verification time, which records the time when the verification was completed, in the format of year, month, day, hour, minute, and second, such as 2025-10-18 14:30:00. Then, all the field contents of a valid label are integrated in order to form a structured data. If the label result set contains multiple valid labels, they are arranged in order according to the same field structure, with each data in a separate row, forming a structured preliminary label dataset.

[0076] This embodiment parses the response content of the natural language model line by line to ensure the complete extraction of the preliminary annotation results and judgment basis, avoiding verification gaps caused by missing information. The refined comparison at the semantic level can accurately identify the consistency between the cited fragment and the original text in the core word collocation relationship. The consistency verification actively excludes invalid annotations through clear semantic and positional dual standards, thereby improving the reliability of the annotation results.

[0077] In a preferred embodiment of the present invention, step 5 above, which involves re-inputting the preliminary annotation data, the text to be annotated, and the label definitions into the natural language model for review and semantic calibration to generate the final annotation data, may include:

[0078] Step 510: Based on the preliminary labeled data, the text to be labeled, and the label definitions, construct the prompt information for review. Specifically, this includes: determining the four core components of the review prompt information, each detailed enough for the model to directly understand the review logic and basis. The first part is the review task description, clearly defining the operational goals and standards of the natural language model. For example, reviewing the preliminary labeled data of e-commerce user reviews requires completing three operations: first, determining whether the preliminary labeled tags conform to the core features in the label definition; second, checking whether the positive and negative original text fragments of the preliminary labeled data completely cover all sentiment information in the text to be labeled; and third, if there are inconsistencies between labels and definitions, missing fragments, or semantic ambiguity, the problem and the reason for the judgment must be clearly stated. The second part is the full text of the text to be labeled, presenting the user review completely. For example, "The clothes in this store have very comfortable fabric and a good fit, but there are a lot of loose threads, and the logistics were two days later than expected, although the customer service response was quite timely," ensuring the natural language model can analyze all sentiment points by comparing with the original text. The third part is the preliminary labeled data. The key information needs to extract the core content related to the review, including preliminary labeling, actual positive segments, actual negative segments, semantic comparison details, and positional comparison details. This information needs to be listed in points to ensure that the natural language model can quickly locate key data. The fourth part is the complete label definition, which needs to refine the core features of each category, such as mixed sentiment positive bias; containing both positive and negative information, with the total number of words in the positive information exceeding the number of words in the negative information, or the number of strong positive words such as "very" and "extremely" used in the positive information exceeding the number of strong negative words such as "too" and "extremely" used in the negative information; mixed sentiment negative bias; containing both positive and negative information, with the total number of words in the negative information exceeding the number of words in the positive information, or the number of strong negative words used in the negative information exceeding the number of strong positive words used in the positive information. Finally, the review task description is presented in the following order: first, the text to be labeled; then, the key information of the preliminary labeled data; and finally, the label definition. Each part is separated by a newline to ensure logical coherence, so that the natural language model can understand the review requirements and all the evidence in sequence.

[0079] Step 511: Input the review prompt information into the natural language model to compare the consistency between the preliminary labeled data and the label definition, forming preliminary review opinions. Specifically, the natural language model first analyzes the review prompt information part by part, clarifying the three operational objectives of the review task, then understanding the emotional content of the text to be labeled word by word, identifying comfortable fabric, good-looking design, and timely customer service response as positive information, and many loose threads and late logistics as negative information. Next, it extracts the core information of the preliminary labeled data, labeling tags, positive and negative segments, etc., and finally remembers the core feature of mixed positive and negative sentiment bias in the label definition. Subsequently, the natural language model compares the consistency in three steps: the first step compares the matching of the labeled tags with the label definition; the preliminary labeled tags are mixed positive sentiment bias, requiring further investigation. The first step involved checking if the number of positive information words or strong positive words exceeded the number of negative ones. The initial positive segment had a total of 13 words; the clothing fabric was very comfortable and the style was nice. The negative segment had a total of 5 words, with a few loose threads. However, there was still negative information in the text to be annotated that had not been included. The logistics were two days later than expected, with a total of 8 words and positive information. The customer service response was quite timely, with a total of 7 words. The initial annotation had some missing information. The second step involved comparing the completeness of the positive and negative segments with the text to be annotated. The initial positive segment omitted the timely customer service response, and the negative segment omitted the two-day delay in logistics, resulting in incomplete coverage of emotional information. The third step involved recalculating the semantic weights. After supplementing the missing information, the total number of positive words was 13 plus 7 equals 20 words, and the total number of negative words was 5 plus 8 equals 13 words.

[0080] Finally, the natural language model integrates and compares the results to form preliminary review opinions. For example, the initially labeled positive and negative segments have missing information. The positive segment was not included because the customer service response was quite timely, while the negative segment was not included because the logistics were two days later than expected, resulting in incomplete coverage of sentiment information. If the missing information is supplemented, the total number of words for the positive segment is 20, and the total number of words for the negative segment is 13. There is 1 strongly positive word for the positive segment and 0 strongly negative words for the negative segment. The initial labeling of the mixed sentiment with a positive bias is consistent with the core features defined by the label. The semantic and positional matching degree between the labeled positive and negative segments and the text to be labeled is good, with no contradictions or excessive deviations.

[0081] Step 512: Based on the preliminary review opinions, perform semantic calibration on the preliminary labeled data that has inconsistencies or semantic ambiguities, and generate a semantic calibration decision. Specifically, this includes: analyzing the core issues of the preliminary review opinions, where the preliminary labeling omitted some sentiment information in the text to be labeled. For example, the positive aspect was timely customer service response, while the negative aspect was a two-day delay in logistics. Although this did not affect the consistency between the labels and the definitions, it resulted in incomplete sentiment information, requiring supplementation to ensure the comprehensiveness of the labeling. Then, a calibration direction was determined. The first step was to supplement the omitted positive and negative information, incorporating the timely customer service response into the actual positive segment and the two-day delay in logistics into the actual negative segment. The second step was to re-count the total number of words and strong words in the positive and negative segments to verify whether the labeled tags still conformed to the definitions. The third step was to retain the labels and only update the segment content if the labels were still appropriate after verification; if the labels were inappropriate, the labels were adjusted, and then verification was performed. After supplementation, the total number of words for positive was 20, and for negative it was 13; there was one strong positive word and zero negative words, which conforms to the characteristic that the positive semantic weight is greater than the negative in a mixed sentiment positive bias, and the labels do not need to be adjusted.

[0082] Finally, a semantic calibration decision is generated, clarifying the specific operations and basis. For example, when supplementing positive fragments, add "The customer service response was quite timely" to the actual positive fragments to ensure coverage of all positive sentiment information; when supplementing negative fragments, add "The logistics were two days later than expected" to the actual negative fragments to ensure coverage of all negative sentiment information; retain the annotation labels; after the supplementation, the total number of positive words is 20 more than the 13 words of negative words, and the number of strongly positive words is more than the number of strongly negative words, which still meets the label definition of mixed sentiment with a positive bias, so no label adjustment is required; the calibration basis is based on the complete sentiment information of the text to be annotated, following the principle that the annotation must cover all sentiment points, and the labels after the supplementation are consistent with the definition.

[0083] Step 513: Based on the semantic calibration decision, calibrate the preliminary labeled data accordingly to obtain the calibrated labeled results. Specifically, this includes: Modifying the corresponding fields of the preliminary labeled data one by one according to each item in the semantic calibration decision. The first calibration operation modifies the "Actual Positive Fragment" field. After adding "The fabric is very comfortable, the style is good, and the customer service response is quite timely," the field content becomes "The fabric is very comfortable, the style is good, and the customer service response is quite timely," ensuring complete correspondence with all positive information in the text to be labeled. The second calibration operation modifies the "Actual Negative Fragment" field. After adding "There are too many loose threads, the logistics is two days later than expected," the field content becomes "There are too many loose threads, and the logistics is two days later than expected," ensuring coverage of all negative information. The third calibration operation updates the "Semantic Comparison Details" field, supplementing the semantic comparison results of the newly added fragments, such as... The newly added positive segment, "Customer service replies were quite timely," aligns with the original text's core keywords "customer service," "replies," and "timely," and the collocation is consistent. The newly added negative segment, "Logistics was two days later than expected," also aligns with the original text's core keywords "logistics," "late," and the collocation is consistent, with no overall semantic contradiction. The fourth calibration operation involves updating the "Location Comparison Details" field to mark the location boundaries of the newly added segments. In the text to be annotated, the corresponding sequence numbers for "Customer service replies were quite timely" are 36 to 46. After comparing the marked boundaries with the actual boundaries, the matching ratio is recorded as 1, with 0 characters of deviation. The corresponding sequence numbers for "Logistics was two days later than expected" are 25 to 35, with a matching ratio of 1 and 0 characters of deviation. The fifth calibration operation maintains the mixed sentiment positive bias in the label field. Since it still meets the definition after verification, all modifications are completed, and the content of each field is integrated to form the calibrated labeling results, ensuring that each modification accurately corresponds to the calibration decision without omissions or errors.

[0084] Step 514: Verify the accuracy and completeness of the calibrated annotation results to obtain the final annotation data, including the final annotation labels and corresponding original text fragments. Specifically, this includes: First, determining the two core dimensions for verification to ensure that the calibrated results are both accurate and complete. The accuracy verification consists of three steps: The first step is to check the consistency between the annotation labels and the label definitions. For mixed sentiment with a positive bias, it is necessary to satisfy the requirement that there is both positive and negative information, and the positive semantic weight is greater than the negative. After verification and supplementation, the total number of positive words is 20 and the number of negative words is 13, with 1 strong positive word and 0 negative words, which meets the definition. The second step is to check the matching of all fragments with the text to be annotated. For example, the content of "the fabric of the clothes is very comfortable, the style is also good, the customer service responded promptly, there are a lot of loose threads, and the logistics was two days late" can all be found in the original text with completely corresponding content, without any typos or alterations. The third step is to check the position boundaries. The position matching ratio of all fragments is 1, and the number of words with deviation is 0, which meets the position standard.

[0085] The integrity verification is divided into two steps. The first step is to check whether all the sentiment information of the text to be labeled is covered. All five sentiment points in the original text have been included in the corresponding segments without any omissions. The second step is to check whether the fields of the labeling results are complete. The fields such as labeling tags, original text to be labeled, actual positive segments, actual negative segments, semantic comparison details, position comparison details, and verification results all have complete content without any gaps or ambiguities.

[0086] After verification, the calibrated annotation results will be determined as the final annotation data. This data must explicitly include the following: the final annotation label has a mixed positive sentiment bias, corresponding to the positive paragraph of the original text; the fabric of the clothes is very comfortable and the style is good; the customer service response is quite timely; the negative paragraph has a lot of loose threads; the logistics is two days later than expected; and key information in the verification process, such as word count comparison and position matching results.

[0087] This embodiment constructs a review prompt message that includes all information, reducing the possibility of biased judgments due to missing information. The accuracy and completeness verification can further identify potential problems after calibration, improving the reliability of the final result. The final labeled data is associated with tags and corresponding original text fragments, making the sentiment annotation of e-commerce reviews closer to real emotional expression and improving the annotation quality.

[0088] In a preferred embodiment of the present invention, step 6 above, which updates the dynamic tag description library based on the final annotation data to achieve continuous adjustment of the annotation system, may include:

[0089] Step 610: Based on the final labeled data, classify and analyze the deviations that occurred during the labeling process, identify problems such as ambiguous label definitions, ambiguous task descriptions, or insufficient label coverage, and generate a problem analysis report. Specifically, this includes: first, determining the three core analysis directions of deviations, corresponding to ambiguous label definitions, ambiguous task descriptions, and insufficient label coverage, respectively, to ensure comprehensive analysis and relevance to the e-commerce review sentiment labeling scenario. The first step is to collect deviation examples from the final labeled data, filtering out cases with disputes or modification records during the labeling process. For example, a review might have a "mediocre customer service attitude" and no particular impression of the product quality. Initially, some might label it as a mixed sentiment neutral bias, but the existing label library lacks neutral bias labels, so it can only be forcibly classified as a mixed sentiment positive bias. Such cases are included as candidates for insufficient label coverage. Another example is a product that looks good but has extremely slow logistics; some might consider it to fit a mixed sentiment positive bias, while others might consider it to fit a mixed sentiment negative bias. These are cases of disputes caused by unclear dimensional weights in the mixed sentiment definition. For example, candidates with ambiguous label definitions are included. The second step is to classify and statistically analyze the deviation instances. First, the total number of cases for the three types of deviations is counted. Assuming a total of 40 deviation cases are collected, there are 12 cases with ambiguous label definitions, 8 cases with ambiguous task descriptions, and 20 cases with insufficient label coverage. Then, the proportion of each type of deviation case to the total number of deviation cases is calculated by dividing the number of each type of deviation case by the total number of deviation cases. For example, the proportion of cases with ambiguous label definitions is 12 divided by 40. The result is used to determine the degree of influence of each type of deviation. Simultaneously, specific instance details are added for each type of deviation. For example, among the cases with ambiguous label definitions, 8 are due to unclear dimension weights in mixed sentiment analysis, and 4 are due to unclear scope of strongly positive words. Among the cases with ambiguous task descriptions, 6 are due to the task description not clearly specifying whether damaged packaging should be classified as a product quality dimension, and 2 are due to the failure to explain whether delayed pre-sale shipments should be considered a logistics dimension. Among the cases with insufficient label coverage, 15 lack neutral sentiment labels, and 5 lack detailed mixed sentiment labels.

[0090] The third step is to analyze the root causes of the deviations. The reason for the ambiguous label definitions is that existing label definitions only mention semantic weights without providing specific judgment criteria such as dimension priority and word strength thresholds. The reason for the ambiguous task descriptions is that existing task descriptions do not clearly list all evaluation dimensions and subcategories. The reason for the insufficient label coverage is that the existing label library only includes positive, negative, mixed sentiment positive bias, and mixed sentiment negative bias, without considering neutral sentiment and subdivided mixed sentiment scenarios. The fourth step is to generate a problem analysis report, which includes three parts: first, the types and percentages of deviations, clearly defining the number and percentage of cases for each type of deviation; second, specific examples and problem descriptions, with 3-5 typical cases attached to each deviation type, explaining the points of contention; and third, the analysis of the causes of deviations, explaining the root causes for each type of deviation to ensure the report is comprehensive.

[0091] Step 611: Based on the problem analysis report, formulate an update strategy for the dynamic tag description library. Specifically, this includes: First, developing an update strategy to address the issue of ambiguous tag definitions. Based on the reasons for the ambiguity in the report, determine the update direction as refining tag definitions and supplementing actionable judgment criteria. For example, regarding the definition of mixed sentiment with a positive bias, in addition to the existing weight of positive semantics being greater than negative, supplement this by prioritizing the sentiment tendency of the core dimensions of the product, with the core dimensions having a higher weight than non-core dimensions. If the core dimension is positive and the non-core dimension is negative, it should be prioritized as a positive bias. Simultaneously, clarify that strong positive words include "very," "extremely," and "super," and strong negative words include "extremely," "too much," and "terrible," etc., and specify the number of strong words. The second step involves developing an update strategy to address ambiguities in task descriptions. Based on the report's issue of unclear evaluation dimensions in task descriptions, the task description text is updated to clearly list all evaluation dimensions and subcategories. For example, the original task descriptions for sentiment annotation of e-commerce reviews are updated to annotate the core dimensions of products (quality, style, function, packaging) and service dimensions (logistics, customer service, after-sales service) in e-commerce reviews. Damaged packaging is categorized as a core product dimension, delayed pre-sale shipments as a logistics dimension, and slow customer service response as a customer service dimension, thus avoiding ambiguity caused by unclear dimension classifications.

[0092] The third step is to develop an update strategy to address the issue of insufficient tag coverage. Based on the report's lack of neutral and granular mixed sentiment tags, the types of new tags to be added are determined: First, neutral sentiment tags are added, defined as comments with no obvious positive or negative bias, or where positive and negative biases are of equal strength and have no core dimension priority, such as product color matching the picture, neither particularly surprising nor disappointing. Second, granular mixed sentiment tags are added, such as positive product but negative logistics, positive customer service but negative product, etc., defined as clearly distinguishing the sentiment bias of two core dimensions, facilitating accurate analysis of satisfaction across dimensions. Simultaneously, update priorities are determined, with cases of deviation due to insufficient tag coverage accounting for the highest proportion, making them the primary update task; vague tag definitions are secondary, and ambiguous task descriptions are last. The fourth step is to clarify the execution details of the update strategy, including the updated content such as tag definition text, task description text, and a list of new tags; the execution order is to first add tags, then refine the definitions, and finally update the task descriptions; the verification standard is to ensure that the new tags do not conflict with the original tags and that the new definitions are unambiguous.

[0093] Step 612: Execute the update strategy, adjusting and updating the tag set and task description text in the dynamic tag description library to form an adjusted and updated tagging system. Specifically, this includes: First, updating the tag set. Adding new tags: In addition to the existing positive, negative, mixed sentiment positive bias, and mixed sentiment negative bias tags, add neutral sentiment and five sub-categories of mixed sentiment tags: positive for products and negative for logistics, positive for customer service and negative for products, resulting in a total of 10 tags. Then, complete the definitions for the new tags. For example, neutral sentiment is defined as comments with no obvious positive or negative expressions, or where the intensity and word count of positive and negative expressions differ by less than 10%, and there is no difference in priority between core dimensions; positive for products and negative for logistics is defined as comments where the core product dimension is positive and the logistics dimension is negative, and both types of expressions are relatively clear and need to be consistent. First, the two dimensions of sentiment are labeled. Then, the compatibility between the new and existing labels is checked to ensure there is no semantic overlap or contradiction. For example, the difference between neutral sentiment and mixed sentiment lies in whether there is a positive or negative bias, to avoid confusion. Second, the original label definitions are refined. According to the standards determined in the update strategy, the definition text of mixed sentiment positive bias and mixed sentiment negative bias is modified, and the dimension priority and vocabulary strength standards are supplemented. For example, mixed sentiment positive bias; if the comments contain both positive and negative expressions, if the core dimension of the product is positive and the non-core dimension is negative, it is given priority to be positive bias. If the core dimension has no obvious bias, the number of strong words in the positive expression is more than that in the negative expression, or the number of words in the positive expression is more than 10% higher than that in the negative expression, it is also judged as positive bias. After modification, relevant personnel are organized to interpret and test the new definitions to ensure there is no ambiguity.

[0094] The third step is to update the task description text, incorporating the determined detailed evaluation dimensions and subcategories into the task description to form a new task description text. This e-commerce review sentiment labeling needs to cover the core dimensions of the product: quality, style, function, and packaging. Damaged packaging is categorized as quality-related. The service dimension includes logistics, including delayed pre-sale shipments. Customer service includes response speed. After-sales service includes returns and exchanges. The sentiment type needs to be determined based on the tag definitions. Neutral sentiment reviews are labeled separately, and for mixed sentiment reviews, the tendency of the two core dimensions needs to be clearly defined. After updating, compare with the original task description to ensure all ambiguities are clarified. The fourth step is to integrate the adjusted content to form a new labeling system. The updated tag set, including newly added tags and definitions, and the new task description text, is integrated into the dynamic tag description library, while recording the update time and content.

[0095] Step 613: Generate new task guidelines and tag options based on the adjusted and updated annotation system to achieve continuous adjustment of the annotation system. Specifically, this includes: First, generating new task guidelines. These guidelines must contain all key information from the updated annotation system, structured as follows: Annotation Objectives, Evaluation Dimension Descriptions, Tag Definitions, and Annotation Examples. The Annotation Objectives section clearly and accurately annotates the sentiment tendencies of each dimension in e-commerce reviews, including neutral and subdivided mixed sentiments. The Evaluation Dimension Descriptions section details the subcategories and affiliations of the core product dimensions and service dimensions. The Tag Definitions section fully presents the new definitions of all tags, including neutral sentiment, positive product sentiment, negative logistics sentiment, and other newly added tags. The Annotation Examples section provides 2-3 typical cases for each tag, such as neutral sentiment... Example: The product size is just right, with no other special feelings. A positive example of a product with negative logistics feedback is: The clothes have a great fit, but the delivery was five days late. Meanwhile, the instructions indicate updated content, such as the addition of a neutral sentiment tag, refined criteria for judging mixed sentiment, and clear attribution of evaluation dimensions, helping users quickly understand the changes. The second step is to generate new tag options. Based on the updated tag set, a list of directly selectable tags is compiled, including positive sentiment, negative sentiment, neutral sentiment, mixed sentiment with a positive bias, mixed sentiment with a negative bias, positive product with negative logistics feedback, positive product with negative customer service feedback, positive logistics with negative product feedback, positive customer service with negative product feedback, and positive customer service with negative logistics feedback. This ensures that each tag option completely corresponds to the definition in the dynamic tag description library, without omissions or errors.

[0096] The third step is to synchronize the new task guidelines and tag options to the annotation process. Upload the new task guidelines to the annotation personnel's work platform, conduct brief training sessions explaining key update points such as the usage scenarios of neutral tags and the methods for judging subdivided mixed tags. Embed the new tag options into the annotation tool's selection menu, replacing the original options to ensure that annotation personnel can only select the updated tags during annotation, avoiding deviations caused by using old tags. Simultaneously, add a tag definition query function to the annotation tool; clicking on a tag option will display the corresponding detailed definition, facilitating easy reference for annotation personnel. The fourth step is to collect initial usage feedback and confirm the adjustment effects. In the first week of using the new system, collect questions and annotation cases from annotation personnel to check for new ambiguities or omissions. If scenarios involving positive after-sales issues or negative after-sales issues are not covered, supplement the relevant subdivided tags to ensure the annotation system can continuously adapt to actual annotation needs.

[0097] This embodiment addresses issues such as ambiguous labels, unclear descriptions, and insufficient coverage through an update strategy tailored to deviations. The mechanism for continuously collecting feedback allows the labeling system to be gradually optimized as user expression habits and product types change. It also has positive implications for improving the shortcomings of static label libraries in adapting to dynamic scenarios.

[0098] like Figure 2As shown, embodiments of the present invention also provide a data annotation and processing system based on a natural language model, comprising:

[0099] The building module is used to build a dynamic tag description library, which stores the task name, tag name, tag definition and task description text of multiple annotation tasks, forming an extensible annotation knowledge base;

[0100] The generation module is used to generate structured prompts based on a scalable annotation knowledge base, including complete task descriptions, detailed label definitions, and strict output format requirements.

[0101] The acquisition module is used to input structured prompts and text to be labeled into a pre-trained natural language model. Through the semantic understanding and reasoning of the natural language model, the module acquires the response content of the natural language model, including the preliminary labeling results and the basis for judgment.

[0102] The annotation module is used to parse and verify the consistency of the natural language model response content, compare the preliminary annotation results with the corresponding original text fragments of the text to be annotated, and generate preliminary annotation data.

[0103] The verification module is used to re-input the preliminary labeled data, the text to be labeled, and the label definition into the natural language model for verification and semantic calibration, and generate the final labeled data.

[0104] The update module is used to update the dynamic label description library based on the final labeled data, so as to realize the continuous adjustment of the labeling system.

[0105] It should be noted that this system is a system corresponding to the above method. All implementation methods in the above method embodiments are applicable to this embodiment and can achieve the same technical effect.

[0106] Embodiments of the present invention also provide a computing device, including: a processor and a memory storing a computer program, wherein the computer program, when executed by the processor, performs the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0107] Embodiments of the present invention also provide a computer-readable storage medium storing instructions that, when executed on a computer, cause the computer to perform the method described above. All implementations in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0108] The above description represents the preferred embodiments of the present invention. It should be noted that those skilled in the art can make various improvements and modifications without departing from the principles of the present invention, and these improvements and modifications should also be considered within the scope of protection of the present invention.

Claims

1. A data annotation and processing method based on natural language models, characterized in that, The method includes: Step 1: Construct a dynamic label description library to store the task names, label names, label definitions, and task description text for multiple labeling tasks, forming an extensible labeling knowledge base, including: Receive the task creation instruction, and generate and store the corresponding task name set based on the multiple task identifiers included in the instruction; Based on the set of task names, one or more associated tag items are matched for each task name to construct a structured task tag mapping table; the semantic similarity between each tag item in the task tag mapping table is calculated, and a semantically consistent tag grouping structure is constructed based on the semantic similarity analysis results; The task label mapping table is reconstructed based on the label grouping structure to form a reorganized task label mapping table; according to the label item organization structure in the reorganized task label mapping table, a corresponding detailed label definition is created and stored for each label item to form a label definition dictionary. Based on the task name set and task tag mapping table, the corresponding task description text is automatically generated for each task name, and a task description text library is built. The task name set, task tag mapping table, tag definition dictionary and task description text library are integrated and linked, and stored in a dynamically updated structured database to form an extensible annotation knowledge base. Step 2: Based on a scalable annotation knowledge base, generate structured prompts including a complete task description, detailed label definitions, and strict output format requirements; Step 3: Input the structured prompts and the text to be labeled into the pre-trained natural language model. Through the semantic understanding and reasoning of the natural language model, obtain the natural language model response content, including preliminary labeling results and judgment criteria, including: The structured prompts and the text to be labeled are integrated according to the preset input format to form a complete input sequence including task description, label definition, output format requirements and the text to be labeled; By utilizing the semantic encoding layer of a natural language model, multi-level semantic understanding is performed on the complete input sequence, and deep feature representations of task semantics and text semantics are extracted. Based on deep feature representation, the reasoning and decision-making layer of the natural language model is used for analysis to generate the original response content, which includes the preliminary annotation results of the text to be annotated and the corresponding judgment basis. The original response content is subjected to structured parsing using preset analysis rules, and the specific content of the preliminary annotation results and the detailed description of the judgment basis are extracted respectively. The initial annotation results are associated and matched with the judgment criteria, and quality assessment parameters are generated based on the semantic matching degree between the annotation results and the label definition. Finally, a complete response content including annotation results, judgment criteria and quality assessment parameters is formed. Step 4: Parse and verify the consistency of the natural language model response content, compare the preliminary annotation results with the corresponding original text fragments of the text to be annotated, and generate preliminary annotation data; Step 5: Input the preliminary labeled data, the text to be labeled, and the label definition back into the natural language model for verification and semantic calibration to generate the final labeled data; Step 6: Based on the final labeled data, update the dynamic label description library to achieve continuous adjustment of the labeling system.

2. The data annotation and processing method based on a natural language model according to claim 1, characterized in that, Based on a scalable annotation knowledge base, it generates structured prompts including a complete task description, detailed label definitions, and strict output format requirements, including: Receive processing requests for annotation tasks, analyze the task identifiers in the processing requests, and generate task query instructions. Based on the task query command, the corresponding task description text is retrieved from the dynamic tag description library using the task identifier, and a set of tag names associated with the task is obtained. Based on the set of label names, the corresponding detailed label definitions are extracted from the label definition dictionary to form a complete set of label definitions for the target labeling task. By integrating the task description text with the complete set of label definitions, the core semantic content of the annotation task is constructed; The core semantic content is combined with the preset standardized output format requirements to generate an initial structured prompt framework. The initial structured prompt framework is then integrated and adjusted to generate the final structured prompt information.

3. The data annotation and processing method based on a natural language model according to claim 2, characterized in that, The natural language model response is parsed and its consistency is verified. The preliminary annotation results are compared with the corresponding original text fragments to be annotated, generating preliminary annotation data, including: The response content of the natural language model is structured and parsed to extract preliminary annotation results and corresponding judgment criteria; The original text fragments cited in the preliminary annotation results are compared semantically with the corresponding actual original text fragments in the text to be annotated to obtain semantic comparison results used to characterize the semantic consistency between the two. The text position boundaries marked by the preliminary annotation results are compared with the position boundaries of the corresponding original text segments in the text to be annotated, and position comparison results are generated to characterize the degree of position matching between the two. The semantic comparison results and the position comparison results are integrated to perform consistency verification on the preliminary annotation results. Invalid annotations that have semantic contradictions or positional boundary deviations with the original text are identified and excluded, and a set of verified annotation results is generated. Based on the validated set of annotation results, structured preliminary annotation data is constructed.

4. The data annotation and processing method based on a natural language model according to claim 3, characterized in that, The initial labeled data, the text to be labeled, and the label definitions are then input back into the natural language model for review and semantic calibration, generating the final labeled data, including: Based on the preliminary annotation data, the text to be annotated, and the label definition, construct a prompt message for review; The review prompts are input into the natural language model to compare the consistency between the initial labeled data and the label definitions, and to form preliminary review opinions. Based on the preliminary review opinions, semantic calibration processing is performed on the preliminary labeled data that has inconsistencies or semantic ambiguities to generate semantic calibration decisions; Based on the semantic calibration decision, the preliminary labeled data is calibrated accordingly to obtain the calibrated labeled results; The accuracy and completeness of the calibrated annotation results are verified to obtain the final annotation data, including the final annotation labels and the corresponding original text fragments.

5. The data annotation and processing method based on a natural language model according to claim 4, characterized in that, Based on the final labeled data, the dynamic label description library is updated to achieve continuous adjustment of the labeling system, including: Based on the final labeled data, the deviations that occurred during the labeling process are classified and analyzed to identify problems such as ambiguous label definitions, unclear task descriptions, or insufficient label coverage, and a problem analysis report is generated. Based on the problem analysis report, formulate an update strategy for the dynamic tag description library; The update strategy is executed to adjust and update the tag set and task description text in the dynamic tag description library, forming an adjusted and updated tagging system. New task guidelines and tag options are generated based on the adjusted and updated annotation system, enabling continuous adjustment of the annotation system.

6. A data annotation and processing system based on a natural language model, wherein the system implements the method as described in any one of claims 1 to 5, characterized in that, include: The building module is used to build a dynamic tag description library, which stores the task name, tag name, tag definition and task description text of multiple annotation tasks, forming an extensible annotation knowledge base; The generation module is used to generate structured prompts based on a scalable annotation knowledge base, including complete task descriptions, detailed label definitions, and strict output format requirements. The acquisition module is used to input structured prompts and text to be labeled into a pre-trained natural language model. Through the semantic understanding and reasoning of the natural language model, it acquires the response content of the natural language model, including the preliminary labeling results and the basis for judgment. The annotation module is used to parse and verify the consistency of the natural language model response content, compare the preliminary annotation results with the corresponding original text fragments of the text to be annotated, and generate preliminary annotation data. The verification module is used to re-input the preliminary labeled data, the text to be labeled, and the label definition into the natural language model for verification and semantic calibration, and generate the final labeled data. The update module is used to update the dynamic label description library based on the final labeled data, so as to realize the continuous adjustment of the labeling system.

7. A computing device, characterized in that, include: One or more processors; A storage device for storing one or more programs, which, when executed by one or more processors, cause the one or more processors to implement the method as described in any one of claims 1 to 5.

8. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores a program that, when executed by a processor, implements the method as described in any one of claims 1 to 5.

Images