Text data automatic labeling method and system integrating pre-training NLP model

Abstract

The invention provides a text data automatic labeling method and system integrating a pre-training NLP model, and relates to the technical field of artificial intelligence, the method comprises the following steps: obtaining original text data to be labeled, the text data comprising customer consultation, complaint or feedback dialogue records, evaluations and work orders; performing word segmentation, stop word filtering and part-of-speech tagging processing on the original text data to generate structured text data; inputting the structured text data into a pre-trained NLP model, and correcting semantic features output by the model through a domain adaptation layer to obtain corrected semantic features; and carrying out few-sample learning on the corrected semantic features, enhancing the recognition capability on a low-frequency text mode through a contrast learning mechanism, and generating an enhanced feature vector resistant to long-tail interference. According to the method, automatic annotation of the text data is realized by automatically processing data acquisition and preprocessing and integrating the pre-training NLP model.

Description

Technical Field

[0001] The present invention relates to the field of artificial intelligence technology, and particularly to a method and system for automatic annotation of text data integrating a pre-trained NLP model. Background Art

[0002] In the fields of customer service and text analysis, automatic annotation of text data such as customer consultations, complaints, etc. is a key technology for realizing intelligent customer service, service quality evaluation, and problem traceability. However, traditional text annotation methods may have many technical bottlenecks: models based on supervised learning are affected by the "long-tail effect", and for problem types that rarely appear in customer service, the recognition accuracy may decrease due to insufficient training data, and it may be difficult to meet the requirement of covering all problems.

[0003] Pre-trained NLP models in the field of customer service may have problems such as domain term deviation and lack of context dependence, which may lead to distortion of semantic feature extraction and affect the accuracy of label extraction; multi-dimensional label extraction sometimes uses independent models for processing, which may have problems of lack of label consistency and waste of computing resources, and it is difficult to adapt to customer service scenarios with high real-time requirements; model iteration is sometimes lagging behind business changes, which may have problems of timeliness lag and low data utilization rate. Summary of the Invention

[0004] The technical problem to be solved by the present invention is to provide a method and system for automatic annotation of text data integrating a pre-trained NLP model, which realizes automatic annotation of text data by automatically processing data acquisition and preprocessing and integrating a pre-trained NLP model.

[0005] To solve the above technical problems, the technical solution of the present invention is as follows: In a first aspect, a method for automatic annotation of text data integrating a pre-trained NLP model, the method includes: Step S1: Obtain the original text data to be annotated, where the text data includes conversation records, evaluations, and work orders of customer consultations, complaints, or feedback; Step S2: Perform word segmentation, stop word filtering, and part-of-speech tagging on the original text data to generate structured text data; Step S3: Input the structured text data into the pre-trained NLP model, and correct the semantic features output by the model through a domain adaptation layer to obtain corrected semantic features; Step S4: Perform few-shot learning on the corrected semantic features, and strengthen the recognition ability of low-frequency text patterns through a contrast learning mechanism to generate enhanced feature vectors resistant to long-tail interference; Step S5: Based on the enhanced feature vector, synchronously perform multi-dimensional label extraction operations, and output a label set containing the following four types of labels: requirement type label, sentiment tendency label, product or service entity label, and problem keyword label; Step S6: For each label in the label set, locate its key decision words in the enhanced feature vector through the attention mechanism, generate a natural language description of the basis for label determination, and form an enhanced label set with explanatory text; Step S7: Trigger the incremental training of the pre-trained NLP model based on the distribution of low-frequency labels and the new frequency of domain terms in the enhanced label set.

[0006] In a second aspect, a text data automatic annotation system integrating a pre-trained NLP model includes: An acquisition module, configured to acquire the original text data to be annotated, where the text data includes conversation records, evaluations, and work orders of customer consultations, complaints, or feedback; A preprocessing module, configured to perform word segmentation, stop word filtering, and part-of-speech tagging on the original text data to generate structured text data; A correction module, configured to input the structured text data into the pre-trained NLP model, and correct the semantic features output by the model through the domain adaptation layer to obtain corrected semantic features; A feature vector module, configured to perform few-shot learning on the corrected semantic features, strengthen the recognition ability of low-frequency text patterns through the contrast learning mechanism, and generate an enhanced feature vector resistant to long-tail interference; A label module, configured to based on the enhanced feature vector, synchronously perform multi-dimensional label extraction operations, and output a label set containing the following four types of labels: requirement type label, sentiment tendency label, product or service entity label, and problem keyword label; An interpretive module, configured to for each label in the label set, locate its key decision words in the enhanced feature vector through the attention mechanism, generate a natural language description of the basis for label determination, and form an enhanced label set with explanatory text; An incremental module, configured to trigger the incremental training of the pre-trained NLP model based on the distribution of low-frequency labels and the new frequency of domain terms in the enhanced label set.

[0007] In a third aspect, a computing device includes: One or more processors; A storage device, configured to store one or more programs, and when the one or more programs are executed by the one or more processors, enable the one or more processors to implement the method described above.

[0008] Fourth aspect, a computer-readable storage medium stores a program which, when executed by a processor, implements the method described above.

[0009] The above solution of the present invention has at least the following beneficial effects: The whole process automates data acquisition and preprocessing, connects to the data source in real time and automatically sorts and stores it, integrates preprocessing steps such as garbled cleaning and word segmentation, improves efficiency compared with traditional manual annotation, and at the same time generates four types of labels synchronously through a multi-task classifier, saving computing resources and shortening the annotation time; accurately responds to the challenges of domain characteristics and long-tail data. The domain adaptation layer dynamically adjusts the feature weights based on the term library, improves the accuracy of domain-specific vocabulary, and combines few-shot learning and contrastive learning mechanisms to strengthen the recognition of low-frequency text patterns, enhancing the sensitivity of low-frequency semantic responses; the interpretability and business compliance are significantly strengthened. Natural language explanations of label decisions are generated through the attention mechanism, and incremental training is triggered based on the low-frequency label ratio or term library update rate to adapt to business requirements in real time. BRIEF DESCRIPTION OF THE DRAWINGS

[0010] Figure 1 is a schematic flowchart of a method for automatically annotating text data integrating a pre-trained NLP model provided by an embodiment of the present invention.

[0011] Figure 2 is a schematic diagram of a system for automatically annotating text data integrating a pre-trained NLP model provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0012] Hereinafter, exemplary embodiments of the present disclosure will be described in more detail with reference to the drawings. Although the exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited by the embodiments set forth herein. On the contrary, these embodiments are provided so that the present disclosure can be more thoroughly understood and the scope of the present disclosure can be completely conveyed to those skilled in the art.

[0013] As Figure 1 shown, an embodiment of the present invention proposes a method for automatically annotating text data integrating a pre-trained NLP model, and the method includes the following steps: Step S1: Obtain the original text data to be annotated, and the text data includes conversation records, evaluations, and work orders of customer consultations, complaints, or feedback; Step S2: Perform word segmentation, stop word filtering, and part-of-speech tagging on the original text data to generate structured text data; Step S3: Input the structured text data into the pre-trained NLP model, and correct the semantic features output by the model through the domain adaptation layer to obtain corrected semantic features; Step S4: Conduct few-shot learning on the corrected semantic features, strengthen the recognition ability of low-frequency text patterns through the contrast learning mechanism, and generate enhanced feature vectors resistant to long-tail interference; Step S5: Based on the enhanced feature vectors, synchronously perform multi-dimensional label extraction operations, and output a label set containing the following four types of labels: demand type labels, sentiment tendency labels, product or service entity labels, and problem keyword labels; Step S6: For each label in the label set, locate its key decision words in the enhanced feature vectors through the attention mechanism, generate a natural language description of the basis for label determination, and form an enhanced label set with explanatory text; Step S7: Trigger the incremental training of the pre-trained NLP model based on the low-frequency label distribution and the new frequency of domain terms in the enhanced label set.

[0014] In the embodiment of the present invention, by collecting multi-type raw data such as customer consultations, complaints, evaluations, and work orders, the limitations of a single data dimension are avoided; through word segmentation, stop word filtering, and part-of-speech tagging, unstructured text is converted into structured data that can be understood by machines, redundant information is removed, data noise is reduced, and the processing efficiency and accuracy of subsequent models are improved; at the model optimization level, the semantic understanding is made precise with the help of the domain adaptation layer, the long-tail problem is overcome by using few-shot and contrast learning, and incremental training is triggered based on the low-frequency label and the new frequency of terms to dynamically adapt to business changes; at the label value level, a multi-dimensional label system including demand types, sentiment tendencies, etc. is constructed to support refined operations, and enhanced labels with natural language explanations are generated through the attention mechanism to enhance human-machine collaborative trust.

[0015] In a preferred embodiment of the present invention, the above-mentioned step S1: Obtain the original text data to be labeled, and the text data includes conversation records, evaluations, and work orders of customer consultations, complaints, or feedbacks, may include: Step S11, obtain the original text data to be labeled from the e-commerce platform customer service system in real time, and the text data includes conversation records, evaluations, and work orders of customer consultations, complaints, or feedbacks; Step S12, after sorting by timestamp, store the original text data in a structured database and label the data source type.

[0016] In the embodiment of the present invention, by obtaining customer feedback data in real time, storing it structurally, and integrating multiple sources, the latest customer feedback data can be obtained in a timely manner to quickly respond to business needs; at the same time, the structured storage is convenient for subsequent data management and analysis, effectively improving the processing efficiency; the multi-source integration realizes the unified management of multiple types of data such as conversations, evaluations, and work orders, comprehensively covering the entire process of customer feedback.

[0017] In the embodiments of the present invention, when specifically applied, it can be achieved through the following technical solutions. For example: In step S11 above, connect to the customer service system of the e-commerce platform (such as Qianniu, JD.com customer service) through the API interface, and pull the conversation records between customers and customer service in real time (including text, text translated from emoticons), product reviews (such as follow-up reviews, text extraction of reviews with pictures), and after-sales work orders (such as return and exchange application descriptions, dispute records).

[0018] In step S12 above, sort by timestamp and store in a structured database (such as MySQL), and mark the data source type (such as consultation, complaint, review, or work order).

[0019] In a preferred embodiment of the present invention, step S2: perform word segmentation, stop word filtering, and part-of-speech tagging on the original text data to generate structured text data, which may include: Step S21, scan each character encoding of the original text data and perform processing to generate a purified text; Step S22, according to the purified text, calculate the probability distribution of each character as the start, middle, or end position of a word at the current position, combine the conditional random field state transition scores, and use dynamic programming to generate the global maximum probability segmentation path, and output a basic word segmentation sequence with clear word boundaries; Step S23, according to the stop word dictionary stored in the hash table, traverse each word in the basic word segmentation sequence. If the word hits the stop word hash table and is not marked as a protected entity by the named entity recognition module, remove the word; if the word belongs to the entity protection list, retain it in the new sequence; generate a filtered word sequence carrying the core semantic units; Step S24, for each target word in the filtered word sequence, extract the context window of the 2 words before and after it, and combine them into a feature template; query the pre-trained part-of-speech probability matrix, and calculate the joint probability of the target word belonging to each part-of-speech category in its context environment; based on dynamic programming, construct a part-of-speech state transition grid, iteratively calculate the cumulative probability of all part-of-speech tagging paths from the first word to the last word in the sequence, and select the path with the maximum probability as the final part-of-speech sequence; combine each word with its optimal part-of-speech tag into a binary tuple to generate structured text data with grammatical marks.

[0020] In an embodiment of the present invention, by purifying the original text, noise interference is eliminated, and standardized input is provided for subsequent word segmentation, thereby improving the accuracy of word segmentation; the word segmentation method based on probability distribution can make full use of contextual features compared to traditional algorithms, effectively solve the problem of ambiguous segmentation, and further improve the accuracy of word segmentation; stop word filtering reduces the interference of meaningless words on subsequent analysis, while protecting key entity information and improving the effectiveness of text semantic representation; and part-of-speech tagging gives grammatical structure information to the text, assists semantic analysis tasks such as dependency syntax analysis and sentiment analysis, and deepens the depth of text understanding.

[0021] In the embodiments of the present invention, when applied specifically, it can be achieved through the following technical solutions, for example: In step S21 , each character of the original text is traversed to check whether its Unicode encoding belongs to a valid character set (such as ASCII, commonly used Chinese character set, etc.), and garbled or illegal characters are removed; punctuation marks are standardized (such as converting full-width to half-width), and invisible control characters (such as line breaks and tabs) are removed or replaced with spaces according to business needs.

[0022] Unify uppercase and lowercase (e.g. convert all English to lowercase), process repeated characters (e.g. "好啊啊啊" → "好啊"), and correct spelling errors (based on pre-trained language models or dictionaries).

[0023] In step S22 , the purified text sequence is received, each character is used as an input unit, and the character is mapped into a multi-dimensional feature vector (such as the character itself, the context, the character type, etc.), and the label (B / M / E / S) corresponding to each character is predicted.

[0024] Design feature templates to capture local and global information of text. Common templates include: Current character features: the character itself, whether it is uppercase, whether it is a number, etc. Contextual features: the first / last 1-2 characters and their combination; Statistical features: character frequency, whether it is a word prefix / suffix in the dictionary; Position feature: the position of the character in the sentence (beginning or end of the sentence).

[0025] Define the legal transfer rules between tags, for example: B→E / M (the beginning of a word can only be followed by the middle or end of a word), M→M / E (the middle of a word can only be followed by the middle or end of a word), E→B / S (the end of a word can only be followed by the beginning of a new word or a single word), S→B / S (the single word can only be followed by the beginning of a new word or a single word).

[0026] Collect large-scale segmented texts (such as Corpus A, Corpus B, etc.), convert the segmentation results into B / M / E / S tag sequences. For example, for the original text: "Bridge D in City C", the segmentation is: "City C / Bridge / D", and the tags are: BEBMEBE; for each character in the training corpus, extract features according to a predefined feature template, and convert the text features into numerical features (such as one-hot encoding).

[0027] Train the model using the maximum likelihood estimation (MLE) or the improved iterative scaling algorithm (IIS). The steps are as follows: randomly initialize the feature weights, input the training samples, calculate the probability of each tag sequence, adjust the feature weights according to the difference between the true tags and the predicted tags, and repeat the training until the model converges (such as the change in the log-likelihood function is less than the threshold).

[0028] Evaluate the model performance using the validation set. Common metrics: Accuracy is the proportion of correctly predicted tags to the total number of tags; the F1 value is an indicator that comprehensively considers precision and recall; the disambiguation ability is to test the segmentation effect of the model on ambiguous texts (such as "The table tennis racket has been sold out").

[0029] Remove features with little contribution to performance to reduce overfitting, add L1 / L2 regularization terms to prevent the model from being too complex, and combine the results of multiple CRF models to improve stability.

[0030] Design features according to linguistic knowledge (such as Chinese word formation rules), select features that contribute greatly to classification through statistical methods; update parameters in small batches to accelerate convergence, stop training when the performance on the validation set no longer improves to prevent overfitting, use a larger learning rate at the beginning of training and reduce it later to achieve stable convergence; based on the general domain model, fine-tune using a small amount of data in the target domain; incorporate the domain dictionary into the feature template (such as a medical term dictionary).

[0031] Extract the same features as during training for the new text, use the trained model parameters, calculate the tag probability of each character, combine the state transition scores, find the optimal tag sequence, and generate word boundaries according to the tag sequence (such as B→E indicating a word).

[0032] In step S23 above, quickly match each word in the basic segmentation sequence with the stop word dictionary stored in the hash table (time complexity O(1)); call the named entity recognition (NER) module to determine whether the word belongs to a protected entity (such as a person's name, a place name, an organization name, etc.); remove the words that match the stop words and are not protected entities, and retain the words in the entity protection list to generate a filtered word sequence.

[0033] In step S24 above, the feature template construction is to convert the target word and its context into a feature representation that can be used for model calculation. The specific process is as follows: For any target word in the filtered word sequence, we create a window containing contextual information. The specific steps are as follows: With the target word as the center, we take two words forward and two words backward, and then add the target word itself to form a window containing five words. This window acts like a "magnifying glass," helping the model to capture the linguistic context surrounding the target word. If the target word is preceded by fewer than two words (for example, if it is the first or second word in the sequence), the empty space in front is filled with the special symbol `BOS` (Begin of Sequence). If the target word is followed by fewer than two words (for example, if it is the penultimate or second-to-last word in the sequence), the empty space after it is filled with the special symbol `EOS` (End of Sequence).

[0034] Example description: Suppose the filtered word sequence is: `["I", "like", "eat", "apple", "and", "banana"]`; when the target word is `"eat"` (position 3), its context window is `["I", "like", "eat", "apple", "and"]` (normal situation, no need to fill); when the target word is `"I"` (position 1), its context window is `["BOS", "BOS", "I", "like", "eat"]` (filled with `BOS` in the front); when the target word is `"banana"` (position 6), its context window is `["and", "apple", "banana", "EOS", "EOS"]` (filled with `EOS` in the back).

[0035] This processing ensures that no matter where the target word is in the sequence, a fixed length of contextual information is obtained, thereby unifying the input format of the model and preserving the key contextual information of boundary words.

[0036] The following features are generated based on the context window: lexical features are the current word itself (such as "apple"), the preceding / following 1-2 words (such as "eat", "very"), and word combinations (such as "eat apple", "apple is very"); morphological features are word length (such as "apple" is 2 in length), whether it contains numbers / letters (such as "iPhone14" contains letters and numbers), prefixes / suffixes (such as words starting with "old" are mostly surnames); statistical features are the frequency of words in the training corpus (such as the frequency of "的" is extremely high), the probability of a word being a specific part of speech (such as the probability of "apple" being a noun); dictionary features are whether it is in a predefined dictionary (such as whether it is a word in a verb dictionary), and the common part of speech of the word in the dictionary (such as "running" is usually a verb).

[0037] For the target word (w_i), the following methods can be used to construct feature templates to capture the key information of itself and its context: Directly use the target word itself as a feature (i.e., (w[i])). For example, when the target word is "apple", this feature represents the word "apple" itself; extract the previous word of the target word (i.e., (w[i - 1])). For example, when the target word is "apple" and the previous word is "eat", then this feature is "eat", which is used to reflect the direct previous context of the target word; extract the next word of the target word (i.e., (w[i + 1])). For example, when the target word is "apple" and the next word is "is very sweet", then this feature is "is very sweet", which is used to reflect the direct next context of the target word; combine the previous word and the current target word into a feature (i.e., (w[i - 1]+w[i])). For example, when the previous word is "eat" and the target word is "apple", then the combined feature is "eat apple".

[0038] Judge whether the target word contains numbers. For example, when the target word is "iPhone14", this feature is "contains numbers"; if it is "apple", then it is "does not contain numbers". This feature helps to distinguish parts of speech (e.g., numbers are often collocated with quantifiers or are part of time or quantity words).

[0039] Through the above feature templates, the context information of the target word can be characterized from different dimensions: there is both the semantic information of the word itself, the collocation relationship of the context before and after, and the morphological features of the word (such as whether it contains numbers). These feature combinations can help the model more accurately judge the part of speech of the target word (for example, "apple" is more likely to be a noun in "eat apple" and more likely to be an adjective in "apple green").

[0040] Map each feature template to a numerical feature (such as one - hot encoding) to form a high - dimensional feature vector. For example: the word "apple" → [0, 0, 1, 0,...] (assuming it is the 3rd word in the vocabulary), the word length = 2 → [0, 1, 0, 0,...] (assuming the length feature dimension is 4).

[0041] The Hidden Markov Model (HMM) is used to model the probability distribution of part - of - speech sequences. The state transition grid is a visual representation of the HMM. The construction steps are as follows: State set (Q): The set of part - of - speech tags (such as nouns, verbs, adjectives, etc.); Observation set (V): The set of words (such as "apple", "eat"); Parameters: The initial probability (pi) is the probability of each part of speech as the first word of a sentence, the state transition probability (A) is the transition probability between parts of speech (such as the probability of noun → verb), and the emission probability (B) is the probability of generating a certain word given a part of speech (such as the probability of a noun generating "apple").

[0042] For each position (from 1 to n) in the corresponding word sequence, each time point contains all possible part-of-speech states. Connect the states of adjacent time points, with the weight being the state transition probability; at time point 1, set the initial probability (pi) for each part-of-speech state; for each subsequent time point (t) (from 2 to n), copy all possible part-of-speech states, and from each state at time point (t - 1), connect to all states at time point (t), with the weight of the edge being the state transition probability (A); at each state node, label the emission probability (B) of the corresponding word generated by this part of speech.

[0043] The Viterbi algorithm finds the optimal path in the constructed state transition grid: At time point 1, calculate the initial score for each part-of-speech state (initial probability × emission probability); for each subsequent time point (t), calculate the cumulative score of transitioning from all possible previous states to the current state, and retain the path with the maximum score; starting from the state with the highest score at time point (n), trace back to time point 1 to generate the optimal part-of-speech sequence and generate the structured text with part-of-speech tags.

[0044] In a preferred embodiment of the present invention, the above step S3: inputting the structured text data into a pre-trained NLP model and correcting the semantic features output by the model through a domain adaptation layer to obtain the corrected semantic features may include: Step S31, input the word sequence in the structured text data generated in step S2 into the embedding layer of the pre-trained NLP model, convert each word into a high-dimensional word vector through word vector mapping operations, and perform context feature extraction through multiple layers of Transformer encoders to output the original semantic feature vector; Step S32, input the original semantic feature vector into the domain adaptation layer, calculate the cosine similarity between each feature dimension of the original semantic feature vector and all term vectors in the term library; select the set of term vectors with similarity exceeding a preset threshold, perform normalized weighted summation based on the similarity values between each term vector in the set and the original feature to generate a domain weight vector; expand the domain weight vector into a diagonal matrix form according to the feature dimensions to form a domain weight matrix; [[ID=;14]] Step S33, perform matrix multiplication on the domain weight matrix and the original semantic feature vector to scale the original features in each dimension according to domain relevance; perform a residual connection on the scaled feature vector and the original semantic feature vector, and fuse the original feature and the domain-enhanced feature through element-wise addition operations to obtain a fused feature vector; perform layer normalization on the fused feature vector to output the corrected semantic feature vector.

[0045] In the embodiments of the present invention, by means of a pre-trained NLP model and a Transformer encoder, general semantic features can be extracted, resource consumption and training time can be reduced, laying a foundation for processing; a domain weight matrix is generated by calculating the cosine similarity with a thesaurus, accurately adapting to the professional semantic requirements of different domains; through matrix operations, residual connections, and layer normalization, while preserving the original semantics, the domain feature expression is enhanced; high-quality input is provided for downstream NLP tasks, improving task metrics and the practicality of the model; without a large amount of labeled data, domain adaptation can be completed only relying on the thesaurus, reducing costs, shortening the deployment cycle, and enhancing the generality and scalability of the model.

[0046] In the embodiments of the present invention, when specifically applied, it can be implemented through the following technical solutions, for example: In step S31 above, the pre-trained model is usually constructed based on multiple layers of Transformer encoders. Each encoder layer contains two core components: The self-attention mechanism allows the model to, when processing each word, pay attention to the semantic information of all other words in the sentence, thereby capturing long-distance dependencies (for example, "apple" represents a fruit in "eat an apple" and a brand in "Apple Inc."). The feedforward neural network performs a non-linear transformation on the features output by self-attention to enhance the semantic representation ability.

[0047] In addition, each encoder layer also contains a residual connection and layer normalization, which are used to stabilize the training process and prevent gradient disappearance.

[0048] Convert words into dense vectors, which contain the basic semantics of the words (such as the vector of "apple" is different from that of "banana"), add the position information of the words in the sentence, solve the problem that the Transformer itself does not have sequence order perception (such as distinguishing "I eat an apple" and "An apple eats me"), and is used to process sentence pair inputs (such as in a question-and-answer scenario) to distinguish the semantic boundaries of different paragraphs.

[0049] Randomly mask some words in the sentence, and let the model predict the masked words according to the context, forcing the model to learn the dependencies between words; judge whether two sentences are consecutive paragraphs to enhance the model's understanding of the discourse semantics (this objective has been removed in some models such as RoBERTa).

[0050] The input word (such as "apple") is converted into a high-dimensional word vector (such as 768-dimensional) through a pre-trained embedding matrix. This vector contains the semantic commonalities of the word (such as words in the fruit category being close in the vector space) and uniqueness (the vector differences between different words); adding word embeddings, position embeddings, and segment embeddings to form the final input vector, ensuring that the model simultaneously obtains word semantics, position order, and paragraph structure information.

[0051] For each word, calculate its "attention weight" with all other words in the sentence. The higher the weight, the stronger the semantic association between the two. For example, in "I like to eat apples", the attention weight of "eat" and "apples" is higher than that of "I", so the model can capture the verb-object relationship of "eat apples"; split self-attention into multiple "heads", each head capturing semantic relationships from different sub-spaces (such as some heads focusing on syntactic structures and some heads focusing on entity relationships), and finally merge the outputs to enhance the diversity of features; the features output by self-attention undergo a non-linear transformation (such as the ReLU activation function) to further refine the abstract semantics (such as elevating the vector of "apple" from a specific physical concept to a more abstract semantic representation).

[0052] The outputs of multiple layers of Transformer encoders contain semantic information at different levels (more syntactic features at the lower levels and more semantic features at the higher levels). Fuse the outputs of all layers through weighted summation. For example, the output of the lower layer accounts for 30%, the middle layer accounts for 40%, and the upper layer accounts for 30% to form an original semantic feature vector containing rich context; the weights can be automatically learned through training or fixed average weights can be used to ensure that the final vector contains both basic syntactic information and high-level semantic understanding (such as the semantic differences of "apple" in different contexts will be captured by the higher-level network and reflected in the vector).

[0053] Through pre-training on large-scale unsupervised data, the model can learn the general laws of language (such as part-of-speech collocations, entity relationships) without having to retrain the underlying network for specific tasks; compared with traditional word vectors (such as Word2Vec), the word vectors of pre-trained models are "dynamic", and the same word will generate different vectors in different contexts (such as the vectors of "apple" in "eat apples" and "Apple Inc." are different), which is more in line with the semantic variability of natural language; as a "basic component for language understanding", the pre-trained model only needs to be fine-tuned with a small amount of domain data to adapt to specific tasks (such as sentiment analysis, question-answering systems), greatly reducing the training cost.

[0054] In step S32 above, for each dimension of the original semantic feature vector, calculate its cosine similarity with the domain term vectors in the term library (such as "machine learning", "neural network") to measure the semantic relevance between the feature and the domain terms; select the term vectors with similarity exceeding the threshold (such as 0.7) to form a set of relevant terms (such as "apple" may be associated with "iPhone", "iOS" in the technology field); normalize according to the similarity between the term vector and the original feature to obtain the weight coefficients (such as the similarity of "iPhone" is 0.8, and the similarity of "iOS" is 0.9, then the weights are 0.47 and 0.53 respectively); expand these weight coefficients into a diagonal matrix (such as 768×768), where the values on the diagonal are the weights of the corresponding dimensions, and the rest of the elements are 0.

[0055] In step S33 above, multiply the domain weight matrix by the original semantic feature vector to scale the eigenvalue of each dimension according to the domain relevance (such as enhancing the dimensions related to the technology field and keeping the general dimensions unchanged); add the scaled feature vector to the original feature vector to retain the original information to prevent overfitting of domain knowledge (such as the meaning of "apple" in the general context is not completely covered by the technology field weight); perform normalization processing on the fused feature vector to stabilize the feature distribution and output the corrected semantic feature vector.

[0056] In a preferred embodiment of the present invention, step S4 above: perform few-shot learning on the corrected semantic features, and enhance the recognition ability of low-frequency text patterns through a contrast learning mechanism to generate an enhanced feature vector resistant to long-tail interference, which may include: Step S41, extract the semantic feature vectors of high-frequency samples from the annotation library to form a high-frequency sample set, and at the same time extract the semantic feature vectors of low-frequency samples to form a low-frequency sample set; for each target sample in the low-frequency sample set, randomly select high-frequency samples of the same category as positive samples and high-frequency samples of different categories as negative samples; combine the target sample, positive sample, and negative sample into a triple form to generate a set of contrast learning sample pairs; Step S42, calculate the geometric center of all sample feature vectors of each category in the high-frequency sample set as the prototype vector of the category; for the target sample in the contrast learning sample pair, calculate its Euclidean distance from the prototype vector of the positive sample category and its Euclidean distance from the prototype vector of the negative sample category respectively; apply an inverse weight coefficient to the Euclidean distance according to the category distribution frequency of the low-frequency samples; Step S43, with the goal of minimizing the distance between the target sample and the positive sample prototype and maximizing the distance from the negative sample prototype at the same time; adjust the parameters of the feature encoding layer through gradient backpropagation to make the low-frequency samples gather towards the same-class prototypes in the feature space and form an interval boundary with different-class prototypes, and iterate and update until the distance loss function converges, and output an enhanced feature vector with the characteristics of anti-long-tail distribution.

[0057] In the embodiments of the present invention, high-frequency samples are used to provide semantic references for low-frequency samples, and a contrastive learning framework is utilized to force the model to learn the intra-class similarity and inter-class differences, alleviating the problem of fuzzy feature representation caused by insufficient labeled data for low-frequency samples; the prototype vector, as the central representation of class semantics, can effectively generalize the class commonalities; the inverse ratio weight mechanism enables the model to pay more attention to the feature differences of rare classes, avoiding the neglect of low-frequency classes due to the dominance of high-frequency classes in training; through the distance constraint in the geometric space, the model's recognition ability for low-frequency text patterns is significantly improved. Even in the long-tail distribution scenario, a reliable classification boundary can be established by comparing the features of high-frequency classes, reducing the misjudgment problem of rare classes.

[0058] In the embodiments of the present invention, when specifically applied, it can be implemented through the following technical solutions, for example: In step S41 above, according to the occurrence frequency of samples in the annotation library, the semantic feature vectors are divided into a high-frequency sample set (such as categories with an occurrence frequency ≥ 100 times) and a low-frequency sample set (such as categories with an occurrence frequency < 10 times).

[0059] For each target sample in the low-frequency sample set, randomly select 1 - 5 positive samples from the high-frequency samples of the same category and 5 - 10 negative samples from the high-frequency samples of other categories; combine each target sample with the corresponding positive and negative samples to form a "target-positive-negative" triple, for example: (low-frequency sample A, high-frequency sample B of the same category, high-frequency sample C of a different category), and finally form a set of contrastive learning sample pairs.

[0060] In step S42 above, for each category in the high-frequency sample set, calculate the geometric center (i.e., the mean vector) of all sample feature vectors of this category as the prototype vector of this category (such as the prototype of category K is the average feature of all high-frequency samples of this category); for each target sample, calculate the distance between it and the prototype of the positive sample category (such as the distance from target sample A to the prototype of category K) and the distance between it and the prototype of the negative sample category (such as the distance from target sample A to the prototype of category M) respectively.

[0061] According to the category distribution frequency of low-frequency samples (such as category K accounts for 1% in low-frequency samples and category M accounts for 5%), multiply the distance result by the inverse ratio weight (such as the weight of category K is 1 / 0.01 = 100 and category M is 1 / 0.05 = 20), so that the distance error of rare category samples obtains a greater weight.

[0062] In step S43 above, optimize two objectives simultaneously through the loss function: narrow the distance between the target sample and the prototype of the positive sample (to gather similar features of the same class); widen the distance between the target sample and the prototype of the negative sample (to separate dissimilar features of different classes).

[0063] Adjust the parameters of the feature encoding layer using backpropagation through gradients, such as the weights of the last few layers of a Transformer, to move the feature vectors of low-frequency samples towards the prototypes of the same class and away from the prototypes of different classes in the space; Iterative convergence: Continuously train until the distance loss function (such as Triplet Loss) no longer significantly decreases. At this time, clear class boundaries are formed by low-frequency samples in the feature space, and enhanced feature vectors are output.

[0064] In a preferred embodiment of the present invention, in the above step S5: Based on the enhanced feature vectors, perform multi-dimensional label extraction operations synchronously, and output a label set including the following four types of labels: demand type label, sentiment tendency label, product or service entity label, problem keyword label, which may include: Step S51, input the enhanced feature vectors output in step S4 into the shared feature layer of the multi-task classifier, and perform feature dimension compression through a fully connected neural network to generate a reduced-dimensional feature representation shared by four tasks; Step S52, input the reduced-dimensional feature representation into the first sub-classifier to calculate the probability distribution of the text belonging to each demand category; map the probability values to the [0, 1] interval through normalization operations, and select the category corresponding to the maximum probability value as the demand type label; input the same reduced-dimensional feature representation into the second sub-classifier to calculate the sentiment tendency boundary value: construct positive and negative sentiment decision planes, and calculate the geometric distance from the feature representation to the decision plane; for each word position feature slice in the enhanced feature vector, calculate the probability of entity markers, and solve the globally optimal marker sequence through the Viterbi algorithm, and merge consecutive markers to form entity labels; calculate the cosine similarity between the feature vector of each word and the full-text feature vector, fuse the inverse document frequency weights of the words in the full text, and select the top N words ranked in descending order of the comprehensive score as the problem keyword labels; Step S53, establish a label storage structure, create a label record containing four fields for each text; write the demand type label into the "category" field, the sentiment tendency label into the "sentiment" field; write the product or service entity label into the "entity" field in the form of a list; write the problem keyword labels into the "keyword" field in the form of a list in descending order of weights, and output a label set including four types of structured labels.

[0065] In the embodiments of the present invention, the enhanced feature vectors are dimensionally reduced through a shared feature layer, avoiding repeated feature extraction for different label tasks, reducing the consumption of computing resources, while retaining the core semantics of the text, and achieving efficient parallel extraction of multi-dimensional labels; the extracted demand type labels accurately locate the text service attributes based on probability classification, providing a basis for work order allocation, etc., the sentiment tendency labels quickly judge positive and negative sentiments through geometric distance calculation to assist in public opinion analysis, the entity labels use sequence labeling and the Viterbi algorithm to identify key objects to clarify the core entities, and the keyword labels combine semantic relevance and inverse document frequency weights to refine the core of the text theme; the standardized four-field label record supports efficient storage and query in the database, facilitating integration into downstream scenarios such as the customer service system and the data analysis platform, enabling automatic routing of work orders, assisting in business decision-making, and reducing the labeling cost of machine learning models; combined with the enhanced feature vectors, the recognition of low-frequency demand types or rare entities improves the accuracy rate, reducing misjudgments caused by long-tail data; feature dimensional reduction and multi-task collaboration optimize the label quality, improving the accuracy rate of demand type label classification, increasing the speed of sentiment judgment, and the keyword extraction can better reflect the text theme compared with simple word frequency statistics.

[0066] In the embodiments of the present invention, when specifically applied, it can be implemented through the following technical solutions, for example: In the above step S51, the enhanced feature vectors (such as 768-dimensional) are input into the shared feature layer of the multi-task classifier, and this layer is composed of a fully connected neural network (such as including 128 neurons); through the linear transformation and activation function (such as ReLU) of the fully connected layer, the high-dimensional features are compressed into low-dimensional shared features (such as from 768 dimensions to 128 dimensions), and this process retains the core semantic commonalities of the text; the dimensionally reduced feature representations are simultaneously output to four sub-classifiers (demand type, sentiment tendency, entity recognition, keyword extraction) to achieve feature sharing and multi-task parallel processing.

[0067] In the above step S52, the dimensionally reduced features are input into the first sub-classifier (such as a Softmax classifier), and the probabilities of the text belonging to each demand category (such as "consultation", "complaint", "suggestion") are calculated; after normalizing the probability values, the category corresponding to the maximum probability is selected as the final label (such as the probability of "consultation" being 0.7, "complaint" being 0.2, then the label is "consultation").

[0068] The features are input into the second sub-classifier, and the sentiment decision plane parameters (such as weight vectors and biases) are generated through the fully connected layer; the geometric distance from the feature vector to the decision plane is calculated: a positive distance is determined as positive sentiment, and a negative distance is determined as negative sentiment (such as a distance > 0.5 is "positive", < -0.5 is "negative").

[0069] For each word position in the enhanced feature vector, calculate the entity tagging probabilities (such as "B-Product", "I-Product", "O"); use the Viterbi algorithm to solve for the globally optimal tagging sequence, and merge consecutive tags to form entities (such as the tag "E-Mobile Phone" marked as "B-Product I-Product", merged into the entity "E-Mobile Phone").

[0070] Calculate the cosine similarity between each word feature vector and the full-text feature vector to measure the relevance of the word to the text theme; combine the inverse document frequency (IDF) weight of the word (lower-frequency words have higher weights) to calculate the comprehensive score (such as similarity × IDF); select the top N words (such as N = 5) in descending order of score as keywords (such as "Price", "Service", "Refund").

[0071] In the above step S53, generate a structured record containing four fields for each text; field filling: "Category" field: write the requirement type label (such as "Consultation"); "Sentiment" field: write the sentiment tendency label (such as "Positive", "Negative"); "Entity" field: store the identified entities in a list form (such as ["E-Mobile Phone", "After-sales Service"]); "Keyword" field: store the keyword list sorted in descending order of weight (such as [{"Price": 0.85}, {"Refund": 0.72}]); label set output: summarize the label records of all texts into a structured label set to support batch query and analysis.

[0072] In a preferred embodiment of the present invention, in the above step S6: for each label in the label set, locate its key decision word in the enhanced feature vector through the attention mechanism, generate a natural language description of the basis for label determination, and form an enhanced label set with explanatory text, which may include: Step S61, for each label in the label set, trace back to the sub-classifier decision layer that generated the label in step S5, and extract the attention weight matrix of the decision layer for the enhanced feature vector output in step S4; Step S62, traverse the weight vector corresponding to the current label in the attention weight matrix, identify the weight peak positions where the weight values exceed the preset peak threshold, and map the peak positions back to the word indices in the structured text data generated in step S2; Step S63, extract the original words from the structured text data according to the word indices, generate a key decision word sequence sorted in descending order of weight values, and merge them into semantic phrases when adjacent word indices are consecutive; Step S64, according to the key decision word sequence, fill the requirement type label, sentiment label, and entity label into the corresponding templates; Step S65: Append the generated explanatory text to the original label to form an enhanced label tuple, and all label tuples constitute an enhanced label set with explanatory text.

[0073] In the embodiments of the present invention, by means of the attention mechanism, the weight matrix corresponding to the label is traced back to locate the key decision-making words in the text that affect the label determination, improving the interpretability of the model decision-making; mapping the weight peaks to the original text word indices to directly associate the label with the text expression, enhancing semantic traceability; generating natural language explanations by filling in the key decision-making words with templates to reduce the understanding cost of business personnel; using the sequence of key decision-making words to assist in detecting misjudgments of the model for facilitating model optimization; in business scenarios such as customer service and medical care, reducing manual review and enhancing the credibility of the system; the templated generation method ensures the standardization and unity of label explanations, facilitating comparative analysis; the natural language explanations promote human-machine collaboration, enabling non-technical personnel to quickly understand the meaning of the label.

[0074] In the embodiments of the present invention, when specifically applied, it can be implemented through the following technical solutions, for example: In step S61 above, determine the sub-classifier corresponding to the current label (for example, the demand type label corresponds to the first sub-classifier, and the sentiment label corresponds to the second sub-classifier); enter the last decision layer of this sub-classifier (such as the fully connected layer or the Softmax layer), and extract the attention weight matrix of each dimension of the enhanced feature vector when generating the current label at this layer. This matrix records the degree of attention of the model decision-making to each feature dimension; the rows of the weight matrix correspond to the dimensions of the enhanced feature vector, and the columns correspond to the label categories. Extract the column vector corresponding to the current label as the weight vector of this label.

[0075] In step S62 above, traverse the weight vector of the current label one by one, and identify the positions where the weight values are higher than the preset threshold (such as 0.7). These positions are called "weight peaks", representing the feature dimensions that the model focuses on during decision-making.

[0076] Since each dimension of the enhanced feature vector corresponds to the feature of a certain word in the structured text (for example, the i-th dimension corresponds to the word wi), map the index of the feature dimension corresponding to the weight peak to the word index in the structured text; filter out the invalid indices that exceed the text length to ensure mapping to the actual existing word positions.

[0077] In step S63 above, according to the mapped word indices, extract the corresponding original words from the structured text data (for example, index 0 corresponds to "I", index 1 corresponds to "purchase"); sort the extracted words in descending order of the weight values to form a sequence of key decision-making words (for example, "fault" with a weight of 0.9 is ranked before "service" with a weight of 0.7); check whether the sorted word indices are continuous (for example, indices 3, 4, 5 correspond to "battery", "battery life", "short"). If they are continuous, merge them into a semantic phrase ("short battery life") to improve the readability of the explanatory text.

[0078] In the above step S64, it is determined whether the current label belongs to the demand type, sentiment tendency or entity label; the demand type label template: "due to the appearance of the keyword [key decision word sequence], it is determined to be [demand type]" (such as "due to the appearance of 'refund' and 'quality problem', it is determined to be 'complaint'"); the sentiment tendency label template: "based on the weight comparison of the keywords [positive words] and [negative words], it is determined to be [sentiment tendency]" (such as "based on the weight of 'satisfaction' 0.8 and the weight of 'disappointment' 0.2, it is determined to be 'positive'"); the entity label template: "by identifying the continuous keyword [entity phrase], it is determined to be [entity type]" (such as "through 'EMate60', it is determined to be 'product entity'"); keyword filling: the sorted key decision word sequence is filled in the corresponding position according to the template rules to generate a natural language explanation text.

[0079] In step S65, the original tag and the generated explanation text are combined into a tuple (e.g., ("complaint", "due to the occurrence of 'refund' 'quality issues', it is determined to be 'complaint'"). Each tuple contains a tag field and an explanation field, forming a structured record. All tag tuples are aggregated into an enhanced tag set, supporting retrieval and analysis by tag type or explanation text.

[0080] In a preferred embodiment of the present invention, the above step S7: triggering incremental training of the pre-trained NLP model based on the distribution of low-frequency tags and the frequency of newly added domain terms in the enhanced tag set may include: Step S71: traverse all tags in the enhanced tag set, count the occurrence frequency of each demand type tag, and mark the tags with a cumulative frequency lower than 0.1% of the total sample size as long-tail tags; Step S72: monitor the number of newly added records in the domain terminology database in real time, and calculate the ratio of the number of newly added terms in the past 30 days to the total number of terminology database as the monthly update rate; Step S73: Incremental training is triggered when the following two conditions are met simultaneously: Condition 1: The proportion of long-tail tags to all demand type tags is greater than 15%; Condition 2: Monthly terminology database update rate > 5%; Step S74: extract all original text data with long-tail tags in step S1 as first-category samples; construct pseudo-annotated samples from the newly added terms and their context fragments as second-category samples; In step S75, the first and second category samples are input into the pre-trained NLP model for incremental training, the underlying parameters of the pre-trained NLP model are frozen, the weight matrix of the domain adaptation layer in step S3 and the fully connected layer parameters of the multi-task classifier in step S5 are updated, and the training is terminated when the label recall rate of the pre-trained NLP model on the validation set is improved by less than 1%.

[0081] In the embodiments of the present invention, by monitoring the update rate of low-frequency tags and domain terms, incremental training is automatically triggered, enabling the model to continuously adapt to emerging business scenarios and avoiding performance degradation caused by data distribution shift. Training is triggered only when the proportion of long-tail tags exceeds 15% and the term update rate exceeds 5%, balancing the model update cost and benefit, avoiding resource waste caused by frequent training, and ensuring that the model is optimized at key nodes. Incremental training is performed on the original text corresponding to low-frequency tags to improve the model's recognition ability for rare demand types. The newly added terms and their contexts are constructed into pseudo-labeled samples, enabling the model to quickly learn new industry vocabulary, shortening the cycle from the emergence of business terms to model adaptation, and maintaining technological advancement. The underlying parameters of the pre-trained model are frozen, and only the parameters of the domain adaptation layer and multi-task classifier are updated, reducing training time and avoiding catastrophic forgetting, ensuring that the model retains the learned knowledge while adapting to new data. The termination condition is that the improvement rate of the label recall rate in the validation set is <1%, ensuring that the training is sufficient but not excessive and preventing overfitting.

[0082] In the embodiments of the present invention, when specifically applied, it can be implemented through the following technical solutions, for example: In step S71 above, traverse all demand type tags (such as "consultation" and "complaint") in the enhanced label set, and use a counter to record the occurrence times of each tag. For example, assume there are a total of 100,000 pieces of text, among which "cross-border payment consultation" appears 50 times and "metaverse product complaint" appears 8 times; calculate 0.1% of the total sample size as the long-tail tag threshold. If the total sample size is 100,000, the threshold is 100,000 × 0.1% = 100 times; mark the tags with occurrence frequencies lower than the threshold as long-tail tags. For example, "metaverse product complaint" appears 8 times < 100 times and is marked as a long-tail tag; "cross-border payment consultation" appears 50 times < 100 times and is also marked as a long-tail tag.

[0083] In step S72 above, monitor the domain term library (such as the financial domain term library) in real time, and record the newly added term entries in the past 30 days. For example, the term library originally had 1,000 terms, and 60 new terms (such as "blockchain smart contract" and "AI large model risk control") were added in the past 30 days; divide the number of newly added terms by the total amount of the term library to obtain the monthly update rate. The formula is: `Monthly update rate = Number of newly added terms / Total amount of term library × 100%`. For example, 60 / 1,000 = 6%, and the monthly update rate is 6%.

[0084] In step S73 above, count the number of long-tail tags among all demand type tags, and divide it by the total number of tag types to obtain the proportion of long-tail tags. For example, there are 200 total demand tags, among which 35 are long-tail tags, and the proportion is 35 / 200 = 17.5%.

[0085] Check whether the long-tail label ratio > 15% (e.g., 17.5% > 15%, satisfying condition 1); check whether the monthly update rate of the term library > 5% (e.g., 6% > 5%, satisfying condition 2); when both conditions are met, trigger incremental training.

[0086] In the above step S74, trace back to the original text data in step S1, and filter out all texts with long-tail labels. For example, the original user feedback text with the label "Metaverse product complaint"; remove the noise in the text (such as special symbols, garbled codes), and retain the structured word sequence (such as "The purchased Metaverse product cannot be logged in, and the customer service does not handle it").

[0087] For each newly added term in the past 30 days (such as "blockchain smart contract"), extract the context fragment containing the term from the historical text data (such as the contract clause text containing "blockchain smart contract"); mark the newly added term as the corresponding entity type (such as "blockchain smart contract" is labeled as "Technical term - Blockchain"), and construct a pseudo-labeled sample (such as "[The execution process of blockchain smart contract] needs to comply with regulations" → labeled as the entity "blockchain smart contract").

[0088] In the above step S75, read the storage file of the pre-trained model (such as `pytorch_model.bin` of HuggingFace, `checkpoint` file of TensorFlow), parse the file structure to obtain the model parameters and configuration information; read the model configuration file (such as `config.json`), obtain the model architecture parameters (such as the number of Transformer layers, hidden layer dimension, number of attention heads, etc.), and ensure that the loaded model structure is consistent with that during training; extract the weight matrix and bias terms from the binary parameter file (such as the embedding layer weight, weight parameters of each layer of the Transformer encoder), and map them to the corresponding network layers according to the model architecture hierarchy.

[0089] Reconstruct the model architecture according to the configuration information. For example, create components such as multiple layers of Transformer encoders, embedding layers, domain adaptation layers, and multi-task classifiers; create encoders with the specified number of layers (such as 12 layers) according to the configuration, and each layer contains a self-attention module and a feed-forward neural network; instantiate the domain weight matrix storage structure in step S3 (such as a diagonal matrix); construct a fully connected network structure of a shared feature layer and four sub-classifiers.

[0090] Map the parsed pre-trained parameters to the reconstructed model architecture hierarchically. For example: load the embedding layer weight matrix to the model input layer; load the weight parameters of each layer of the Transformer encoder to the corresponding self-attention module and feed-forward network; if there are pre-trained values for the initial parameters of the domain adaptation layer and multi-task classifier, load them, otherwise initialize them randomly.

[0091] Locate the underlying parameters of the Transformer encoder (such as the first 10 layers), and prohibit gradient updates by setting `requires_grad=False`. For example, in PyTorch, use `for param in model.transformer.encoder.layers[:10].parameters(): param.requires_grad = False`; the freezing logic is based on the assumption that "the underlying network learns general language rules, and the upper layers learn domain-specific knowledge" to ensure that the model retains the basic semantic understanding ability during incremental training.

[0092] Set `requires_grad=True` to allow gradient updates for the weight matrix of the domain adaptation layer (such as the domain weight matrix in step S3) and the parameters of the fully connected layer of the multi-task classifier. For example, open the update permissions of `model.domain_adapter.weight` and `model.classifier.fc.parameters()`.

[0093] Load the model parameters to the corresponding computing device according to the hardware resources (CPU / GPU). For example, use `model.to(device)` to transfer the model to the GPU for accelerated training; if the hardware supports it (such as the TensorCore of NVIDIA GPU), enable the mixed-precision training configuration (such as `torch.cuda.amp` in PyTorch) when loading the model to reduce memory occupancy and accelerate calculations; if the sample size is large, configure the distributed training strategy (such as DataParallel or DistributedDataParallel) when loading the model to achieve multi-GPU parallel computing.

[0094] Check whether the dimensions of the loaded parameters match the model architecture (such as the embedding layer weight dimension being [vocabulary size, 768]) to avoid dimension errors caused by version incompatibility; perform forward propagation using a small number of samples (such as 10 texts) to verify whether the model output meets the expectations (such as the probability distribution range of the demand type label being between [0, 1]) to ensure that the loaded model functions properly; verify the semantic representation ability of the model through word vector similarity tests to ensure that the pre-trained knowledge of the underlying Transformer encoder is not lost.

[0095] Calculate the label recall rate (such as the recall rate of long-tail labels) on the validation set after each round of training; when the recall rate improvement rate for 5 consecutive rounds of training < 1% (such as from 70% to 70.5%, the improvement rate 0.5% < 1%), terminate the training.

[0096] Such as Figure 2As shown in the figure, an embodiment of the present invention further provides a text data automatic annotation system integrating a pre-trained NLP model, including: An acquisition module, configured to acquire the original text data to be annotated, where the text data includes conversation records, evaluations, and work orders of customer consultations, complaints, or feedback; A preprocessing module, configured to perform word segmentation, stop word filtering, and part-of-speech tagging on the original text data to generate structured text data; A correction module, configured to input the structured text data into the pre-trained NLP model, and correct the semantic features output by the model through a domain adaptation layer to obtain corrected semantic features; A feature vector module, configured to perform few-shot learning on the corrected semantic features, strengthen the recognition ability of low-frequency text patterns through a contrast learning mechanism, and generate enhanced feature vectors resistant to long-tail interference; A label module, configured to synchronously perform multi-dimensional label extraction operations based on the enhanced feature vectors, and output a label set including the following four types of labels: demand type labels, sentiment tendency labels, product or service entity labels, and problem keyword labels; An interpretive module, configured to, for each label in the label set, locate the key decision words in the enhanced feature vectors through an attention mechanism, generate a natural language description of the basis for label determination, and form an enhanced label set with interpretive text; An incremental module, configured to trigger incremental training of the pre-trained NLP model based on the low-frequency label distribution in the enhanced label set and the new frequency of domain terms.

[0097] It should be noted that this system corresponds to the above method. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0098] An embodiment of the present invention further provides a computing device, including: a processor and a memory storing a computer program. When the computer program is run by the processor, it executes the method as described above. All implementation manners in the above method embodiments are applicable to this embodiment and can achieve the same technical effects.

[0099] The above is the preferred implementation manner of the present invention. It should be pointed out that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and refinements can be made, and these improvements and refinements should also be regarded as the protection scope of the present invention.

Claims

1. An automated text data annotation method integrating a pre-trained NLP model, characterized in that, The method includes: Step S1: Obtain the original text data to be annotated, where the text data includes conversation records, evaluations, and work orders of customer consultations, complaints, or feedback; Step S2: Perform word segmentation, stop word filtering, and part-of-speech tagging on the original text data to generate structured text data; Step S3: Input the structured text data into a pre-trained NLP model, and correct the semantic features output by the model through a domain adaptation layer to obtain corrected semantic features; Step S4: Perform few-shot learning on the corrected semantic features, and enhance the recognition ability of low-frequency text patterns through a contrastive learning mechanism to generate enhanced feature vectors resistant to long-tail interference; Step S5: Based on the enhanced feature vectors, synchronously perform multi-dimensional label extraction operations, and output a label set including the following four types of labels: demand type labels, sentiment tendency labels, product or service entity labels, and problem keyword labels; Step S6: For each label in the label set, locate its key decision-making words in the enhanced feature vectors through an attention mechanism, generate a natural language description of the basis for label determination, and form an enhanced label set with explanatory text; Step S7: Trigger incremental training of the pre-trained NLP model based on the low-frequency label distribution and the new frequency of domain terms in the enhanced label set.

2. The automated text data annotation method for the integrated pre-trained NLP model according to claim 1, wherein Step S2: Perform word segmentation, stop word filtering, and part-of-speech tagging on the original text data to generate structured text data, including: Scan and process each character encoding of the original text data to generate purified text; According to the purified text, calculate the probability distribution of each character as the start, middle, or end position of a word at the current position, and combine it with the conditional random field state transition score to generate a global maximum probability segmentation path through dynamic programming, and output a basic word segmentation sequence with clear word boundaries; According to the stop word dictionary stored in the hash table, traverse each word in the basic word segmentation sequence. If the word hits the stop word hash table and is not marked as a protected entity by the named entity recognition module, remove the word; if the word belongs to the entity protection list, retain it in the new sequence; generate a filtered word sequence carrying core semantic units; For each target word in the filtered word sequence, extract the context window of the 2 words before and after it and combine them into a feature template; query the pre-trained part-of-speech probability matrix to calculate the joint probability of the target word belonging to each part-of-speech category in its context environment; construct a part-of-speech state transition grid based on dynamic programming, iteratively calculate the cumulative probability of all part-of-speech tagging paths from the first word to the last word in the sequence, and select the path with the maximum probability as the final part-of-speech sequence; combine each word with its optimal part-of-speech label into a binary group to generate structured text data with grammatical markings.

3. The automated text data annotation method for the integrated pre-trained NLP model according to claim 2, characterized in that, Step S3: Input the structured text data into a pre-trained NLP model, and correct the semantic features output by the model through a domain adaptation layer to obtain corrected semantic features, including: Input the word sequence in the structured text data generated in step S2 into the embedding layer of the pre-trained NLP model, convert each word into a high-dimensional word vector through word vector mapping operation, and perform context feature extraction through multiple layers of Transformer encoders to output the original semantic feature vector; Input the original semantic feature vector into the domain adaptation layer. For each feature dimension of the original semantic feature vector, calculate its cosine similarity with all term vectors in the term library; select the set of term vectors whose similarity exceeds the preset threshold, and perform normalized weighted summation based on the similarity values between each term vector in the set and the original feature to generate the domain weight vector; expand the domain weight vector into a diagonal matrix form according to the feature dimension to form the domain weight matrix; Perform matrix multiplication on the domain weight matrix and the original semantic feature vector to scale the original feature according to domain relevance in each dimension; perform residual connection on the scaled feature vector and the original semantic feature vector, and fuse the original feature and the domain-enhanced feature through element-wise addition operation to obtain the fused feature vector; perform layer normalization on the fused feature vector to output the corrected semantic feature vector.

4. The method for automatically annotating text data of an integrated pre-trained NLP model according to claim 3, wherein Step S4: Perform few-shot learning on the corrected semantic features, and enhance the recognition ability of low-frequency text patterns through the contrast learning mechanism to generate an enhanced feature vector resistant to long-tail interference, including: Extract the semantic feature vectors of high-frequency samples from the annotation library to form a high-frequency sample set, and at the same time extract the semantic feature vectors of low-frequency samples to form a low-frequency sample set; for each target sample in the low-frequency sample set, randomly select high-frequency samples of the same category as positive samples and high-frequency samples of different categories as negative samples; combine the target sample, positive sample and negative sample into a triple form to generate a set of contrast learning sample pairs; Calculate the geometric center of all sample feature vectors of each category in the high-frequency sample set as the prototype vector of the category; for the target sample in the contrast learning sample pair, calculate its Euclidean distance from the prototype vector of the positive sample category and the Euclidean distance from the prototype vector of the negative sample category respectively; according to the category distribution frequency of the low-frequency sample, apply an inverse weight coefficient to the Euclidean distance; With the goal of minimizing the distance between the target sample and the positive sample prototype and maximizing the distance from the negative sample prototype at the same time; adjust the parameters of the feature encoding layer through gradient backpropagation, so that the low-frequency samples gather towards the same-class prototype in the feature space and form an interval boundary with the different-class prototype, and iterate and update until the distance loss function converges, and output an enhanced feature vector with anti-long-tail distribution characteristics.

5. The method for automatically annotating text data of an integrated pre-trained NLP model according to claim 4, wherein Step S5: Based on the enhanced feature vector, synchronously perform multi-dimensional label extraction operations to output a label set containing the following four types of labels: requirement type label, sentiment tendency label, product or service entity label, problem keyword label, including: Input the enhanced feature vector output in step S4 into the shared feature layer of the multi-task classifier, and perform feature dimension compression through a fully connected neural network to generate a reduced-dimensional feature representation shared by four tasks; Input the dimensionality-reduced feature representation into the first sub-classifier to calculate the probability distribution of the text belonging to each demand category; map the probability values to the interval [0, 1] through normalization operation, and select the category corresponding to the maximum probability value as the demand type label; input the same dimensionality-reduced feature representation into the second sub-classifier to calculate the sentiment tendency boundary value: construct the positive and negative sentiment decision planes, and calculate the geometric distance from the feature representation to the decision plane; for each feature slice at the word position in the enhanced feature vector, calculate the probability of entity marking, and solve the globally optimal marking sequence through the Viterbi algorithm, and merge the continuous markings to form entity labels; calculate the cosine similarity between the feature vector of each word and the full-text feature vector, fuse the inverse document frequency weight of the word in the full text, and select the top N words with the highest comprehensive scores in descending order as the problem keyword labels; Establish a label storage structure, and create a label record containing four fields for each text; write the demand type label into the "category" field, and write the sentiment tendency label into the "sentiment" field; write the product or service entity label into the "entity" field in the form of a list; write the problem keyword labels into the "keyword" field in the form of a list sorted by weight in descending order, and output a label set containing four types of structured labels.

6. The method for automatically annotating text data of an integrated pre-trained NLP model according to claim 5, wherein Step S6: For each label in the label set, locate the key decision word in the enhanced feature vector through the attention mechanism, generate a natural language description of the basis for label determination, and form an enhanced label set with explanatory text, including: For each label in the label set, trace back to the sub-classifier decision layer that generated the label in step S5, and extract the attention weight matrix of the decision layer for the enhanced feature vector output in step S4; Traverse the weight vector corresponding to the current label in the attention weight matrix, identify the weight peak positions where the weight values exceed the preset peak threshold, and map the peak positions back to the word indices in the structured text data generated in step S2; Extract the original words from the structured text data according to the word indices, sort them in descending order of weight values to generate a key decision word sequence, and merge them into semantic phrases when adjacent word indices are continuous; According to the key decision word sequence, fill the demand type label, sentiment label, and entity label into the corresponding templates; Attach the generated explanatory text to the original label to form an enhanced label tuple, and all label tuples form an enhanced label set with explanatory text.

7. The method for automatically annotating text data of an integrated pre-trained NLP model according to claim 6, wherein Step S7: Trigger the incremental training of the pre-trained NLP model based on the low-frequency label distribution and the new frequency of domain terms in the enhanced label set, including: Traverse all labels in the enhanced label set, count the occurrence frequency of each demand type label, and mark the labels with a cumulative frequency lower than 0.1% of the total sample size as long-tail labels; Monitor the new records in the domain term library in real time, and calculate the proportion of the number of new terms in the past 30 days to the total amount of the term library as the monthly update rate; Trigger incremental training when both of the following two conditions are met: Condition 1: The proportion of the number of long-tail labels to all demand type labels > 15%; Condition 2: The monthly update rate of the term library > 5%; Extract all the original text data carrying long-tail tags in step S1 as the first type of samples; construct the newly added terms and their context fragments into pseudo-labeled samples as the second type of samples; Input the first type of samples and the second type of samples into the pre-trained NLP model for incremental training. Freeze the underlying parameters of the pre-trained NLP model, update the weight matrix of the domain adaptation layer in step S3 and the fully connected layer parameters of the multi-task classifier in step S5, and terminate the training when the label recall rate improvement rate of the pre-trained NLP model on the validation set is <1%.

8. An automated text data annotation system integrating a pre-trained NLP model, the system implementing the method according to any one of claims 1 to 7, characterized in that, Including: An acquisition module for acquiring the original text data to be labeled, where the text data includes conversation records, evaluations, and work orders of customer consultations, complaints, or feedback; A preprocessing module for performing word segmentation, stop word filtering, and part-of-speech tagging on the original text data to generate structured text data; A correction module for inputting the structured text data into the pre-trained NLP model and correcting the semantic features output by the model through the domain adaptation layer to obtain corrected semantic features; A feature vector module for performing few-shot learning on the corrected semantic features, strengthening the recognition ability of low-frequency text patterns through a contrastive learning mechanism, and generating enhanced feature vectors resistant to long-tail interference; A label module for synchronously performing multi-dimensional label extraction operations based on the enhanced feature vectors and outputting a label set containing the following four types of labels: demand type labels, sentiment tendency labels, product or service entity labels, and problem keyword labels; An interpretive module for, for each label in the label set, locating its key decision words in the enhanced feature vectors through an attention mechanism, generating a natural language description of the basis for label determination, and forming an enhanced label set with interpretive text; An incremental module for triggering incremental training of the pre-trained NLP model based on the low-frequency label distribution in the enhanced label set and the frequency of newly added domain terms.

Images