A business customer service system based on large model sentiment recognition labeling and correction

Abstract

The application discloses a business customer service system based on large model sentiment recognition labeling and correction, and belongs to the technical field of natural language processing and deep learning. In view of the problems of the existing sentiment recognition system, such as dependence on large-scale artificial labeling, unstable label quality and poor performance of small samples, a multi-model collaborative labeling and iterative optimization mechanism is adopted, a fused label is generated through automatic selection of a basic model, small sample LoRA fine-tuning, double-model divergence detection and large model arbitration, and a refined training set is constructed for iterative fine-tuning, so that a closed-loop optimization system is formed. The application can effectively reduce the labeling cost, improve the label consistency and the sentiment recognition accuracy under complex semantics, and is suitable for business information, financial public opinion and intelligent customer service scenes.

Description

TECHNICAL FIELD

[0001] The application belongs to the technical field of natural language processing and deep learning, and particularly relates to a business customer service system based on large model sentiment recognition labeling and correction. BACKGROUND

[0002] Sentiment recognition is an important task in the field of natural language processing, which aims to automatically identify and classify the sentiment implied in the text, such as positive, negative or neutral. This technology plays a key role in intelligent customer service systems, social media analysis, customer feedback mining, financial information interpretation, and public opinion monitoring. Traditional sentiment recognition methods mainly rely on bag-of-words model, TF-IDF features or manually constructed sentiment dictionaries, combined with support vector machines, logistic regression, random forests and other traditional machine learning algorithms to build classification models. The effectiveness of this method depends largely on the quality of manual feature engineering and the injection of domain knowledge, which limits its generalization ability and makes it difficult to adapt to different fields or complex language expression scenarios.

[0003] With the rapid development of deep learning technology, sentiment recognition models based on neural networks, such as convolutional neural networks, recurrent neural networks, long short-term memory networks and Transformer architecture, have gradually become the mainstream technology route. In particular, the widespread use of pre-trained language models such as BERT and RoBERTa has significantly improved the performance of sentiment recognition tasks on multiple public datasets. However, these advanced deep learning models usually require large-scale, high-quality human-labeled data for supervised fine-tuning training to achieve ideal performance. In the face of complex scenarios in actual business applications, such as multi-label sentiment classification, semantic metaphor, subtle changes in tone and strong context dependence in financial news comments, corporate announcements, customer service conversations and other texts, relying solely on regular fine-tuning of pre-trained models faces serious challenges. First, obtaining large-scale high-quality sentiment-labeled data is costly, especially in professional fields such as finance, medicine and law, which require annotators with professional knowledge, further exacerbating the difficulty and cost of data acquisition. Second, sentiment labels are inherently subjective, and different annotators may have inconsistent judgments, resulting in noise and bias in the labeled data. This "label drift" problem directly affects the effectiveness and stability of model training. Finally, in small sample scenarios where annotation resources are scarce, the model has difficulty learning data features, resulting in a significant decline in its generalization ability, and the performance of the fine-tuned model is often unsatisfactory. Therefore, the existing technology has the core problems of excessive dependence on large-scale labeled data, difficulty in ensuring the quality of labeling, and poor model performance in low-resource conditions, which restricts the widespread application and performance improvement of sentiment recognition technology in high-demand business scenarios. SUMMARY

[0004] To solve the above technical problems, the present application proposes a business customer service system based on large model sentiment recognition labeling and correction to solve the problems existing in the prior art.

[0005] In the first aspect, to achieve the above object, the present application provides a business customer service method based on large model sentiment recognition labeling and correction, comprising the following steps:

[0006] Using a large language model to automatically screen and construct a commercial sentiment corpus, and dividing the data set;

[0007] Using a small amount of artificially labeled samples to independently fine-tune a plurality of selected base models;

[0008] Using the fine-tuned base model to batch predict unlabeled data to generate sentiment labels and reason explanations;

[0009] Comparing the prediction results of the base model, screening out disagreement samples with inconsistent labels;

[0010] Submitting the disagreement samples to a large model with stronger reasoning ability for auxiliary judgment to generate fusion labels;

[0011] Merging the artificial labeling samples and the fusion label samples to construct a refined training set;

[0012] Using the refined training set to iteratively fine-tune the base model and evaluate the model performance to form a closed-loop optimization mechanism.

[0013] Optionally, the process of constructing a commercial sentiment corpus comprises:

[0014] Parsing public sentiment analysis data sources, automatically screening financial news, corporate announcements or industry review texts;

[0015] Constructing a commercial sentiment corpus containing multiple samples, and performing data cleaning, format unification and structure standardization operations;

[0016] Divide the corpus into a no-label main training set, an artificial labeling fine-tuning set and a standard test set, and output a structured data file.

[0017] Optionally, the process of independently fine-tuning a plurality of selected base models comprises:

[0018] Extracting commercial texts with real sentiment labels from the original training set, and labeling them using a three-classification system;

[0019] Input the labeled data into the LLaMA-3.1-8B-it and Qwen-2.5-7B model framework;

[0020] Performing small sample fine-tuning efficiently using LoRA parameter injection technology, and saving the optimal model weight parameters.

[0021] Optionally, the process of screening out the divergent samples with inconsistent labels comprises:

[0022] Using the fine-tuned double model to batch predict the unlabeled data set, outputting the sentiment prediction label and generative reason explanation of each text;

[0023] Comparing the label results of the double model on each text, performing a consistency judgment function;

[0024] Labeling samples with inconsistent labels as conflict samples and including them in the conflict mediation pool.

[0025] Optionally, the process of generating a fusion label comprises:

[0026] Constructing a standardized prompt template for the divergent sample, which contains text content, double model labels and reason explanation;

[0027] Submitting the prompt template to the GPT-3.5 or DeepSeek model interface for semantic reasoning;

[0028] Receiving the comprehensive judgment result returned by the large model as the final label, and labeling it as a divergent mediation sample.

[0029] Optionally, the process of iterative fine-tuning comprises:

[0030] Merging the human-labeled samples and the fusion label samples to form a refined training set;

[0031] Using the refined training set to fine-tune the double model for a new round, and monitoring the training loss value, accuracy and label fitting rate;

[0032] Comparing the changes in model performance, if the preset gain threshold is met, save the model and proceed to the next round of training, otherwise trigger the sample selection optimization strategy.

[0033] In a second aspect, the present application also provides a business customer service system based on large model sentiment recognition labeling and correction, which is used to implement a business customer service method based on large model sentiment recognition labeling and correction, and the system comprises:

[0034] A data management module for automatically screening and constructing a business sentiment corpus, and dividing the data set;

[0035] A model training module for independently fine-tuning a plurality of selected base models using a small amount of human-labeled samples;

[0036] a prediction and disagreement detection module configured to use the fine-tuned base model to batch predict unlabeled data and compare the prediction results to filter out disagreement samples with inconsistent labels;

[0037] a label fusion module configured to submit the disagreement samples to a large model with stronger reasoning capability for auxiliary judgment and generate fusion labels;

[0038] an iterative optimization module configured to merge the artificial labeled samples and the fusion label samples to construct a refined training set, and iteratively fine-tune the base model and evaluate its performance to form a closed-loop optimization mechanism.

[0039] In a third aspect, the present application further provides a computer terminal device, comprising:

[0040] one or more processors;

[0041] a memory coupled to the processor, configured to store one or more programs;

[0042] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the business customer service method based on large model sentiment recognition labeling and correction in the first aspect described above.

[0043] In a fourth aspect, the present application further provides a computer readable storage medium having a computer program stored thereon, wherein the computer program is executed by a processor to implement the steps of the business customer service method based on large model sentiment recognition labeling and correction in the first aspect described above.

[0044] In a fifth aspect, the present application further provides a computer program product comprising a computer program, wherein the computer program is executed by a processor to implement the steps of the business customer service method based on large model sentiment recognition labeling and correction in the first aspect described above.

[0045] Compared with the prior art, the present application has the following advantages and technical effects:

[0046] The business customer service system based on large model sentiment recognition labeling and correction provided by the present application effectively reduces the artificial dependence of high-quality sentiment labeling data through multi-model collaborative labeling and large model arbitration mechanism. The difficult example samples are automatically identified by the disagreement between models, and the fusion labels are generated by the strong reasoning model, which significantly improves the label consistency and recognition accuracy in complex semantic scenarios. The system has automatic iterative optimization capability, can continuously expand high-quality training set on the basis of small sample, enhances the generalization ability of the model to implicit emotions and professional terms in business text, and is suitable for various high-precision sentiment recognition scenes such as finance, customer service and public opinion analysis. BRIEF DESCRIPTION OF DRAWINGS

[0047] The accompanying drawings, which form a part of this application, are included to provide a further understanding of the application and are incorporated in and constitute a part of this application. The drawings illustrate embodiments of the present application and, together with the description, serve to explain the application. In the drawings:

[0048] Figure 1 The overall flowchart of the embodiment of the present application. DETAILED DESCRIPTION

[0049] It should be noted that the embodiments in the present application and the features in the embodiments can be combined with each other without conflict. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0050] It should be noted that the steps shown in the flowchart of the drawings can be executed in a computer system such as a set of computer executable instructions, and although the logical order is shown in the flowchart, in some cases, the steps shown or described herein can be executed in an order different from that shown herein.

[0051] Embodiment one

[0052] As shown in the drawings, the present embodiment provides a business customer service method based on large model sentiment recognition and correction, which comprises: Figure 1

[0053] Using a large language model to automatically screen and construct a commercial sentiment corpus, and dividing the data set;

[0054] Using a small amount of artificially annotated samples to independently fine-tune a plurality of selected basic models;

[0055] Using the fine-tuned basic model to batch predict unannotated data to generate sentiment labels and reason explanations;

[0056] Comparing the prediction results of the basic models, screening out disagreement samples with inconsistent labels;

[0057] Submitting the disagreement samples to a large model with stronger reasoning ability for auxiliary judgment to generate fusion labels;

[0058] Merging the artificial annotation samples and the fusion label samples to construct a refined training set;

[0059] Using the refined training set to iteratively fine-tune the basic model, and evaluating the model performance to constitute a closed-loop optimization mechanism.

[0060] S1: Selection of public models under multi-model fusion architecture and construction of commercial sentiment data set;

[0061] S2: LoRA fine-tuning process based on small sample artificial annotation and initial model training;

[0062] ​S3: Large model label inference and cross-model sentiment judgment divergence detection mechanism;

[0063] S4: Label divergence explanation generation and fusion label generation method assisted by large model;

[0064] S5: Fusion label dataset construction and double model iterative precision optimization fine-tuning;

[0065] S6: Model evaluation based on test set and multi-round fine-tuning effect feedback closed loop;

[0066] As an embodiment in this embodiment, the process of constructing a commercial sentiment corpus includes:

[0067] Parse public sentiment analysis data sources, automatically filter financial news, corporate announcements or industry review texts;

[0068] Construct a commercial sentiment corpus containing multiple samples, and perform data cleaning, format unification and structure standardization operations;

[0069] Divide the corpus into an unlabeled main training set, a manually annotated fine-tuning set and a standard test set, and output a structured data file.

[0070] Further, step S1 specifically includes:

[0071] S1.1, automatically complete large model selection and combination matching using LLM: the system automatically performs basic model selection and evaluation process based on large language model, considers task adaptability, performance indicators and deployment cost, and optimizes large language model combination with strong sentiment recognition ability from multiple public models.

[0072] The system maps each text into a domain vector and calculates the adaptation score. The system maps each text into a domain vector and calculates the adaptation score. First, map each input text to a semantic vector , then calculate the similarity (such as cosine similarity) with the corresponding representation vector of the candidate model, and finally take the average of all similarities to get the overall matching score :

[0073]

[0074] Finally, the two base models LLaMA-3.1-8B-it and Qwen-2.5-7B with the highest scores are automatically selected as the core components of the collaborative recognition framework of the present application. The LLM dynamically evaluates the model capability according to the domain characteristics and semantic complexity of the text to be labeled, and at the same time, combines the model download volume, community activity and historical performance of platforms such as Hugging Face to evaluate the stability and universality of the model. Preferably, a model version with relatively moderate parameters and more convenient deployment is selected.

[0075] S1.2, Construction and Preset Segmentation of Commercial Sentiment Dataset Based on LLM: The system uses LLM to automatically analyze public sentiment analysis data sources, selects high-quality texts such as financial news, corporate announcements and industry reviews, and constructs a commercial sentiment corpus containing 10,000 samples. The whole process includes data cleaning, format unification and sample structure standardization. LLM synchronously completes data division: 9000 unlabeled texts are used for main training set, 400 artificially labeled samples are used for small sample fine-tuning, and 1000 are used for standard test set. A three-class label system (positive / neutral / negative) is used, and a structured data file is output.

[0076] As an embodiment in the present embodiment, the process of independently fine-tuning the selected plurality of base models comprises:

[0077] Commercial texts with real sentiment labels are extracted from the original training set, and a three-class system is used for labeling;

[0078] The labeled data is input into the LLaMA-3.1-8B-it and Qwen-2.5-7B model framework;

[0079] LoRA parameter efficient injection technology is used to perform small sample fast fine-tuning, and the optimal model weight parameter is saved.

[0080] Further, step S2 specifically comprises:

[0081] S2.1, Data preparation and format conversion based on artificially labeled samples: 400 commercial information texts with real sentiment labels are extracted from the original training set, and a standard three-class system (positive, negative, neutral) is used for unified labeling. LLM-driven format conversion tools automatically generate data input structures that meet the requirements of LoRA fine-tuning, including text content, sentiment labels, word segmentation index and token alignment, and other structured information.

[0082] S2.2, Double model independent fine-tuning and parameter storage management: input the training set generated by S2.1 into two model frameworks of LLaMA-3.1-8B-it and Qwen-2.5-7B respectively, perform small sample rapid fine-tuning by using LoRA parameter efficient injection technology, monitor the loss function convergence and model output stability during the process, automatically save the model weight parameter file with the best performance in training, and record the corresponding training configuration for subsequent reproduction and iterative update.

[0083] As an embodiment in the present embodiment, the process of screening out the divergent samples with inconsistent labels includes:

[0084] Using the fine-tuned double model to batch predict the unlabeled data set, output the sentiment prediction label and generative reason explanation of each text;

[0085] Comparing the label results of the double model on each text, a consistency judgment function is executed;

[0086] The samples with inconsistent labels are marked as conflict samples and included in the conflict mediation pool.

[0087] Further, step S3 specifically includes:

[0088] S3.1, Double model batch reasoning to generate sentiment labels and reason explanations: the system uses the fine-tuned two models and to batch predict the remaining unlabeled commercial text data set , call the model to output the corresponding sentiment prediction label and generative reason explanation (reason1, reason2) of each text, and store the reasoning results in a structured format for subsequent divergence analysis and label fusion.

[0089] S3.2, Label result comparison and divergence sample screening mechanism: the system automatically compares and analyzes the label results of the double model on each text, and each text sample outputs a pair of label-explanation pairs, and the system executes a consistency judgment function:

[0090]

[0091] When , the sample is marked as a high-confidence consistent sample; otherwise, it enters the conflict mediation pool for subsequent fusion label generation.

[0092] 1000 samples are selected as priority for subsequent processing; at the same time, the label pair, model reason and confidence information of each sample are recorded to construct a multi-model prediction result matrix, which provides a basis for label fusion and judgment mediation.

[0093] As an embodiment in this embodiment, the process of generating a fusion label includes:

[0094] A standardized prompt template containing text content, double-model labels and reason explanation is constructed for the divergent samples;

[0095] Submit the prompt template to the GPT-3.5 or DeepSeek model interface for semantic reasoning;

[0096] Receive the comprehensive judgment result returned by the large model as the final label, and mark it as a divergence mediation sample.

[0097] Further, step S4 specifically includes:

[0098] S4.1, direct confirmation and recording mechanism of consistent label samples: for samples that meet = in the emotion recognition result, the system automatically saves the consistent label as the final label , that is, ← = , and marks the sample as "high confidence consistent sample", which is used for subsequent stable expansion of the fine-tuning training set. The system automatically records the corresponding reasoning reasons reason1, reason2 and original model ID to ensure traceability and label reliability.

[0099] S4.2, LLM assisted judgment and label fusion mechanism of divergent label samples: for samples with divergence ( ≠ ), the system automatically constructs a standardized prompt template P(x) = [text, , reason1, , reason2], which is submitted to the GPT-3.5 or DeepSeek model interface; the large model performs semantic reasoning according to the input two sets of labels and their corresponding explanations, and returns the result after comprehensive judgment, which is used as the final label for fusion training. At the same time, mark the sample as "divergence mediation sample" to facilitate the verification of LLM contribution and label stability.

[0100] As an embodiment in this embodiment, the process of iterative fine-tuning includes:

[0101] Merge the manually annotated samples and fusion label samples to form a refined training set;

[0102] A new round of fine-tuning is performed on the dual model using the refined training set, and the training loss value, accuracy, and label fitting rate are monitored;

[0103] The performance change of the comparative model is compared, and if the preset gain threshold is met, the model is saved and the next round of training is promoted, otherwise the sample selection optimization strategy is triggered.

[0104] Further, step S5 specifically includes:

[0105] S5.1, fine training set construction and fusion label management mechanism: the system merges the 400 artificially annotated samples constructed in S2 stage and the 1000 high-quality label samples (including consistent samples and divergence mediation samples) generated in S4 stage to form a refined training set D1 required for a new round of fine-tuning, denoted as D1 = D0∪D3, where D0 is the original annotated data set and D3 is the fusion label set generated by LLM; the system automatically performs structure verification, label consistency check and distribution balance adjustment on D1 samples to ensure that the fine-tuning training set has emotion label diversity and semantic coverage.

[0106] S5.2, dual model iterative fine-tuning and training process monitoring mechanism: the updated training set D1 is input into LLaMA-3.1-8B-it and Qwen-2.5-7B respectively, and the same LoRA configuration is used for a new round of parameter fine-tuning. The training process records key indicators such as training loss value L(t), accuracy A(t), and label fitting rate F(t) in real time through an automatic monitoring module. The system compares the performance change of the model before and after ΔA = |A1- A0| to determine whether the performance gain of this round of fine-tuning meets the preset gain threshold ε; if the condition is met, the model parameters are saved and the next round of training is promoted, otherwise the sample selection optimization strategy is triggered to improve the training quality.

[0107] Further, step S6 specifically includes:

[0108] S6.1, multi-round iterative training mechanism and sample increment management strategy: after completing the fine-tuning in S5, the system automatically returns to the S3 stage to perform a new round of label generation, divergence detection and label fusion operation on the remaining unlabeled data, and merges the newly obtained high-confidence label samples into the training set D1 to form an expanded training set D2 = D1∪D_new, where D_new is the new label data obtained in the current iteration; by controlling the amount of new samples added in each round and the training accuracy gain ΔA≥ε, the system realizes the dynamic balance of fine-tuning efficiency and performance improvement until the unlabeled data set is completely traversed or the performance gain tends to be stable.

[0109] S6.2, test set evaluation index calculation and model performance comparison and analysis: after the model iteration is completed, the system loads the final fine-tuned LLaMA-3.1-8B-it and Qwen-2.5-7B, and performs sentiment label prediction on the reserved 1000 commercial sentiment test set D_test, outputs the predicted label label_pred and compares it with the standard label label_true, calculates the model accuracy Accuracy; at the same time, the performance difference of the two models on the same data is quantitatively compared to verify the performance improvement ability and stability of the method in the low-resource label scene.

[0110] With the development of financial technology and the rapid expansion of content ecology, commercial information texts (such as financial news, industry reviews, enterprise announcements, stock public opinion, etc.) produce a large amount of information every day, and their sentiment tendencies have a significant impact on financial decision-making, public opinion monitoring, and intelligent investment and research systems. However, such texts often contain implicit emotional expressions, professional terms, ambiguous rhetoric, and context reversal, which leads to a significant decrease in the accuracy of traditional sentiment recognition systems when faced with ambiguous semantics, polysemous expressions, or fine-grained emotions. At the same time, the acquisition of high-quality label data required for training high-performance sentiment recognition models is costly, especially in the financial field, where professional annotators are scarce, and label consistency is difficult to guarantee, which severely restricts the practical promotion of the model.

[0111] The present method proposes a commercial customer service system based on large model sentiment recognition annotation and correction, which effectively solves the above problems. The present invention starts with a small amount of manually annotated samples, fine-tunes multiple pre-trained language models (such as LLaMA-3.1-8B-it and Qwen-2.5-7B), automatically identifies difficult example samples using model divergence, generates more reliable labels using stronger reasoning ability of large models such as GPT, and incorporates the fused labels into the training set for iterative optimization. The method adopts a "generation-contrast-correction-update" cycle mechanism to continuously improve the model's sentiment recognition ability for complex commercial texts.

[0112] Compared with existing methods, the present method has the following advantages: (1) reduces the dependence on large-scale manual annotation data and reduces the training cost; (2) can dynamically identify and correct the prediction divergence between models, improve label consistency and accuracy; (3) improve the ability to handle difficult examples through large model auxiliary judgment mechanism, and enhance the model's ability to adapt to complex commercial texts; (4) the system has high automation capability, adapts to multiple public models and multiple domain data, and is suitable for financial sentiment analysis, enterprise public opinion judgment, market sentiment index construction, and other commercial application scenarios.

[0113] Reference Figure 1 The technical solution provided by the present invention is a commercial information sentiment information recognition method based on large model iterative correction and collaborative annotation, which includes the following implementation steps:

[0114] S1.1: The system first calls the LLM to recognize the dataset. Semantic feature analysis is performed to extract features such as industry terminology density, semantic ambiguity, and emotional expression intensity to identify that this type of text belongs to the "business information" scenario.

[0115] The system maps each text to a domain vector and calculates an adaptation score. The system first maps each input text to a domain vector and calculates an adaptation score. Mapping to semantic vectors Then, the representation vector corresponding to the candidate model. Calculate the similarity (e.g., cosine similarity), and finally average all similarities to obtain the overall matching score. :

[0116]

[0117] Finally, the two base models with the highest scores, LLaMA-3.1-8B-it and Qwen-2.5-7B, were automatically selected as the core components of the collaborative recognition framework of this invention.

[0118] S1.2: Construction and Pre-defined Segmentation of Business Sentiment Dataset Based on LLM: The system uses LLM to parse public sentiment analysis data sources, automatically filters high-quality texts from fields such as financial news, corporate announcements, and industry commentary, and constructs a business information sentiment analysis corpus containing 10,000 samples. It also automatically completes data cleaning, format unification, and sample structure standardization. The data segmentation strategy is automatically executed by LLM, with a preset output of 9,000 unlabeled texts as the main training set, 400 manually labeled texts for small-sample fine-tuning training, and 1,000 samples as the standard test set. A three-class labeling system (positive / neutral / negative) is uniformly adopted, and structured data files are output.

[0119] S2.1: The system extracts 400 manually labeled samples from the original commercial sentiment corpus as a seed training set, and calls the LoRA framework to fine-tune the LlaMA and Qwen models, generating two initial models L1 (LoRA-LlaMA) and L2 (LoRA-Qwen). LLM automatically writes training scripts and configures training parameters.

[0120] S2.2: LLM as a code validator for training scripts Static logic review is conducted to ensure the stability and compatibility of fine-tuning in three main aspects: the text preprocessing logic must support the segmentation of high-frequency word vectors in the industry, the retention of financial noun entities, and the parsing of nested brackets; verify whether the LoRA parameter insertion position matches the target model structure, the system executes the structure alignment function, and finally outputs the configuration consistency marker of the training script as a safety prerequisite for fine-tuning execution.

[0121] S3: The system uses two finely tuned models. and For the remaining unlabeled commercial text dataset Perform batch prediction for each text sample Output a label-interpretation pair, and the system executes a consistency check function:

[0122]

[0123] when At that time, the sample It is marked as a high-confidence consistent sample; otherwise, it enters the conflict resolution pool. This is used for subsequent fusion tag generation.

[0124] S4: The system constructs a standard input format P(x) = [text, ... reason1 [reason2], and submit it to GPT-3.5 or DeepSeek for judgment, calculate the confidence scores of the two explanations, select the one with higher confidence as the final label label3, and finally form the fused sample set D3.

[0125] S5: The system merges the original 400 artificial samples with the newly generated D3 samples to construct a refined training set. Then, L1 and L2 are input again for LoRA fine-tuning to generate the performance-optimized models L1' and L2'. During training, the system monitors the accuracy improvement rate ΔA. If ΔA > ε, the new model is saved and the next round of data labeling and fine-tuning begins; otherwise, the sample selection and optimization mechanism is triggered.

[0126] S6: The system uses the final models L1' and L2' for prediction evaluation of 1000 test set data, calculates performance indicators such as accuracy, and compares the changes in model performance before and after, forming the following trends: (1) In the Llama-3.1-8B-it model, our method can achieve an accuracy of 44.07% with only 400 labeled samples, which is significantly better than the 31.92% of Zero-Shot (no fine-tuning) and close to the performance of 45.76% of Full-Supervised (full fine-tuning). (2) In the Mistral-7B model, based on 400 labeled samples, our model performs at 40.18%, which is also better than the 28.37% of Zero-Shot. After further expanding to 1400 high-quality pseudo-labels, the accuracy increases to 50.15%, surpassing the 48.10% of the Full-Supervised model, verifying that the performance under the closed-loop mechanism can be continuously improved. If there is still significant room for performance improvement, the system will re-execute steps S3 to S5 to continuously expand the high-quality label set and improve the label quality through dual-model interactive verification, thereby forming a stable closed-loop iterative optimization mechanism until the accuracy convergence condition is met or the data is completely traversed.

[0127] Based on this, this invention provides a commercial customer service method based on large-model sentiment recognition annotation and correction. This invention effectively reduces the reliance on manual annotation of high-quality sentiment data through multi-model collaborative annotation and a large-model arbitration mechanism. It automatically identifies difficult examples by utilizing inter-model divergence and generates fused labels through a strong inference model, significantly improving label consistency and recognition accuracy in semantically complex scenarios. The system has automated iterative optimization capabilities, continuously expanding the high-quality training set based on small samples to enhance the model's generalization ability to implicit emotions and professional terminology in commercial texts. It is applicable to various high-precision sentiment recognition scenarios such as finance, customer service, and public opinion analysis.

[0128] Example 2

[0129] In this embodiment, a computer terminal device is provided, including:

[0130] One or more processors;

[0131] A memory, coupled to the processor, for storing one or more programs;

[0132] When the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the above-described commercial customer service method based on large model emotion recognition annotation and correction.

[0133] In this embodiment, a computer-readable storage medium is also provided, on which a computer program is stored. When the computer program is executed by a processor, it implements the steps of the above-described commercial customer service method based on large model emotion recognition, labeling, and correction.

[0134] In this embodiment, an electronic device is also provided, including a memory and a processor. The memory stores a computer program, and the processor is configured to run the computer program to perform the steps of the above-described commercial customer service method based on large model emotion recognition labeling and correction.

[0135] In this embodiment, a computer program product is also provided, including a computer program that, when executed by a processor, implements the steps of the above-described commercial customer service method based on large model emotion recognition annotation and correction.

[0136] The aforementioned program can run on a processor or be stored in memory (or a computer-readable medium). Computer-readable media includes both permanent and non-permanent, removable and non-removable media, and information storage can be achieved by any method or technology. Information can be computer-readable instructions, data structures, program modules, or other data. Examples of computer storage media include, but are not limited to, phase-change memory (PRAM), static random-access memory (SRAM), dynamic random-access memory (DRAM), other types of random-access memory (RAM), read-only memory (ROM), electrically erasable programmable read-only memory (EEPROM), flash memory or other memory technologies, CD-ROM, digital versatile optical disc (DVD) or other optical storage, magnetic tape, magnetic disk storage or other magnetic storage devices, or any other non-transfer medium that can be used to store information accessible by a computing device.

[0137] These computer programs may also be loaded onto a computer or other programmable data processing equipment to cause a series of operational steps to be performed on the computer or other programmable equipment to produce a computer-implemented process, thereby providing instructions that execute on the computer or other programmable equipment for implementing the process. Figure 1 One or more processes and / or boxes Figure 1 The steps for the functions specified in one or more boxes can be implemented by different modules for different steps.

[0138] This embodiment provides such a device or system. The system, referred to as a commercial customer service system based on large-model emotion recognition, annotation, and correction, includes:

[0139] The data management module is used to automatically filter and build a commercial sentiment corpus and to divide the dataset;

[0140] The model training module is used to independently fine-tune multiple selected base models using a small number of manually labeled samples;

[0141] The prediction and divergence detection module is used to perform batch predictions on unlabeled data using the fine-tuned base model and compare the prediction results to filter out divergent samples with inconsistent labels.

[0142] The label fusion module is used to submit the divergent samples to a large model with stronger reasoning capabilities for auxiliary judgment and generate fused labels.

[0143] The iterative optimization module is used to merge the manually labeled samples with the fused label samples to construct a refined training set, and to iteratively fine-tune and evaluate the performance of the basic model, thus forming a closed-loop optimization mechanism.

[0144] As one implementation method in this embodiment, the data management module includes:

[0145] The data acquisition unit is used to parse publicly available sentiment analysis data sources and automatically filter financial news, corporate announcements, or industry commentary texts.

[0146] The data preprocessing unit is used to perform data cleaning, format unification and structure standardization operations to build a commercial sentiment corpus;

[0147] The data partitioning unit is used to divide the corpus into an unlabeled main training set, a manually labeled fine-tuning set, and a standard test set, and outputs a structured data file.

[0148] As one implementation method in this embodiment, the model training module includes:

[0149] The labeled data preparation unit is used to extract commercial texts with real sentiment labels from the original training set and label them using a three-class classification system;

[0150] The fine-tuning execution unit is used to input labeled data into the LLaMA-3.1-8B-it and Qwen-2.5-7B model frameworks, and uses LoRA parameter efficient injection technology to perform fast fine-tuning of small samples;

[0151] The parameter management unit is used to save the model weight parameters that perform best during the fine-tuning process.

[0152] As one implementation method in this embodiment, the prediction and divergence detection module includes:

[0153] The batch prediction unit is used to perform batch predictions on the unlabeled dataset using a fine-tuned dual model, and outputs the sentiment prediction label and generative reasoning explanation for each text.

[0154] The consistency judgment unit is used to compare the label results of the two models on each text and execute the consistency judgment function.

[0155] The divergence sample screening unit is used to mark samples with inconsistent labels as conflict samples and include them in the conflict resolution pool.

[0156] As one implementation method in this embodiment, the tag fusion module includes:

[0157] The prompt template construction unit is used to construct a standardized prompt template for the divergent sample, which includes text content, dual-model labels and explanations.

[0158] The arbitration model interaction unit is used to submit the prompt template to the GPT-3.5 or DeepSeek model interface for semantic reasoning;

[0159] The fusion label generation unit is used to receive the comprehensive judgment result returned by the large model as the final label and mark it as a divergence mediation sample.

[0160] As one implementation method in this embodiment, the iterative optimization module includes:

[0161] The training set construction unit is used to merge manually labeled samples with fused label samples to form a refined training set.

[0162] The fine-tuning monitoring unit is used to perform a new round of fine-tuning on the dual models using the refined training set, and to monitor the training loss value, accuracy and label fitting rate.

[0163] The optimization control unit is used to compare changes in model performance. If the preset gain threshold is met, the model is saved and the next round of training is advanced; otherwise, the sample selection optimization strategy is triggered.

[0164] The system or apparatus is used to implement the functions of the methods in the above embodiments. Each module in the system or apparatus corresponds to each step in the method, as has been described in the method and will not be repeated here.

[0165] The above implementation method solves the problem of commercial customer service based on large model emotion recognition labeling and correction in related technologies, thereby ensuring that the problems existing in the prior art are resolved.

[0166] The above are merely preferred embodiments of the present invention, but the scope of protection of the present invention is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in the present invention should be included within the scope of protection of the present invention. Therefore, the scope of protection of the present invention should be determined by the scope of the claims.

Claims

1. A business customer service method based on large model sentiment recognition labeling and correction, characterized in that, The method comprises the following steps: automatically screening and constructing a commercial sentiment corpus using a large language model, and dividing the data set; independently fine-tuning a plurality of selected base models using a small amount of manually annotated samples, wherein the plurality of base models are large language models with strong sentiment recognition ability selected from a plurality of public models; using the fine-tuned base models to batch predict unannotated data to generate sentiment labels and reason explanations; comparing the prediction results of the base models to screen out disagreement samples with inconsistent labels; submitting the disagreement samples to a large model with stronger reasoning ability than the base model for auxiliary judgment to generate a fusion label, wherein the process of generating the fusion label comprises: constructing a standardized prompt template for the disagreement samples, which includes text content, sentiment labels predicted by the base model, and corresponding reason explanations; submitting the prompt template to the model interface of the large model for semantic reasoning; receiving the comprehensive judgment result returned by the large model as the final label, and marking it as a disagreement mediation sample; merging the manually annotated samples and the fusion label samples to construct a refined training set; iteratively fine-tuning the base models using the refined training set, and evaluating the model performance to form a closed-loop optimization mechanism, wherein the closed-loop optimization mechanism adopts a "generate-compare-correct-update" loop mechanism.

2. The method of claim 1, wherein, The process of constructing the commercial sentiment corpus comprises: analyzing public sentiment analysis data sources to automatically screen financial news, corporate announcements, or industry review texts; constructing a commercial sentiment corpus containing multiple samples, and performing data cleaning, format unification, and structure standardization operations; dividing the corpus into an unlabeled main training set, a manually annotated fine-tuning set, and a standard test set, and outputting a structured data file.

3. The method of claim 1, wherein, The process of independently fine-tuning a plurality of selected base models comprises: extracting commercial texts with real sentiment labels from the original training set, and using a three-classification system for labeling; inputting the labeled data into LLaMA-3.1-8B-it and Qwen-2.5-7B model frameworks; using LoRA parameter efficient injection technology to perform small sample rapid fine-tuning, and saving the optimal model weight parameters.

4. The method of claim 1, wherein, The process of screening out disagreement samples with inconsistent labels comprises: using the fine-tuned double models to batch predict the unannotated data set, outputting the sentiment prediction label and generative reason explanation of each text; comparing the label results of the double models on each text to perform a consistency judgment function; marking samples with inconsistent labels as conflict samples and including them in the conflict mediation pool.

5. The method of claim 1, wherein, The large model with stronger reasoning ability than the base model is GPT-3.5 or DeepSeek model.

6. The method of claim 1, wherein, The process of iterative fine-tuning comprises: merging the manually annotated samples and the fusion label samples to form a refined training set; using the refined training set to fine-tune the double models for a new round, and monitoring the training loss value, accuracy, and label fitting rate; comparing the changes in model performance, if the preset gain threshold is met, save the model and proceed to the next round of training, otherwise trigger the sample selection optimization strategy.

7. A business customer service system based on large model sentiment recognition annotation and correction, characterized in that, The system for implementing the method of any one of claims 1-6 comprises: A data management module for automatically screening and constructing a business sentiment corpus, and dividing the data set; A model training module for independently fine-tuning a plurality of selected base models using a small amount of manually annotated samples; A prediction and divergence detection module for batch prediction of unannotated data using the fine-tuned base models, and comparing the prediction results to screen out divergence samples with inconsistent labels; A label fusion module for submitting the divergence samples to a large model with stronger reasoning ability for auxiliary judgment to generate a fusion label; An iterative optimization module for merging the manually annotated samples and the fusion label samples to construct a refined training set, and iteratively fine-tuning and performance evaluating the base models to form a closed-loop optimization mechanism.

8. A computer terminal device, characterized by comprise: one or more processors; a memory coupled to the processors for storing one or more programs; when the one or more programs are executed by the one or more processors, the one or more processors implement the steps of the method of any one of claims 1-6.

9. A computer readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1-6.

10. A computer program product comprising a computer program, characterized in that, The computer program is executed by the processor to implement the steps of the method of any one of claims 1-6. The computer program is executed by the processor to implement the steps of the method of any one of claims 1-6.

Images