Label identification method based on large model text classification technology

Abstract

The invention relates to a label identification method based on a large model text classification technology. The label identification method comprises the following steps: adding a client label configuration process in an intelligent service process; pre-training the model; performing large model text classification and client label marking; performing intelligent service when the customer calls, and identifying the intention of the customer; according to the corresponding data table and the mobile phone number of the incoming call client, obtaining a stored corresponding label; and judging whether the customer incoming call expression is consistent with the storage label, and then executing by adopting different strategies. According to the method, firstly, text classification and customer label marking in the electric customer service field are achieved through the large model technology, secondly, customer intention recognition is achieved through the large model technology, differentiated services are provided for customers according to stored labels, and the efficiency of customer service for processing user appeals is improved.

Description

Technical Field

[0001] The present invention relates to the field of intelligent customer service systems, and in particular to a label recognition method based on large model text classification technology. Background Art

[0002] Traditional online customer service systems mainly rely on manual customer service to interact with customers. Although this method can provide personalized services, it is inefficient and difficult to handle a large number of customer inquiries during peak hours. With the development of artificial intelligence technology, intelligent customer service systems have gradually introduced intelligent assistance to replace manual customer service, but the existing system still has shortcomings in understanding customer intentions and providing relevant customized services. Summary of the invention

[0003] In order to overcome the problems of low manual efficiency and poor satisfaction in customer service in existing solutions, the present invention provides a customer tag recognition method. This method relies on large model technology and can analyze the conversation content during the conversation between manual online customer service and customers, and recognize customer tags according to customer demands, thereby completing customized services to improve customer service efficiency and customer experience. The specific technical solution is as follows:

[0004] A label recognition method based on large model text classification technology includes the following processes:

[0005] S100: Add customer tag configuration process to the smart service process;

[0006] S200: Fine-tune the pre-trained model Qwen-1.5-32B and set the model in the field of text classification and intent recognition; the model is pre-trained through the causal language modeling (CLM) task to enhance its language understanding ability; on this basis, the model is further fine-tuned using the annotated dataset to better adapt to the text classification and intent recognition tasks;

[0007] S300: Push the customer's historical chat information to the big model in an asynchronous manner to perform big model text classification and customer labeling;

[0008] S400: The intelligent service system matches the label, the session and the corresponding incoming call number, forms a corresponding data table of the incoming call number and the label, and stores the data table;

[0009] S500: When a customer calls, we provide intelligent services and identify the customer’s intentions;

[0010] S600: Obtain the stored corresponding tag according to the corresponding data table of S400 and the mobile phone number of the calling customer;

[0011] S700: Determine whether the customer's call description matches the stored tag; if so, go to S800; if not, go to S900;

[0012] S800: Perform differentiated configuration according to the label and provide differentiated services; end;

[0013] S900: The association between the corresponding tag and the mobile phone number is cancelled. The validity period is 1 month. After 1 month, if the analysis results of the customer's subsequent call demands meet the tag rules, new tag data will be generated and the service strategy will be executed according to the tag.

[0014] Preferably, the tags include four categories: query for multiple months, query for multiple households, SMS sending requirements, and smart caring services.

[0015] Preferably, the large model is a 40-layer Transformer structure, each layer contains 40 attention heads, the hidden layer size is 5120, and the vocabulary size reaches 152064.

[0016] Preferably, during the training process, the number of samples in each batch (batch size) is 4, and the model is iteratively optimized within 5 cycles (epochs).

[0017] Preferably, to ensure efficient gradient updating, choose

[0018] FP32StateAdamWeightDecay is used as the optimizer, and the cosine annealing strategy is combined with the warm-up phase to dynamically adjust the learning rate to promote the stable convergence of the model.

[0019] Preferably, in order to reduce the parameter scale and balance the model complexity and performance, a low-rank adaptation (LoRA) mechanism is added to the model, and lora_rank is set to 64, lora_alpha is set to 16, and lora_dropout is set to 0.05.

[0020] Preferably, in S900, the association between the corresponding tag and the mobile phone number is released, and the validity period is 1 month. After 1 month, if the analysis results of the customer's subsequent call demands meet the tag rules, new tag data is generated, and the service strategy is executed according to the tag.

[0021] Preferably, in S500, the process of identifying the customer intention includes:

[0022] S510: Define a semantic recognition data set according to the conversation content, and then proceed to S520;

[0023] S520: Clarify the components of the semantic recognition data set, determine the key sentences to be involved in intent recognition, and then go to S530;

[0024] S530: Preprocessing the session data;

[0025] S540: Using the power customer intention understanding model to intelligently identify customer intentions; the specific implementation process includes the following steps:

[0026] Perform data preprocessing on the obtained customer conversation data, mainly including word segmentation and text conversion operations;

[0027] The statistical word segmentation method reflects the closeness between words:

[0028]

[0029] When the mutual information value P(x,y) is greater than zero, it is determined that the correlation between word x and word y is high. When the mutual information value is greater than a certain threshold, it is determined that the two words form a word.

[0030] According to the loss function, the total loss of each training sample is calculated, and the weight is obtained to calculate the vector of each word. The formula is as follows:

[0031]

[0032] Preferably, the specific implementation process of S530 includes the following steps:

[0033] S531: feature transformation;

[0034] for Type of node, through the type-specific transformation matrix To do the conversion:

[0035]

[0036] S532: Standardization;

[0037] For data x, scale the data to a state where the mean is 0 and the variance is 1. The formula is as follows:

[0038]

[0039] S533: Calculate attention;

[0040]

[0041] Among them, wa i The constructed probability vector becomes the attention distribution of the i-th training text.

[0042] Preferably, the text steering uses the word2vec model, and the hyperparameters are set as windows_size=2, epochs=100, and learning_rate=0.01; wherein windows_size is defined as 2 to indicate that the words on the left and right sides of the target word are regarded as context words, and as the window slides, each word in the corpus will become a target word.

[0043] The present invention firstly realizes text classification and customer labeling in the field of electric power customer service through big model technology, and then realizes customer intent recognition through big model technology, and provides differentiated services to customers according to stored labels, thereby improving the efficiency of customer service in handling user demands. BRIEF DESCRIPTION OF THE DRAWINGS

[0044] Figure 1 It is a flowchart of a label recognition method based on large model text classification technology of the present invention. DETAILED DESCRIPTION

[0045] A label recognition method based on large model text classification technology includes the following processes:

[0046] S100: Adding a customer tag configuration process to the smart service process; the tags include four categories: query multiple months, query multiple households, SMS sending requirements, and smart care services;

[0047] S200: Fine-tune the pre-trained model Qwen-1.5-32B and set the model in the field of text classification and intent recognition; the model is pre-trained through the causal language modeling (CLM) task to enhance its language understanding ability. On this basis, the model is further fine-tuned using the annotated dataset to better adapt to the text classification and intent recognition tasks;

[0048] The large model is a 40-layer Transformer structure, each layer contains 40 attention heads, the hidden layer size is 5120, and the vocabulary size reaches 152064;

[0049] During the training process, the number of samples in each batch (batch size) is 4, and the model is iteratively optimized within 5 cycles (epochs);

[0050] In order to ensure efficient gradient updates, FP32StateAdamWeightDecay is selected as the optimizer, and the cosine annealing strategy is combined with the warm-up phase to dynamically adjust the learning rate to promote stable convergence of the model.

[0051] In order to reduce the parameter scale and balance the model complexity and performance, a low-rank adaptation (LoRA) mechanism was added to the model, and lora_rank was set to 64, lora_alpha was set to 16, and lora_dropout was set to 0.05;

[0052] S300: Push the customer's historical chat information to the big model in an asynchronous manner to perform big model text classification and customer labeling;

[0053] S400: The intelligent service system matches the label, the session and the corresponding incoming call number, forms a corresponding data table of the incoming call number and the label, and stores the data table;

[0054] S500: When a customer calls, we provide intelligent services and identify the customer’s intentions;

[0055] In this embodiment, the work order feature data set is defined as {province, work order category, time, account number, contact information, details}, and the total character string of the customer call obtained through voice recognition is "I want to check the recent electricity bill situation". Among them, the work order feature field province corresponds to "Dongli District, Tianjin City", and the customer intention can be calculated by S500 to obtain "electricity bill inquiry". The system will automatically call the tag library data, and the differentiated service "provide electricity bill data information for the past three months" can be obtained through S700;

[0056] The process of identifying customer intent includes:

[0057] S510: Define a semantic recognition data set according to the conversation content, and then proceed to S520;

[0058] S520: Clarify the components of the semantic recognition data set, determine the key sentences to be involved in intent recognition, and then go to S530;

[0059] S530: Preprocessing the session data; the specific implementation process includes the following steps:

[0060] S531: feature transformation;

[0061] for Type of node, through the type-specific transformation matrix To do the conversion:

[0062]

[0063] S532: Standardization;

[0064] For data x, scale the data to a state where the mean is 0 and the variance is 1. The formula is as follows:

[0065]

[0066] S533: Calculate attention;

[0067]

[0068] Among them, wa i The constructed probability vector becomes the attention distribution of the i-th training text;

[0069] The text steering uses the word2vec model, with hyperparameter settings of windows_size = 2, epochs = 100, and learning_rate = 0.01. The definition of windows_size as 2 means that the words on the left and right sides of the target word are regarded as context words. As the window slides, every word in the corpus will become the target word.

[0070] S540: Using the power customer intention understanding model to intelligently identify customer intentions; the specific implementation process includes the following steps:

[0071] Perform data preprocessing on the obtained customer conversation data, mainly including word segmentation and text conversion operations;

[0072] The statistical word segmentation method reflects the closeness between words:

[0073]

[0074] When the mutual information value P(x,y) is greater than zero, it is determined that the correlation between word x and word y is high. When the mutual information value is greater than a certain threshold, it is determined that the two words form a word.

[0075] According to the loss function, the total loss of each training sample is calculated, and the weight is obtained to calculate the vector of each word. The formula is as follows:

[0076]

[0077] S600: Obtain the stored corresponding tag according to the corresponding data table of S400 and the mobile phone number of the calling customer;

[0078] S700: Determine whether the customer's call description matches the stored tag; if so, go to S800; if not, go to S900;

[0079] S800: Perform differentiated configuration according to the label and provide differentiated services; end;

[0080] S900: The association between the corresponding tag and the mobile phone number is cancelled. The validity period is 1 month. After 1 month, if the analysis results of the customer's subsequent call demands meet the tag rules, new tag data will be generated and the service strategy will be executed according to the tag.

Claims

1. A label recognition method based on large model text classification technology, characterized in that: The process includes the following: S100: Add customer tag configuration process to the smart service process; S200: Fine-tune the pre-trained model Qwen-1.5-32B and set the model in the field of text classification and intent recognition; the model is pre-trained through the causal language modeling (CLM) task and further fine-tuned using the annotated dataset; S300: Push the customer's historical chat information to the big model in an asynchronous manner to perform big model text classification and customer labeling; S400: The intelligent service system matches the label, the session and the corresponding incoming call number, forms a corresponding data table of the incoming call number and the label, and stores the data table; S500: When a customer calls, we provide intelligent services and identify the customer’s intentions; S6 00: Get the stored corresponding tag according to the corresponding data table of S400 and the mobile phone number of the calling customer; S700: Determine whether the customer's call description matches the stored tag; if so, go to S800; if not, go to S900; S800: Perform differentiated configuration according to the label and provide differentiated services; end; S9 00: Remove the association between the corresponding tag and the mobile phone number. If the analysis results of the customer's subsequent call demands meet the tag rules, generate new tag data and execute the service strategy according to the tag.

2. According to claim 1, a label recognition method based on large model text classification technology is characterized in that: The tags include four categories: query for multiple months, query for multiple households, SMS sending needs, and smart caring services.

3. According to claim 2, a label recognition method based on large model text classification technology is characterized in that: The large model has a 40-layer Transformer structure, each layer contains 40 attention heads, the hidden layer size is 5120, and the vocabulary size reaches 152064.

4. According to claim 3, a label recognition method based on large model text classification technology is characterized in that: During the training process, the number of samples in each batch (batch size) is 4, and the model is iteratively optimized within 5 cycles (epochs).

5. According to claim 4, a label recognition method based on large model text classification technology is characterized in that: FP32StateAdamWeightDecay is selected as the optimizer, and the cosine annealing strategy is combined with the warm-up phase to dynamically adjust the learning rate.

6. According to claim 5, a label recognition method based on large model text classification technology is characterized in that: A low-rank adaptation (LoRA) mechanism was added to the model, and lora_rank was set to 64, lora_alpha was set to 16, and lora_dropout was set to 0.

05.

7. According to claim 1, a label recognition method based on large model text classification technology is characterized in that: In the above S900, the association between the corresponding tag and the mobile phone number is released, which is valid for 1 month. After 1 month, if the analysis result of the customer's subsequent call request meets the tag rule, new tag data is generated, and the service strategy is executed according to the tag.

8. A tag recognition method based on large model text classification technology according to any one of claims 1 to 7, characterized in that: In S500, the process of identifying the customer intention includes: S510: Define a semantic recognition data set according to the conversation content, and then proceed to S520; S520: Clarify the components of the semantic recognition data set, determine the key sentences to be involved in intent recognition, and then go to S530; S530: Preprocessing the session data; S540: Using the power customer intention understanding model to intelligently identify customer intentions; the specific implementation process includes the following steps: Perform data preprocessing on the obtained customer conversation data, mainly including word segmentation and text conversion operations; The statistical word segmentation method reflects the closeness between words: When the mutual information value P(x,y) is greater than zero, it is determined that the correlation between word x and word y is high. When the mutual information value is greater than a certain threshold, it is determined that the two words form a word. According to the loss function, the total loss of each training sample is calculated, and the weight is obtained to calculate the vector of each word. The formula is as follows:

9. The tag recognition method based on large model text classification technology according to claim 8 is characterized in that: The specific implementation process of S530 includes the following steps: S531: feature transformation; for Type of node, through the type-specific transformation matrix To do the conversion: S532: Standardization; For data x, scale the data to a state where the mean is 0 and the variance is 1. The formula is as follows: S533: Calculate attention; Among them, wa i The constructed probability vector becomes the attention distribution of the i-th training text.

10. The tag recognition method based on large model text classification technology according to claim 8 is characterized in that: The text steering uses the word2vec model with hyperparameter settings of windows_size = 2, epochs = 100, and learning_rate = 0.

01. The definition of windows_size as 2 means that the words on the left and right sides of the target word are regarded as context words. As the window slides, every word in the corpus will become a target word.

Images