Electric customer service dialogue intention and key information identification method based on large model

Abstract

The invention relates to the technical field of digital data processing, in particular to an electric customer service dialogue intention and key information identification method based on a large model. The identification method comprises the following steps of: finely adjusting a large model based on annotation data; constructing an intention and key information identification model for intelligent arrangement; intention and time identification is carried out based on small model assistance; fusing the identification results of the large model and the small model; and feeding back query information based on the identification result. The intention and time are further identified by using a small model, and the identification accuracy is improved. According to empirical data, when certain specific words are matched in an input text, intention categories can be determined, keyword libraries of different intention categories can be maintained, and accurate matching can be carried out. The accuracy of the fine-tuned large model is improved to a certain extent, but when a scene which is difficult to recognize by the large model exists, accurate matching can make up for the defect.

Description

Technical Field

[0001] This invention relates to the field of digital data processing technology, specifically to a method for identifying the intent and key information of power customer service dialogues based on a large model. Background Technology

[0002] Currently, intelligent customer service based on dedicated models identifies customer intent through multiple rounds of interaction, gradually collects relevant information, and generates answers. However, it cannot extract all key information, requiring customers to repeatedly express relevant information. This results in long overall interaction rounds and a poor customer experience.

[0003] Therefore, it is necessary to leverage the intent understanding and content generation capabilities of large models to construct an intelligent agent for power customer service intent recognition and key information recognition, accurately identify customer intent and related elements, call upon business systems to obtain customer information and knowledge information, combine them to generate answers, and improve customer satisfaction. Summary of the Invention

[0004] A customer calls 95598 for business consultation. The information the customer needs to inquire about is transcribed from speech to text and fed into a large-scale model for intent, key information, time, and account number identification. Intent identification is based on full-text recognition of the user's intent, identifying one or more of five scenarios: querying electricity consumption information, querying meter readings, querying payment information, querying electricity bill calculation methods, and querying overdue payments and late fees. Time identification involves recognizing descriptions of time in the input text and converting them into specific years and months. Account number identification identifies the account number the user is querying in the text to determine the query subject. Based on the user information identified by the large-scale model, a standard recognition result is provided.

[0005] This invention utilizes large-scale model intent understanding and content generation capabilities to construct an intelligent agent for electricity bill and consumption, accurately identifying customer intent and key information. It then calls upon business systems to retrieve customer and knowledge information, combining these to generate answers and improve customer satisfaction. The specific technical solution is as follows:

[0006] A method for identifying the intent and key information in power customer service conversations based on a large model includes the following process:

[0007] Step S1: Fine-tune the large model based on labeled data; prioritize "intent > account number > time"; fine-tuning includes: text labeled data, setting dynamic threshold multi-task loss function, fine-tuning the large power lighting model, and obtaining the fine-tuned large model M1;

[0008] Step S2: Construct an intent and key information recognition model for intelligent orchestration; to increase recognition accuracy, the large model is called three times when constructing the intent and key information recognition model. Obtain recognition results, including calling the large model for intent recognition, calling the large model for time recognition, and calling the large model for account number recognition;

[0009] Step S3: Perform intent and time recognition based on a small model;

[0010] Step S4: Merge the recognition results of the large model and the small model;

[0011] When the result of the small model is empty, the result of the large model is output directly;

[0012] When the small model result is not empty, if the recognition intent and time of the large model and the small model match, the large model result is output directly; if the time result of the large model is empty, the time recognition result of the small model is added; if the time of the large model and the small model are different, the time combination is selected as the final result; if there is a difference in intent recognition, the intent recognition result of the small model and the result of the large model are combined, and the intent relevance of the combined result is checked. If the intent relevance is lower than the threshold, the result of the large model is used directly; if the intent relevance is higher than the threshold, the combined result is used to obtain the final intent recognition result.

[0013] Step S5: Feedback on query information based on recognition results; Based on the intent recognition results and key information recognition of the large model, deeply embed them into the business process of the power customer service system, trigger the linkage query mechanism of data within the system, accurately retrieve power business data that matches the user's intent, and feed it back to the user or agent.

[0014] Furthermore, the text annotation data in step S1 includes the following specific processes:

[0015] Based on the raw text data, user intent is annotated to construct a structured labeled dataset: ;

[0016] in, Represents the query text; Represents a set of intent tags; Represents the account number; Represents time information;

[0017] The process of setting the dynamic threshold multi-task loss function specifically includes the following steps:

[0018] Using the power and light big data model as the base model, we simultaneously optimize three core recognition tasks:

[0019] Intent recognition task: Classify the specific business queries of users, namely: querying electricity bill and consumption information, querying meter readings, querying payment information, querying electricity bill calculation methods, and querying overdue payments and late fees; Account number extraction task: Extract 10-16 digit electricity account numbers;

[0020] Time extraction task: Extract query time-related information;

[0021] General formula for loss function: ;

[0022] in, Represents the total loss function; , , The dynamic weights for the three recognition tasks satisfy... ; The intention is to identify the loss; Losses were extracted on behalf of the account number; Loss is extracted over time;

[0023] Dynamic weight formula:

[0024] ;

[0025] ;

[0026] ;

[0027] in, , , : Task base weight, corresponding to intent, account number, and time, with priority "intent > account number > time";

[0028] : The target accuracy of the three tasks;

[0029] The current verification accuracy of the three tasks;

[0030] Constraints: ;

[0031] Evaluation of the effects of fine-tuning: ;

[0032] in, For the joint accuracy of the three recognition tasks,

[0033] Ask for the accuracy rates of intent recognition, account number recognition, and time extraction, respectively.

[0034] Fine-tuning parameter settings

[0035] Learning rate: Multi-task fine-tuning parameter values ​​are lower than single-task fine-tuning to avoid multi-task feature conflicts;

[0036] Training rounds: 6-8 rounds. Multi-task training requires more rounds to converge, combined with an early stopping mechanism for monitoring. ;

[0037] Batch size: 32, balancing feature learning and training efficiency;

[0038] Loss function optimizer: AdamW This helps to alleviate gradient oscillations during multi-task training.

[0039] Dropout probability: 0.2, higher than single-task, suppressing overfitting in multi-task scenarios;

[0040] Monitoring indicators: 3 consecutive rounds If there is no improvement, stop training;

[0041] Through training, a finely tuned large model is obtained. .

[0042] Furthermore, in step S2,

[0043] Different intelligent agent query schemes are set according to the peak and off-peak periods of incoming calls; during off-peak periods, the query call order is fixed; during peak periods of incoming calls, the query order of elements is intelligently allocated according to the time delay of different element queries.

[0044] Encode the input text: ;

[0045] Location coding:

[0046] ;

[0047] ;

[0048] Where pos is the position of the word in the sequence, starting from 0; It is the embedding dimension of the model; It is a dimensional index; even-numbered positions use the sine function, and odd-numbered positions use the cosine function.

[0049] The formula for calculating attention mechanism:

[0050] ;

[0051] The results are obtained through identification using a large model.

[0052] This invention further utilizes a smaller model to identify intent and time, improving recognition accuracy. Based on empirical data, when certain specific words match in the input text, the intent category can be determined, and a keyword library for different intent categories can be maintained for precise matching. While the fine-tuned larger model improves accuracy to some extent, precise matching can compensate for shortcomings in scenarios where the larger model struggles to identify certain elements. Attached Figure Description

[0053] Figure 1 This is a schematic diagram of the process for identifying the dialogue intent and key information of power customer service based on a large model, according to the present invention. Detailed Implementation

[0054] A method for identifying the intent and key information in power customer service conversations based on a large model includes the following process:

[0055] Step S1: Fine-tune the large model based on labeled data; including the following specific processes:

[0056] Step S1-1: Based on the original text data, annotate user intent and construct a structured labeled dataset:

[0057] ;

[0058] in, Represents the query text; This represents a set of intent tags, supporting multiple intent annotations, such as... ={Check electricity bill, check payment}; Represents the account number; Time information;

[0059] Step S1-2: Fine-tune the large model based on the annotation information.

[0060] The core logic of integrating three tasks—intent recognition, account number recognition, and time recognition—is fine-tuned, using the power grid's large-scale model as a base model, while simultaneously optimizing the three core recognition tasks:

[0061] Intent recognition task: Classify the specific business queries of users, such as querying electricity consumption information, querying meter readings, querying payment information, querying electricity calculation methods, and querying overdue payments and late fees, in 5 categories;

[0062] Account Number Extraction Task: Extract 10-16 digit electricity account numbers, such as "123456789012345";

[0063] Time extraction task: Extract query time-related information, such as "May 2025", "last month", and "this month"; General formula for loss function:

[0064] ;

[0065] The dynamic weights for the three recognition tasks satisfy... ,

[0066] The representative intended to identify the loss.

[0067] Loss can be withdrawn on behalf of the account number.

[0068] Loss is extracted over time.

[0069] The loss function uses dynamic weights to calculate the loss function based on different recognition tasks, which improves the model's accuracy.

[0070] Steps S1-3: Measure the training of the three tasks using dynamic weights.

[0071] Dynamic weight formula:

[0072] ;

[0073] ;

[0074] ;

[0075] in, , , The basic weight of the task corresponds to the intent, account number, and time, with priority "intent > account number > time". For example... , , =0.2.

[0076] The target accuracy for the three tasks, for example =97%, =95%, =91%.

[0077] : The current verification accuracy of the three tasks.

[0078] Constraints: This constraint on weight conditions can prevent weights from becoming extreme.

[0079] The lower the current accuracy of a task, the higher its weight, and the more priority is given to addressing weak recognition tasks.

[0080] Evaluation of the effects of fine-tuning:

[0081] ;

[0082] in, For the joint accuracy of the three recognition tasks,

[0083] Ask for the accuracy rates of intent recognition, account number recognition, and time extraction, respectively.

[0084] The overall effectiveness is evaluated by weighting the results according to business priorities, which is more in line with the actual application scenarios of power customer service.

[0085] Fine-tuning parameter settings

[0086] Learning rate: Multi-task fine-tuning parameter values ​​are lower than single-task fine-tuning to avoid multi-task feature conflicts.

[0087] Training rounds: 6-8 rounds. Multi-task training requires more rounds to converge, combined with an early stopping mechanism for monitoring. .

[0088] Batch size: 32, balancing feature learning and training efficiency.

[0089] Loss function optimizer: AdamW This helps to alleviate gradient oscillations during multi-task training.

[0090] Dropout probability: 0.2, higher than single-task, suppressing overfitting in multi-task scenarios.

[0091] Monitoring indicators: 3 consecutive rounds If there is no improvement, stop training.

[0092] Through training, a finely tuned large model is obtained. .

[0093] Step S2: Construct an intent and key information recognition model for intelligent orchestration.

[0094] To increase the accuracy of recognition, the large model is called three times when constructing intent and identifying key information. The process involves obtaining recognition results, including calling a large model for intent recognition, time recognition, and account number recognition. The recognition process is as follows: Figure 1 As shown.

[0095] Different intelligent agent query scheduling schemes are set according to peak and off-peak call times. During off-peak times, the query call order can be fixed, such as "identify intent - identify account number - identify time". During peak call times, the query order of elements is intelligently allocated according to the latency of different element queries. For example, during peak call times, when customer A calls, the text is retrieved, the recognition model is invoked, and the query order is "identify intent - identify account number - identify time"; when customer B calls, the text is retrieved, the recognition model is invoked, and the query order is "identify account number - identify intent - identify time"; when customer C calls, the text is retrieved, the recognition model is invoked, and the query order is "identify account number - identify time - identify intent". Intelligent scheduling during peak call times can reduce latency.

[0096] For example, the input text is used to identify intent and key information to obtain the output of a large model.

[0097] Encode the input text:

[0098] ;

[0099] Location coding:

[0100] ;

[0101] ;

[0102] Where pos is the position of the word in the sequence, starting from 0; It is the embedding dimension of the model, such as 512 in Transformer; It is a dimensional index; even-numbered positions use the sine function, and odd-numbered positions use the cosine function.

[0103] The formula for calculating attention mechanism:

[0104] ;

[0105] Because of the After the dot product, the variance of the result will increase significantly, divided by... This avoids the gradient problem caused by excessively large values, resulting in smoother and more stable attention weights.

[0106] Results are obtained through large-scale model recognition, for example:

[0107] {

[0108] Intent: "Query electricity bill and consumption information"

[0109] "Time": "2025.12"

[0110] Account Number: "12345678901"

[0111] }

[0112] Step S3: Perform intent and time recognition based on a small model.

[0113] This invention further utilizes a smaller model to identify intent and time, improving recognition accuracy. Based on empirical data, when certain specific words match in the input text, the intent category can be determined, and a keyword library for different intent categories can be maintained for precise matching. While the fine-tuned larger model improves accuracy to some extent, precise matching can compensate for shortcomings in scenarios where the larger model struggles to identify certain elements.

[0114] Similarly, for time identification, especially descriptive times such as "current month" or "this month," a corresponding keyword database is maintained for these deterministic times, and precise matching is performed to provide results. For example:

[0115] {

[0116] Intent: "Query electricity bill and consumption information"

[0117] Time: December 2025

[0118] }

[0119] If no keyword is matched, the result will be empty.

[0120] Step 4: Merge the recognition results of the large model and the small model.

[0121] First, the results from the large model are non-empty, while the results from the small model may be empty. Based on the results from both the large and small models, result enhancement is performed to further improve recognition accuracy.

[0122] When the small model result is empty, the large model result is output directly.

[0123] When the small model result is not empty, if the recognition intent and time of the large and small models match, the large model result is output directly. If the large model time result is empty, the small model time recognition result is added. If the large and small models have different times, the time combination is selected as the final result. If there are differences in intent recognition, the intent recognition result of the small model and the large model result are combined, and the intent relevance of the combined result is checked. If the intent relevance is lower than the threshold, the result of the large model is used directly; if the intent relevance is higher than the threshold, the combined result is used to obtain the final intent recognition result.

[0124] The calculation of intent relevance uses statistical principles, which measures the proportion of a certain intent combination to the total number of multiple intents. For example, the relevance of "query meter readings, query payment information" is 0.5.

[0125] The calculation of relevance, the relevance between two or more intentions:

[0126] ;

[0127] in, This indicates the degree of association, indicating the strength of the association between multiple intents. For example, the degree of association between "query meter readings and query payment information" is 0.4, and the degree of association between "query meter readings, query payment information, and query electricity bill calculation method" is 0.1.

[0128] Indicates the total number of combinations of two or more intentions; The total number of all multiple intentions.

[0129] Step 5: Provide query information based on the recognition results.

[0130] Based on the intent recognition results and key information recognition of the large model, it is deeply embedded into the business process of the power customer service system, triggering the linkage query mechanism of data within the system, accurately retrieving power business data that matches the user's intent, and feeding it back to the user or agent.

Claims

1. A method for identifying dialogue intent and key information in power customer service based on a large model, characterized in that, The process includes the following: Step S1: Fine-tune the large model based on labeled data; prioritize "intent > account number > time"; fine-tuning includes: text labeled data, setting dynamic threshold multi-task loss function, fine-tuning the power lighting large model, and obtaining the fine-tuned large model M1; Step S2: Construct an intent and key information recognition model for intelligent orchestration; to increase recognition accuracy, the large model is called three times when constructing the intent and key information recognition model. Obtain recognition results, including calling the large model for intent recognition, calling the large model for time recognition, and calling the large model for account number recognition; Step S3: Perform intent and time recognition based on a small model; Step S4: Merge the recognition results of the large model and the small model; When the result of the small model is empty, the result of the large model is output directly; When the small model result is not empty, if the recognition intent and time of the large model and the small model match, the large model result is output directly; if the time result of the large model is empty, the time recognition result of the small model is added; if the time of the large model and the small model are different, the time combination is selected as the final result; if there is a difference in intent recognition, the intent recognition result of the small model and the result of the large model are combined, and the intent relevance of the combined result is checked. If the intent relevance is lower than the threshold, the result of the large model is used directly; if the intent relevance is higher than the threshold, the combined result is used to obtain the final intent recognition result. Step S5: Feedback on query information based on recognition results; Based on the intent recognition results and key information recognition of the large model, deeply embed them into the business process of the power customer service system, trigger the linkage query mechanism of data within the system, accurately retrieve power business data that matches the user's intent, and feed it back to the user or agent.

2. The method for identifying the dialogue intent and key information of power customer service based on a large model according to claim 1, characterized in that, The text annotation data in step S1 includes the following specific processes: Based on raw text data, user intent is annotated to construct a structured labeled dataset. ； in, Represents the query text; Represents a set of intent tags; Represents the account number; Represents time information; The process of setting the dynamic threshold multi-task loss function specifically includes the following steps: Using the power and light big data model as the base model, we simultaneously optimize three core recognition tasks: Intent recognition task: Classify the specific business queries of users, namely: querying electricity consumption information, querying meter readings, querying payment information, querying electricity calculation methods, and querying overdue payments and late fees; Account Number Extraction Task: Extract 10-16 digit electricity account numbers; Time extraction task: Extract query time-related information; General formula for loss function: ； in, Represents the total loss function; , , The dynamic weights for the three recognition tasks satisfy... ; The intention is to identify the loss; Losses were extracted on behalf of the account number; Loss is extracted over time; Dynamic weight formula: ； ； ； in, , , : Task base weight, corresponding to intent, account number, and time, with priority "intent > account number > time"; , , : The target accuracy of the three tasks; , , The current verification accuracy of the three tasks; Constraints: ; Fine-tuning effect evaluation ； in, For the joint accuracy of the three recognition tasks, Ask for the accuracy rates of intent recognition, account number recognition, and time extraction, respectively. Fine-tuning parameter settings Learning rate: Multi-task fine-tuning parameter values ​​are lower than single-task fine-tuning to avoid multi-task feature conflicts; Training rounds: 6-8 rounds. Multi-task training requires more rounds to converge, combined with an early stopping mechanism for monitoring. ; Batch size: 32, balancing feature learning and training efficiency; Loss function optimizer: AdamW This helps to alleviate gradient oscillations during multi-task training. Dropout probability: 0.2, higher than single-task, suppressing overfitting in multi-task scenarios; Monitoring indicators: 3 consecutive rounds If there is no improvement, stop training; Through training, a finely tuned large model is obtained. .

3. The method for identifying the dialogue intent and key information of power customer service based on a large model according to claim 1, characterized in that, In step S2 Different intelligent agent query schemes are set according to the peak and off-peak periods of incoming calls; during off-peak periods, the query call order is fixed; during peak periods of incoming calls, the query order of elements is intelligently allocated according to the time delay of different element queries. Encode the input text: ； Location coding: ； ； Where pos is the position of the word in the sequence, starting from 0; It is the embedding dimension of the model; It is a dimensional index; even-numbered positions use the sine function, and odd-numbered positions use the cosine function. The formula for calculating attention mechanism: ； The results are obtained through identification using a large model.

Images