A large model-based conversation analysis method and system

Abstract

This invention proposes a large-scale model-based conversation analysis method and system, belonging to the field of artificial intelligence technology. Given dialogue text and target concerns in a business scenario; a portion of the dialogue text, corresponding target concerns, and prompt words are input into a large-scale model, which outputs a description of the specific category of the dialogue under the target concerns, standardizing the labels of each round of dialogue text; a pre-trained encoder is trained in two stages using labeled and unlabeled dialogue texts; a set of dialogue texts to be analyzed, corresponding target concerns, and prompt words are input into the large-scale model, and dialogue texts without target concerns are removed based on the output description of the large-scale model. The remaining dialogue texts are used to extract features using the trained encoder and undergo initial clustering. Based on the initial clustering results, cluster merging and outlier identification are performed to optimize the clustering results and generate a conversation analysis report containing target concerns. This invention improves the clustering effect of conversation analysis.

Description

Technical Field

[0001] This invention belongs to the field of artificial intelligence technology, and in particular relates to a conversation analysis method and system based on a large model. Background Technology

[0002] With the development of large-scale models, conversation analysis, as a core technology in the field of human-computer interaction, is gradually demonstrating its importance in multiple areas. In the marketing field, the dialogue text between human customer service (or AI customer service) and customers contains a variety of useful information. Capturing valuable information from the dialogue records and generating intelligent, natural, and structured responses helps improve work efficiency in this business scenario and provides customers with a more intelligent, accurate, and efficient conversational interaction experience.

[0003] Traditional conversation analysis systems are mostly based on predefined rules or traditional machine learning methods such as classification and clustering. When clustering, they typically use the similarity of encoded features from pre-trained models, with K-MEANS and DBSCAN being common methods. However, these methods often fall short when dealing with complex and varied natural language dialogues, especially in multi-turn dialogues, fuzzy intent parsing, and contextual semantic understanding, exhibiting significant limitations, mainly in the following aspects:

[0004] 1. It heavily relies on classification models and can only analyze data from existing classification models;

[0005] 2. The clustering is too disorganized, lacking semantically guided clustering merging methods;

[0006] 3. Poor clustering performance: The classification model has low accuracy and lacks a feedback mechanism to improve the model's results.

[0007] 4. Directly using the encoding vectors of pre-trained models often lacks prior knowledge about the clustering granularity.

[0008] Existing conversation analysis techniques have bottlenecks in semantic understanding depth, computational efficiency, and classification accuracy under multiple concerns, making it impossible to accurately cluster natural language dialogues under multiple concerns. Summary of the Invention

[0009] In view of this, the present invention proposes a conversation analysis method and system based on a large model, and the technical solution adopted is as follows:

[0010] Firstly, a large-model-based session analysis method includes:

[0011] Given the dialogue text and target concerns in a specific business scenario;

[0012] Input a portion of the dialogue text, the corresponding target focus, and the prompt words into the large model. The large model analyzes the key points of each round of dialogue and outputs a result description of the specific category of the dialogue under the target focus. Based on the result description, the labels of each round of dialogue text are standardized to obtain a portion of the labeled dialogue text.

[0013] The pre-trained encoder is trained in two stages by combining labeled and unlabeled dialogue texts. In the first stage, data augmentation is performed on both labeled and unlabeled dialogue texts, and the cross-entropy loss of the labeled dialogue texts and the self-contrast learning loss of all dialogue texts are calculated. In the second stage, the self-contrast learning loss is calculated at the beginning of each training iteration, then all dialogue texts are clustered to generate pseudo-labels for the unlabeled dialogue texts, and finally the cross-entropy loss is calculated by mixing the labeled and pseudo-labeled dialogue texts.

[0014] Input a set of dialogue texts to be analyzed, the corresponding target attention points, and prompt words into the large model. Based on the output of the large model, dialogue texts that do not contain target attention points are removed. The remaining dialogue texts are used to extract features and perform initial clustering using the trained encoder. Based on the initial clustering results, cluster merging and outlier identification are performed to optimize the clustering results and generate a conversation analysis report containing target attention points.

[0015] Furthermore, the target focus points refer to the information that the service representative wants to focus on based on the dialogue text between the service representative and the customer in the business scenario, including the reasons for customer satisfaction, the reasons for customer dissatisfaction, the reasons for customer refusal, customer ratings, and the reasons why the customer thinks the service representative's answers are irrelevant. Each focus point corresponds to several categories.

[0016] Furthermore, the standardization of the labels for each round of dialogue text based on the result description refers to converting the result description of the specific category under the target concern output by the large model into a category label under that target concern.

[0017] Furthermore, the pre-trained encoder is a BERT encoder.

[0018] Furthermore, the process of performing the first stage of training on the pre-trained encoder includes:

[0019] In each training step, a set of labeled dialogue text and a set of unlabeled dialogue text are taken;

[0020] The labeled and unlabeled dialogue texts are input into the pre-trained encoder twice, one after the other. Each dialogue text corresponds to a pair of features (z, z) output by the pre-trained encoder in the two separate inputs. ′ ), where z is the feature generated by the pre-trained encoder for the first input, z ′ These are the enhanced features generated by the second input encoder;

[0021] For a set of labeled dialogue texts, calculate the cross-entropy loss before and after enhancement, and take the average as the total cross-entropy loss;

[0022] The self-contradictory learning loss is calculated by taking the features of each dialogue text and its corresponding augmented features as positive samples, and taking the features of each dialogue text and the augmented features of another dialogue text as negative samples.

[0023] By combining the total cross-entropy loss and the self-comparative learning loss, and iterating through several training steps, the first stage of training of the pre-trained encoder is completed.

[0024] Furthermore, the second-stage training process for the pre-trained encoder includes:

[0025] For each training step, take one set of labeled dialogue text and one set of unlabeled dialogue text, and perform the following three steps in sequence:

[0026] The first step is to calculate the self-contradictory learning loss using the first-stage method;

[0027] The second step is to cluster the features of labeled and unlabeled dialogue texts to generate pseudo-labels for the unlabeled data.

[0028] The third step is to treat the pseudo-labeled dialogue text as labeled data, mix all dialogue texts, and use the first-stage method to calculate the total cross-entropy loss.

[0029] Combining the self-contradictory learning loss and the total cross-entropy loss, the pre-trained encoder is trained in the second stage by iterating through several training steps.

[0030] Furthermore, the cluster merging includes:

[0031] For small sample clusters in the initial clustering results, select the top 5 clusters that are closest to them, and randomly select a number of samples from each of the top 5 clusters. The large model determines whether the small sample cluster belongs to one or more of the top 5 clusters. If it does, merge the small sample cluster into the corresponding cluster; otherwise, do not merge.

[0032] Furthermore, the outlier identification includes:

[0033] Outlier detection is achieved using the local outlier factor. Half the size of each cluster is selected as the number of neighbors. The outlier factor is standardized using an auxiliary test set as a threshold, and points exceeding the threshold are considered outliers. For each detected outlier, the top 5 clusters closest to it are selected. Several samples are randomly selected from each of the top 5 clusters, and the large model determines whether the outlier belongs to one or more of the top 5 clusters. If it does, the outlier is merged into the corresponding cluster; otherwise, the outlier is removed.

[0034] Furthermore, the conversation analysis report includes information on points of interest and the distribution of different categories under each point of interest.

[0035] Secondly, the present invention provides a large-model-based conversation analysis system for implementing the aforementioned large-model-based conversation analysis method.

[0036] The beneficial effects of this invention are:

[0037] This invention leverages the structured output capabilities of a large model and its natural language understanding capabilities to assist in analyzing points of interest in dialogues. Based on the output of the large model, it employs standardization, preliminary clustering, cluster merging, and outlier identification to obtain optimized clustering results, further guiding the generation of conversation analysis reports that include the target points of interest. Furthermore, the text features used for clustering are generated by an encoder trained in two stages, improving the final clustering effect in multiple ways and generating more accurate conversation analysis reports. Attached Figure Description

[0038] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0039] Figure 1 This is a schematic diagram of the framework for a large-model-based conversation analysis method.

[0040] Figure 2 This is a schematic diagram of the first phase of training;

[0041] Figure 3 This is a diagram illustrating the second phase of training. Detailed Implementation

[0042] The present invention will be further described and illustrated below with reference to specific embodiments. The embodiments described are merely examples of the content of this disclosure and do not limit the scope of the invention. The technical features of each embodiment in the present invention can be combined accordingly, provided that there is no mutual conflict.

[0043] The accompanying drawings are merely illustrative of the invention and are not necessarily drawn to scale. Some of the block diagrams shown in the drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software, in one or more hardware modules or integrated circuits, or in different network and / or processor devices and / or microcontroller devices.

[0044] The flowchart shown in the attached diagram is merely an illustrative example and does not necessarily include all steps. For example, some steps may be broken down, while others may be combined or partially combined; therefore, the actual execution order may change depending on the specific circumstances.

[0045] like Figure 1 As shown, this invention proposes a conversation analysis method and system based on a large model, the main implementation process of which is as follows:

[0046] S1, given the dialogue text and target concerns in the business scenario.

[0047] S2: Input a portion of the dialogue text, the corresponding target focus, and the prompt words into the large model. The large model analyzes the key points of each round of dialogue and outputs a result description of the specific category of the dialogue under the target focus. Based on the result description, the labels of each round of dialogue text are standardized.

[0048] S3 combines labeled and unlabeled dialogue text to train the pre-trained encoder in two stages. In the first stage, data augmentation is performed on the labeled data, cross-entropy loss is calculated, and self-contrast learning loss is calculated by mixing all data. In the second stage, at the beginning of each training iteration, all data are first mixed to calculate self-contrast learning loss, then all data are clustered to generate pseudo-labels for unlabeled data, and finally, the labeled and pseudo-labeled data are mixed to calculate cross-entropy loss.

[0049] S4. Input a set of dialogue texts to be analyzed, the corresponding target attention points and prompt words into the large model. Based on the output of the large model, remove texts that do not contain target attention points. Extract the encoded features of the remaining texts using the encoder trained in step S3 and cluster them. Perform cluster merging and outlier identification based on the initial clustering results, optimize the clustering results, and generate a conversation analysis report containing target attention points.

[0050] In step S1 above, the dialogue text and target focus for each round are written in JSON format, for example, {"text":text,entity:[{"value":value,"description":description}]}, where "text":text represents the dialogue text for one round, "value":value represents the target focus, and "description":description represents the natural language description of the target focus.

[0051] The aforementioned dialogue text refers to a set of conversations between a human customer service representative (or AI customer service representative) and a customer in a specific business scenario. In this set of conversations, the focus can be on reasons for customer satisfaction, reasons for customer dissatisfaction, reasons for customer refusal, reasons why the customer feels that the customer service representative did not answer the question, etc. Each focus can be divided into multiple categories. For example, reasons for customer satisfaction can be categorized as good customer service, good product brand, good product quality, and strong product promotions, etc.

[0052] The dialogue text can be extracted directly from the text conversation, or it can be generated by recognizing the text in the voice call.

[0053] The purpose of natural language descriptions of target concerns is to guide large models to obtain better analysis results. For example, if a target concern is "the reasons for customer satisfaction", the corresponding natural language description could be "what are the reasons why the customer expressed satisfaction in this round of dialogue". Natural language descriptions are further explanations of target concerns and can be based on the experience of those skilled in the art.

[0054] In step S2 above, the big model is used to analyze the key points in the dialogue text. The big model is required to output the analysis results under the target focus based on the input dialogue text. The analysis results here are a description of the specific category of the dialogue text under the target focus.

[0055] For example, given a target focus of "reasons for customer satisfaction," based on the input dialogue text, the output of the large model might be "This promotion is very generous." Based on this output, the dialogue text can be categorized under the label "generous product promotion." It can be observed that there are both similarities and differences between the large model's output and the labels; they are not aligned. By classifying the output of the large model, standardized labels for each round of dialogue text input into the large model can be obtained.

[0056] In this step, there is no limit to the number of target concerns; there can be one or more.

[0057] In this embodiment, the prompt words for large models are generally obtained through benchmark testing to determine the best prompt words for each model. For large models that support structured output and tool calls, their structured output function is used directly. For large models that do not support structured output, the output requirements are noted in the prompt words.

[0058] The prompts that guide the output of large models can be:

[0059] ##Context##

[0060] In telemarketing and customer service scenarios, we often need to determine the customer's true intentions based on the conversation between the customer and the service representative, and extract the key information provided from the conversation.

[0061] ##Target##

[0062] You are an expert in understanding the dialogue between customers and service representatives. Your task is to {{analyze and describe the specific categories of concerns from the dialogue}}. You need to obtain {%dialogue text list%}{{concerns}}({{natural language descriptions of the concerns}}).{%endfor%}};

[0063] The output of the large model is also a JSON list, but this JSON list is unclustered and unaligned.

[0064] In step S3 above, the pre-trained encoder is preferably the Bert encoder, and the Bert encoder is trained in two stages to optimize its ability to further cluster.

[0065] like Figure 2 As shown, in the first stage of training, the text data is divided into labeled and unlabeled parts, where the labels for the labeled data are generated after standardization in step S2. In each training step of this stage, one labeled batch and one unlabeled batch are selected. For the labeled and unlabeled batches, the same dialogue text is input into the BERT encoder twice, sequentially. Due to the dropout function built into the BERT encoder, the BERT encoded features z and z' obtained in the two sequential inputs are... ′ There are subtle differences between the two. To reduce these differences, the encoded feature z... ′ This can be viewed as an enhancement of the encoded feature z. The cross-entropy loss is calculated through this method of data augmentation of labeled data.

[0066]

[0067] in, These represent the cross-entropy loss of the original data and the augmented data, respectively; Let represent the total cross-entropy loss, which is the average of the two cross-entropy losses; K represents the total number of categories under all points of interest, and y a Indicates a label, Let z represent the label of the i-th sample. i ) j Let N represent the encoded feature of the i-th sample in the j-th category, and N represent the number of samples in this batch.

[0068] In addition, self-comparative learning loss is calculated by mixing labeled and unlabeled data;

[0069] In this embodiment, the features of each dialogue text and its corresponding enhanced features are used as positive samples, and the features of each dialogue text and the enhanced features of another dialogue text are used as negative samples to calculate the self-contradictory learning loss.

[0070] Taking into account the two losses mentioned above, the first phase of training is completed.

[0071] like Figure 3 As shown, in the second phase of training, in each training iteration, a set of labeled dialogue text and a set of unlabeled dialogue text are taken, and the following three steps are performed sequentially:

[0072] The first step is to calculate the self-contradictory learning loss by mixing all the data, using the same self-contradictory learning method as in the first stage.

[0073] The second step is to cluster all the data to generate pseudo-labels for the unlabeled data.

[0074] Before clustering, the BERT encoder is used to obtain the encoded features of the samples. The generated encoded features are then subjected to k-means clustering. Some of the samples participating in the clustering have labels. The Hungarian algorithm is used to align these labels and assign a label to each cluster. Based on this, each unlabeled data point obtains a cluster label, which is then used as a pseudo-label for the unlabeled data.

[0075] The third step involves calculating the cross-entropy loss by mixing labeled and pseudo-labeled data, thus completing further supervised training.

[0076] After several iterations, the second stage of training is completed.

[0077] In the training process of this invention, self-supervised learning and semi-supervised learning are combined to improve clustering performance. Self-supervised learning learns discriminative category representations after each clustering to facilitate subsequent clustering. A dual learning strategy is employed, with training performed at both the instance level and the cluster level. In instance-level learning, by ensuring similar instances are grouped into the same class and distinguished from instances of other classes, intra-class compactness and inter-class separability are enhanced, especially under strong data augmentation. In cluster-level learning, model parameters are updated based on cluster assignments, improving the model's ability to distinguish categories. In the above learning process, this invention combines cross-entropy loss and contrastive loss to obtain encoding features beneficial to clustering. This invention effectively identifies different instances through the first stage of training, and in the second stage, it uses clustered data to generate consistent instance-level pseudo-labels. By using specific pseudo-labels as self-supervised signals, different categories are clearly distinguished, improving clustering performance and accelerating convergence.

[0078] In semi-supervised learning, the first stage augments the labeled data, and these augmentations are then combined in contrastive learning to fully utilize the limited labeled data to initialize the clustering representation. A semi-supervised contrastive loss is proposed, which includes positive sample pairs (each sample feature and its corresponding augmented sample feature) and negative sample pairs (different sample features). To enhance the discriminative power of known categories, a cross-entropy loss is added as a supervisory signal for the labeled data.

[0079] This invention proposes a clustering and representation learning method. After the first stage of training, directly using all data for clustering and representation learning can lead to situations where labeled samples of the same category are assigned to different clusters, a phenomenon known as catastrophic forgetting. To mitigate this problem, at the beginning of each training iteration, semi-supervised contrastive learning is used to strengthen the memory of known categories by augmenting samples, thereby guiding the subsequent clustering process. Centroid-guided clustering and self-supervised learning then continue.

[0080] In step S4 above, such as Figure 1 As shown, a set of dialogue texts to be analyzed, the corresponding target attention points and prompt words are input into the large model. According to the output of the large model, texts without target attention points are removed, and the remaining texts are used to extract encoded features and cluster them using the encoder trained in step S3.

[0081] In this step, post-processing methods such as outlier detection and cluster merging are used to further improve the clustering results.

[0082] (1) Cluster merging

[0083] To determine whether the initial clustering results contain small sample clusters, a threshold for the number of samples within a cluster can be set. For small sample clusters, select the top 5 clusters closest to them, randomly select several samples from each of the top 5 clusters, and let the large model determine whether the small sample cluster belongs to one or more of the top 5 clusters. If it does, merge the small sample cluster into the corresponding cluster; otherwise, do not merge.

[0084] When the large model makes its judgment, samples from the small sample cluster and several samples selected from each of the Top 5 clusters are input into the large model. A prompt word is provided to guide the large model in determining whether the small sample cluster belongs to the same class as the selected samples from each of the Top 5 clusters. If the large model determines that they belong to the same class, the small sample cluster is merged into the cluster corresponding to the selected samples. Here, the large model may determine that the small sample cluster belongs to multiple Top 5 clusters. In this case, merging the small sample cluster into multiple clusters simultaneously indicates that the small sample cluster contains different categories under the same point of interest.

[0085] (2) Outlier identification

[0086] This invention employs Local Outlier Factor (LOF) to detect outliers, a density-based unsupervised machine learning algorithm. Before outlier detection, half the size of each cluster is selected as the number of neighbors, and the outlier factor standardized using the test set is used as a threshold. Points exceeding the threshold are considered outliers. This invention addresses the low accuracy issues of fixed outlier factors and neighbor sizes in the LOF algorithm by using a dynamic neighbor size and global threshold scheme.

[0087] This invention further optimizes the clustering results by performing cluster merging and outlier identification on the initial clustering results, and generates a session analysis report containing the target points of interest based on the optimized clustering results.

[0088] In one specific embodiment of the present invention, for the detected outlier, the Top 5 clusters closest to the outlier are selected, and several samples are randomly selected from each of the Top 5 clusters. The large model determines whether the outlier belongs to one or more of the Top 5 clusters. If it does, the outlier is merged into the corresponding cluster; otherwise, the outlier is removed.

[0089] When the large model makes its judgment, outliers and several samples selected from each of the Top 5 clusters are input into the large model. A prompt is provided to guide the large model in determining whether the outlier belongs to the same class as the samples selected from each cluster. If the large model determines that they do, the outlier is merged into the cluster corresponding to the selected samples. Here, the large model may determine that the outlier belongs to multiple Top 5 clusters. In this case, merging the outlier into multiple clusters simultaneously indicates that the outlier contains different categories under the same point of interest.

[0090] A conversation analysis report is generated based on the optimized clustering results. The conversation analysis report includes information on points of interest and the distribution of different categories under each point of interest. This is an objective data generated by analyzing the key points of the conversation. The conversation analysis results can be used to guide the work of human customer service or configure AI customer service scripts, configure product promotion efforts, etc. to improve customer satisfaction, help improve work efficiency in this business scenario, and provide customers with a more intelligent, accurate and efficient conversation interaction experience.

[0091] To verify the effectiveness of this invention, the distribution of various categories under multiple concerns, such as customer ratings, reasons for customer dissatisfaction, and reasons for customer satisfaction, was analyzed based on customer-customer conversation texts over a certain period. A total of 54,032 training data entries, categorized into 263 classes, were used for testing.

[0092] This embodiment tested the performance of pre-trained encoders, pre-trained encoders + first-stage training, pre-trained encoders + two-stage training, pre-trained encoders + two-stage training + outlier identification, pre-trained encoders + two-stage training + cluster merging, pre-trained encoders + two-stage training + cluster merging + outlier identification, pre-trained encoders + DBSCAN clustering model, and pre-trained encoders + HDBSCAN clustering model. NMI, ACC, and SIL were used as evaluation metrics, and the test results are shown in Table 1.

[0093] Model Name NMI ACC SIL bge-base-zh 82.4 60.66 9.3439 bge-base-zh-pretrained 88.67 75.76 13.3848 bge-base-zh-tuned 90.64 78.86 29.9871 bge-base-zh-tuned+cluster merging 92.52 80.81 None bge-base-zh-tuned+Outlier Identification 93.71 82.69 None bge-base-zh-tuned+cluster merging+outlier identification (this invention) 95.15 88.35 None bge-base-zh+DBSCAN 41.11 35.88 4.7565 bge-base-zh+HDBSCAN 44.09 38.92 5.0814

[0094] In Table 1 above, bge-base-zh represents the pre-trained encoder, pretrained represents the first stage of training, and tuned represents the two-stage training. It can be seen that the training in these two stages has brought different degrees of improvement to the model performance.

[0095] Regarding the metrics used:

[0096] 1. NMI (Normalized Mutual Information) is a metric used to measure the similarity between two partitions (e.g., the clustering result obtained by an algorithm and the true labels). It is based on the concept of information entropy, measuring the amount of information one partition contains about another partition.

[0097] 2. The Silhouette Coefficient (SIL) is a commonly used metric for evaluating the performance of unsupervised clustering. It focuses on how closely clustered data points should be within the same cluster and how widely separated different clusters should be. This metric is not applicable to schemes that perform post-processing optimization on the initial clustering.

[0098] 3. ACC (Accuracy), in clustering tasks, measures the degree of overlap between the clustering results and the true categories using the "optimal mapping" method. That is, it first attempts to match each cluster label with a true label to achieve the highest classification accuracy, and then calculates that accuracy.

[0099] As shown in Table 1, the present invention has the best effect.

[0100] This embodiment also provides a large-model-based session analysis system for implementing the above analysis method, including:

[0101] The input module is used to input dialogue text and target concerns in a given business scenario;

[0102] The large model analysis model is used to input a portion of the dialogue text, the corresponding target focus, and the cue words into the large model. The large model analyzes the key points of each round of dialogue and outputs a result description of the specific category of the dialogue under the target focus. Based on the result description, the labels of each round of dialogue text are standardized to obtain a portion of the labeled dialogue text.

[0103] The encoder training module is used to train the pre-trained encoder in two stages by combining labeled and unlabeled dialogue text. In the first stage of training, data augmentation is performed on the labeled and unlabeled dialogue texts, the cross-entropy loss of the labeled dialogue texts is calculated, and the self-contrast learning loss of all dialogue texts is calculated. In the second stage of training, the self-contrast learning loss is calculated at the beginning of each training iteration, then all dialogue texts are clustered to generate pseudo-labels for the unlabeled dialogue texts, and finally the cross-entropy loss is calculated by mixing the labeled and pseudo-labeled dialogue texts.

[0104] The clustering analysis module is used to input a set of dialogue texts to be analyzed, the corresponding target attention points, and prompt words into the large model. Based on the output of the large model, dialogue texts that do not contain target attention points are removed. The remaining dialogue texts are used to extract features and perform initial clustering using the trained encoder. Based on the initial clustering results, cluster merging and outlier identification are performed to optimize the clustering results and generate a conversation analysis report containing target attention points.

[0105] For the system embodiments, since they basically correspond to the method embodiments, relevant details can be found in the descriptions of the method embodiments; the implementation methods of the remaining modules will not be repeated here. The system embodiments described above are merely illustrative. The units described as separate components may or may not be physically separate, and the components shown as units may or may not be physical units; that is, they may be located in one place or distributed across multiple network units. Some or all of the modules can be selected to achieve the purpose of the present invention according to actual needs. Those skilled in the art can understand and implement this without any creative effort.

[0106] The system embodiments of the present invention can be applied to any device with data processing capabilities, such as a computer or other similar device. The system embodiments can be implemented in software, hardware, or a combination of both. Taking software implementation as an example, as a logical device, it is formed by the processor of any data processing device loading the corresponding computer program instructions from non-volatile memory into memory for execution.

[0107] The above-described embodiments are merely illustrative of several implementations of the present invention, and while the descriptions are specific and detailed, they should not be construed as limiting the scope of the invention. Those skilled in the art can make various modifications and improvements without departing from the concept of the present invention, and these modifications and improvements all fall within the scope of protection of the present invention.

Claims

1. A conversation analysis method based on a large model, characterized in that, include: Given the dialogue text and target concerns in a specific business scenario; Input a portion of the dialogue text, the corresponding target focus, and the prompt words into the large model. The large model analyzes the key points of each round of dialogue and outputs a result description of the specific category of the dialogue under the target focus. Based on the result description, the labels of each round of dialogue text are standardized to obtain a portion of the labeled dialogue text. The pre-trained encoder is trained in two stages by combining labeled and unlabeled dialogue texts. In the first stage, data augmentation is performed on both labeled and unlabeled dialogue texts, and the cross-entropy loss of the labeled dialogue texts and the self-contrast learning loss of all dialogue texts are calculated. In the second stage, the self-contrast learning loss is calculated at the beginning of each training iteration, then all dialogue texts are clustered to generate pseudo-labels for the unlabeled dialogue texts, and finally the cross-entropy loss is calculated by mixing the labeled and pseudo-labeled dialogue texts. Input a set of dialogue texts to be analyzed, the corresponding target attention points, and prompt words into the large model. Based on the output of the large model, dialogue texts that do not contain target attention points are removed. The remaining dialogue texts are used to extract features and perform initial clustering using the trained encoder. Based on the initial clustering results, cluster merging and outlier identification are performed to optimize the clustering results and generate a conversation analysis report containing target attention points.

2. The large-model-based session analysis method according to claim 1, characterized in that, The target focus points refer to the information that the service representative wants to focus on based on the dialogue text between the service representative and the customer in the business scenario, including the reasons for customer satisfaction, the reasons for customer dissatisfaction, the reasons for customer refusal, customer ratings, and the reasons why the customer thinks the service representative's answers are irrelevant. Each focus point corresponds to several categories.

3. The large-model-based session analysis method according to claim 1, characterized in that, The standardization of labels for each round of dialogue text based on the result description refers to converting the result description of the specific category under the target concern output by the large model into category labels under that target concern.

4. The large-model-based session analysis method according to claim 1, characterized in that, The pre-trained encoder mentioned above uses the BERT encoder.

5. The large-model-based session analysis method according to claim 1 or 4, characterized in that, The process of performing the first stage of training on the pre-trained encoder includes: In each training step, a set of labeled dialogue text and a set of unlabeled dialogue text are taken; The labeled and unlabeled dialogue texts are input into the pre-trained encoder twice, one after the other. Each dialogue text corresponds to a pair of features (z, z) output by the pre-trained encoder in the two separate inputs. ′ ), where z is the feature generated by the pre-trained encoder for the first input, z ′ These are the enhanced features generated by the second input encoder; For a set of labeled dialogue texts, calculate the cross-entropy loss before and after enhancement, and take the average as the total cross-entropy loss; The self-contradictory learning loss is calculated by taking the features of each dialogue text and its corresponding augmented features as positive samples, and taking the features of each dialogue text and the augmented features of another dialogue text as negative samples. By combining the total cross-entropy loss and the self-comparative learning loss, and iterating through several training steps, the first stage of training of the pre-trained encoder is completed.

6. The conversation analysis method based on a large model according to claim 5, characterized in that, The process of performing the second phase of training on the pre-trained encoder includes: For each training step, take one set of labeled dialogue text and one set of unlabeled dialogue text, and perform the following three steps in sequence: The first step is to calculate the self-contradictory learning loss using the first-stage method; The second step is to cluster the features of labeled and unlabeled dialogue texts to generate pseudo-labels for the unlabeled data. The third step is to treat the pseudo-labeled dialogue text as labeled data, mix all dialogue texts, and use the first-stage method to calculate the total cross-entropy loss. Combining the self-contradictory learning loss and the total cross-entropy loss, the pre-trained encoder is trained in the second stage by iterating through several training steps.

7. The large-model-based session analysis method according to claim 1, characterized in that, The cluster merging includes: For small sample clusters in the initial clustering results, select the top 5 clusters that are closest to them, and randomly select a number of samples from each of the top 5 clusters. The large model determines whether the small sample cluster belongs to one or more of the top 5 clusters. If it does, merge the small sample cluster into the corresponding cluster; otherwise, do not merge.

8. The conversation analysis method based on a large model according to claim 1, characterized in that, The outlier identification includes: Outlier detection is achieved using the local outlier factor. Half the size of each cluster is selected as the number of neighbors. The outlier factor is standardized using an auxiliary test set as a threshold, and points exceeding the threshold are considered outliers. For each detected outlier, the top 5 clusters closest to it are selected. Several samples are randomly selected from each of the top 5 clusters, and the large model determines whether the outlier belongs to one or more of the top 5 clusters. If it does, the outlier is merged into the corresponding cluster; otherwise, the outlier is removed.

9. The large-model-based session analysis method according to claim 1, characterized in that, The conversation analysis report includes information on points of interest and the distribution of different categories under each point of interest.

10. A conversation analysis system based on a large model, characterized in that, include: The input module is used to input dialogue text and target concerns in a given business scenario; The large model analysis model is used to input a portion of the dialogue text, the corresponding target focus, and the cue words into the large model. The large model analyzes the key points of each round of dialogue and outputs a result description of the specific category of the dialogue under the target focus. Based on the result description, the labels of each round of dialogue text are standardized to obtain a portion of the labeled dialogue text. The encoder training module is used to train the pre-trained encoder in two stages by combining labeled and unlabeled dialogue text. In the first stage of training, data augmentation is performed on the labeled and unlabeled dialogue texts, the cross-entropy loss of the labeled dialogue texts is calculated, and the self-contrast learning loss of all dialogue texts is calculated. In the second stage of training, the self-contrast learning loss is calculated at the beginning of each training iteration, then all dialogue texts are clustered to generate pseudo-labels for the unlabeled dialogue texts, and finally the cross-entropy loss is calculated by mixing the labeled and pseudo-labeled dialogue texts. The clustering analysis module is used to input a set of dialogue texts to be analyzed, the corresponding target attention points, and prompt words into the large model. Based on the output of the large model, dialogue texts that do not contain target attention points are removed. The remaining dialogue texts are used to extract features and perform initial clustering using the trained encoder. Based on the initial clustering results, cluster merging and outlier identification are performed to optimize the clustering results and generate a conversation analysis report containing target attention points.

Images