Session analysis method and system based on large model

Abstract

The invention provides a session analysis method and system based on a large model, and belongs to the technical field of artificial intelligence. Giving a dialogue text and a target concern in a service scene; inputting part of the dialogue text, the corresponding target concern and the cue word into the large model, outputting the result description of the specific category of the round of dialogue under the target concern, and standardizing the label of each round of dialogue text; carrying out two-stage training on the pre-training encoder by combining the tagged dialogue text and the untagged dialogue text; inputting a group of dialogue texts to be analyzed, corresponding target concern points and cue words into the large model, removing dialogue texts without the target concern points according to result description output by the large model, extracting features of the rest dialogue texts by utilizing a trained encoder, performing initial clustering, performing cluster merging and outlier recognition according to an initial clustering result, and performing outlier recognition according to the initial clustering result. And optimizing a clustering result, and generating a session analysis report containing the target concern. According to the invention, the session analysis clustering effect is improved.

Description

Technical Field

[0001] The present invention belongs to the field of artificial intelligence technology, and in particular relates to a conversation analysis method and system based on a large model. Background Art

[0002] With the development of big models, conversation analysis, as a core technology in the field of human-computer interaction, has gradually shown its importance in many fields. In the field of marketing, the conversation texts between human customer service (or AI customer service) and customers contain a variety of useful information. Capturing valuable information in the conversation records and generating intelligent, natural, and structured responses can help improve work efficiency in this business scenario and provide customers with a more intelligent, accurate, and efficient conversation interaction experience.

[0003] Traditional conversation analysis systems are mostly based on predefined rules or traditional machine learning methods based on classification and clustering. When clustering, they are generally clustered by the similarity of the encoding features of the pre-trained model, with K-MEANS and DBSCAN methods being the most common. However, these methods often seem to be unable to cope with complex and changeable natural language conversations, especially in multi-round conversations, fuzzy intent analysis, and contextual semantic understanding. There are significant limitations, which are mainly reflected in the following aspects:

[0004] 1. It is highly dependent on classification models and can only analyze data in existing classification models;

[0005] 2. The clustering is too chaotic and lacks a semantically guided clustering merging method;

[0006] 3. The clustering effect is poor, because the accuracy of the classification model is low and there is a lack of feedback mechanism to feed back the model results;

[0007] 4. Directly using the encoding vectors of the pre-trained model often lacks prior knowledge of the clustering granularity.

[0008] Existing conversation analysis technology has bottlenecks in semantic understanding depth, computational efficiency, and classification accuracy under different focus points, and is unable to accurately cluster natural language conversations under multiple focus points. Summary of the invention

[0009] In view of this, the present invention proposes a conversation analysis method and system based on a large model, and the technical solution adopted is as follows:

[0010] In a first aspect, a conversation analysis method based on a large model includes:

[0011] Conversation text and target focus in a given business scenario;

[0012] Input part of the conversation text, the corresponding target focus points and prompt words into the big model, and let the big model analyze the key points of each round of conversation and output the result description of the specific category of the conversation under the target focus points; standardize the label of each round of conversation text according to the result description to obtain part of the labeled conversation text;

[0013] The pre-trained encoder is trained in two stages by combining labeled and unlabeled conversation texts. In the first stage of training, data augmentation is performed on labeled and unlabeled conversation texts, and the cross entropy loss of labeled conversation texts and the self-contrastive learning loss of all conversation texts are calculated. In the second stage of training, the self-contrastive learning loss is calculated at the beginning of each training iteration, and then all conversation texts are clustered to generate pseudo labels for unlabeled conversation texts. Finally, the cross entropy loss is calculated by mixing labeled conversation texts and pseudo-labeled conversation texts.

[0014] A set of conversation texts to be analyzed, the corresponding target focus points and prompt words are input into the big model. The conversation texts that do not contain the target focus points are eliminated according to the result description output by the big model. The remaining conversation texts are extracted features and initially clustered using the trained encoder. Cluster merging and outlier identification are performed based on the initial clustering results, the clustering results are optimized, and a conversation analysis report containing the target focus points is generated.

[0015] Furthermore, the target focus refers to the information that one wants to focus on based on the conversation text between the service representative and the customer in a business scenario, including reasons for customer satisfaction, reasons for customer dissatisfaction, reasons for customer rejection, customer ratings, and reasons why the customer believes the service representative's answer is irrelevant. Each focus point corresponds to several categories.

[0016] Furthermore, the label of each round of dialogue text is standardized according to the result description, which refers to converting the result description of a specific category under the target focus output by the large model into a category label under the target focus.

[0017] Furthermore, the pre-trained encoder adopts a BERT encoder.

[0018] Furthermore, the process of performing the first stage training on the pre-trained encoder includes:

[0019] Each training step takes a set of labeled dialogue texts and a set of unlabeled dialogue texts;

[0020] The labeled conversation text and the unlabeled conversation text are input into the pre-trained encoder twice respectively. Each conversation text corresponds to a pair of features (z, z) output by the pre-trained encoder twice. ′ ), where z is the feature generated by the first input pre-trained encoder, z ′ It is the enhanced feature generated by the second input encoder;

[0021] For a set of labeled conversation texts, calculate the cross entropy loss before and after enhancement, and take the average as the total cross entropy loss;

[0022] Take the features of each dialogue text and its corresponding enhanced features as positive samples, and the features of each dialogue text and the enhanced features of another dialogue text as negative samples, and calculate the self-contrastive learning loss;

[0023] Combining the total cross entropy loss and self-contrastive learning loss, several training steps are iterated to complete the first stage training of the pre-trained encoder.

[0024] Furthermore, the process of performing the second stage training on the pre-trained encoder includes:

[0025] Each training step takes a set of labeled dialogue texts and a set of unlabeled dialogue texts and performs the following three steps in sequence:

[0026] In the first step, the self-contrastive learning loss is calculated using the first-stage method;

[0027] The second step is to cluster the features of labeled and unlabeled conversation texts to generate pseudo labels for unlabeled data.

[0028] In the third step, the pseudo-labeled dialogue text is regarded as labeled data, and all dialogue texts are mixed and the total cross entropy loss is calculated using the method of the first stage;

[0029] Combining the self-contrastive learning loss and the total cross entropy loss, several training steps are iterated to complete the second stage training of the pre-trained encoder.

[0030] Furthermore, the cluster merging includes:

[0031] For the small sample cluster in the initial clustering results, select the Top5 cluster closest to it, randomly select several samples from each of the Top5 clusters, and use the large model to determine whether the small sample cluster belongs to one or more of the Top5 clusters. If so, merge the small sample cluster into the corresponding cluster, otherwise do not merge it.

[0032] Furthermore, the outlier identification includes:

[0033] The local outlier factor is used to detect outliers, half of the size of each cluster is selected as the number of neighbors, and the auxiliary test set standardized outlier factor is used as the threshold. Points greater than the threshold are regarded as outliers. For the detected outliers, the Top5 clusters closest to the outliers are selected, and several samples are randomly selected from each of the Top5 clusters. The large model determines whether the outlier belongs to one or more of the Top5 clusters. If so, the outlier is merged into the corresponding cluster, otherwise the outlier is removed.

[0034] Furthermore, the conversation analysis report includes focus point information and the distribution of different categories under each focus point.

[0035] In a second aspect, the present invention provides a big model-based conversation analysis system for implementing the above-mentioned big model-based conversation analysis method.

[0036] The beneficial effects of the present invention are:

[0037] The present invention assists in analyzing the focus points in the conversation based on the structured output capability of the large model and the natural language understanding capability. Based on the output results of the large model, standardization, preliminary clustering, cluster merging and outlier identification are used to obtain optimized clustering results, which further guide the generation of conversation analysis reports containing target focus points. In addition, the text features used for clustering are generated by the encoder obtained through two-stage training, which improves the final clustering effect in many aspects and generates a more accurate conversation analysis report. BRIEF DESCRIPTION OF THE DRAWINGS

[0038] The accompanying drawings are used to provide a further understanding of the present invention and constitute a part of the specification. Together with the embodiments of the present invention, they are used to explain the present invention and do not constitute a limitation of the present invention. In the accompanying drawings:

[0039] Figure 1 This is a schematic diagram of the framework of the conversation analysis method based on the big model;

[0040] Figure 2 This is a schematic diagram of the first stage of training;

[0041] Figure 3 Schematic diagram of the second stage training. DETAILED DESCRIPTION

[0042] The present invention is further described and illustrated below in conjunction with specific embodiments. The embodiments are merely exemplary of the present disclosure and do not define the scope of limitation. The technical features of each embodiment of the present invention may be combined accordingly without conflicting with each other.

[0043] The accompanying drawings are only schematic illustrations of the present invention and are not necessarily drawn to scale. Some of the block diagrams shown in the accompanying drawings are functional entities that do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or in one or more hardware modules or integrated circuits, or in different networks and / or processor devices and / or microcontroller devices.

[0044] The flowcharts shown in the accompanying drawings are only exemplary and do not necessarily include all the steps. For example, some steps may be decomposed, while some steps may be combined or partially combined, so the actual execution order may change according to the actual situation.

[0045] like Figure 1 As shown, the present invention proposes a conversation analysis method and system based on a large model, and the main implementation process is as follows:

[0046] S1, dialogue text and target focus under given business scenarios.

[0047] S2, input part of the dialogue text, the corresponding target focus points and prompt words into the big model, and the big model analyzes the key points of each round of dialogue and outputs the result description of the specific category of the dialogue under the target focus point; standardize the label of each round of dialogue text according to the result description.

[0048] S3, combines labeled conversation text and unlabeled conversation text to perform two-stage training on the pre-trained encoder. In the first stage of training, data augmentation is performed on the labeled data, the cross entropy loss is calculated, and all data are mixed to calculate the self-contrastive learning loss; in the second stage of training, at the beginning of each training iteration, all data are mixed to calculate the self-contrastive learning loss, then all data are clustered to generate pseudo-labels for unlabeled data, and finally the labeled data and pseudo-labeled data are mixed to calculate the cross entropy loss.

[0049] S4, input a group of conversation texts to be analyzed, the corresponding target focus points and prompt words into the large model, and remove the texts that do not contain the target focus points according to the result description output by the large model. The remaining texts are extracted and clustered using the encoder trained in step S3, and cluster merging and outlier point identification are performed according to the initial clustering results, the clustering results are optimized, and a conversation analysis report containing the target focus points is generated.

[0050] In the above step S1, each round of dialogue text and target focus are written in json format, for example, {"text":text,entity:[{"value":value,"description":description}]}, where "text":text represents a round of dialogue text, "value":value represents the target focus, and "description":description represents the natural language description of the target focus.

[0051] The conversation text is a set of conversations between a human customer service (or AI customer service) and a customer in a certain business scenario. In this set of conversations, the focus may be the reasons for customer satisfaction, customer dissatisfaction, customer rejection, reasons why the customer thinks the customer service did not answer the question, etc. Each focus may be divided into multiple categories. For example, the reasons for customer satisfaction may be divided into good customer service, good product brand, good product quality, strong product activities, etc.

[0052] The conversation text can be extracted directly from the text conversation, or it can be generated in text format after recognizing the text in the voice call.

[0053] The natural language description of the target focus is to guide the large model to obtain better analysis results. For example, if a target focus is "reasons for customer satisfaction", the corresponding natural language description can be "what are the reasons why customers expressed satisfaction in this round of conversation". The natural language description is a further explanation of the target focus and can be preset based on the experience of technicians in this field.

[0054] In the above step S2, the big model is used to analyze the key points in the dialogue text, and the big model is required to output the analysis results under the target focus point based on the input dialogue text. The analysis results here are the result descriptions of the specific categories of the dialogue text under the target focus point.

[0055] For example, for a target focus of "reasons for customer satisfaction", based on a round of input dialogue text, the output result of the large model may be "this campaign is very powerful", and based on this output result, the dialogue text can be attributed to the label "product campaign is powerful". It can be found that there are both similarities and differences between the output of the large model and the label, and the two are not aligned. By classifying the output results of the large model, the standardized labels of each round of dialogue text input into the large model can be obtained.

[0056] In this step, the number of target focus points is not limited, and can be one or more.

[0057] In this embodiment, the prompt words for large models are generally benchmarked to obtain the best prompt words for each model. For large models that support structured output and tool calls, their structured output function is used directly. For large models that do not support structured output, the output requirements are indicated in the prompt words.

[0058] The prompt words to guide the large model to output results can be:

[0059] ##Context##

[0060] In telemarketing and customer service scenarios, we often need to determine the customer's true intention based on the conversation between the customer and the service representative, as well as extract the key information provided from the conversation.

[0061] ##Target##

[0062] You are an expert in understanding conversations between customers and service representatives. Your task is to {{analyze and describe the specific categories under the concerns from the conversations}}. You need to obtain {% conversation text list %} {{concerns}} ({{natural language description of concerns}}). {% endfor %}};

[0063] The output result of the large model is also a JSON list, which is not clustered and aligned.

[0064] In the above step S3, the pre-trained encoder is preferably a Bert encoder, and the Bert encoder is trained in two stages to optimize its further clustering capability.

[0065] like Figure 2 As shown in Figure 1, in the first stage of training, the text data is divided into two parts, labeled and unlabeled. The labels of the labeled data are generated after standardization in step S2. In each training step of this stage, a labeled batch and an unlabeled batch are taken. For the labeled batch and the unlabeled batch, the same conversation text is input into the Bert encoder twice. Due to the dropout function built into the Bert encoder, the Bert encoding features z and z obtained twice are ′ There is a slight difference between the two. In order to reduce this difference, the encoding feature z ′ It can be regarded as an enhancement of the encoded feature z, and the cross entropy loss is calculated by this method of data enhancement for labeled data:

[0066]

[0067] in, Represent the cross entropy loss of original data and enhanced data respectively; represents the total cross entropy loss, which is the average of the two cross entropy losses; K represents the total number of categories under all attention points, y a Indicates the label, represents the label of the i-th sample, (z i ) j represents the encoding feature of the i-th sample under the j-th category, and N represents the number of samples in the batch.

[0068] In addition, labeled data and unlabeled data are mixed to calculate the self-contrastive learning loss;

[0069] In this embodiment, the features of each dialogue text and its corresponding enhanced features are used as positive samples, and the features of each dialogue text and the enhanced features of another dialogue text are used as negative samples to calculate the self-contrastive learning loss.

[0070] Combining the above two losses, the first stage of training is completed.

[0071] like Figure 3 As shown, in the second stage of training, in each training iteration, a set of labeled dialogue texts and a set of unlabeled dialogue texts are taken, and the following three steps are performed in sequence:

[0072] In the first step, all data are mixed to calculate the self-contrastive learning loss, using the same self-contrastive learning method as the first stage.

[0073] In the second step, all data are clustered to generate pseudo labels for unlabeled data;

[0074] Before clustering, we first use the Bert encoder to obtain the encoding features of the samples, and perform k-means clustering on the generated encoding features. Some samples participating in clustering have labels, and this part of the label information is used to perform Hungarian algorithm alignment, and each cluster is assigned a label. Based on this, each unlabeled data obtains a cluster label, which is used as a pseudo-label for the unlabeled data.

[0075] In the third step, the cross entropy loss is calculated by mixing the labeled data and pseudo-labeled data to complete further supervised training.

[0076] Repeat several iterations to complete the second stage of training.

[0077] In the training process of the present invention, self-supervised learning and semi-supervised learning are combined to improve clustering performance, wherein self-supervised learning learns discriminative category representations after each clustering to promote subsequent clustering. A dual learning strategy is adopted, and training is performed at the instance level and cluster level respectively. In instance-level learning, by ensuring that similar instances are classified into the same class and distinguished from instances of other classes, the intra-class compactness and inter-class separability are enhanced, especially under strong data enhancement. In cluster-level learning, model parameters are updated based on cluster assignment to improve the model's ability to distinguish categories. In the above learning, the present invention combines cross entropy loss and contrast loss to obtain encoding features that are conducive to clustering. The present invention effectively identifies different instances through the first stage of training, generates consistent instance-level pseudo-labels using clustering data through the second stage, and clearly distinguishes different categories by using specific pseudo-labels as self-supervisory signals, thereby improving clustering performance and accelerating convergence.

[0078] In semi-supervised learning, the first stage performs data augmentation on the labeled data and mixes them in contrastive learning to make full use of the limited labeled data to initialize the cluster representation. A semi-supervised contrastive loss is proposed, which includes positive sample pairs (each sample feature and its corresponding enhanced sample feature) and negative sample pairs (different sample features). In order to enhance the ability to distinguish known categories, a cross entropy loss is added for the supervisory signal of labeled data.

[0079] The present invention proposes a clustering and representation learning. After the first stage of training, all data are directly used for clustering and representation learning. It is possible that labeled samples of the same category are assigned to different clusters. This phenomenon is called catastrophic forgetting. In order to alleviate this problem, at the beginning of each training iteration, semi-supervised contrastive learning is first used to enhance the memory of known categories by enhancing samples to guide the subsequent clustering process. Then centroid-guided clustering and self-supervised learning are continued.

[0080] In the above step S4, if Figure 1 As shown, a group of dialogue texts to be analyzed, the corresponding target focus points and prompt words are input into the large model, and the texts without target focus points are eliminated according to the result description output by the large model. The remaining texts are extracted and clustered using the encoder trained in step S3.

[0081] In this step, the clustering results are further improved using post-processing methods of outlier detection and cluster merging.

[0082] (1) Cluster merging

[0083] To determine whether the initial clustering results contain a small sample cluster, you can set a threshold for the number of samples within the cluster. For a small sample cluster, select the Top5 clusters closest to it, randomly select several samples from each of the Top5 clusters, and use the large model to determine whether the small sample cluster belongs to one or more of the Top5 clusters. If so, merge the small sample cluster into the corresponding cluster, otherwise do not merge it.

[0084] When the big model makes a judgment, the samples in the small sample cluster and the selected samples from each of the Top5 clusters are input into the big model, and a prompt word is input to guide the big model to judge whether the small sample cluster is of the same category as the selected samples from each cluster. If the big model judges that it is, it will be merged into the cluster corresponding to the samples. Here, the big model may judge that the small sample cluster belongs to multiple of the Top5 clusters. At this time, the small sample cluster is merged into multiple clusters at the same time, indicating that the small sample cluster contains different categories under the same focus.

[0085] (2) Outlier Identification

[0086] The present invention uses the local outlier factor (LOF) to detect outliers, which is a density-based unsupervised machine learning algorithm. Before outlier detection, half of each cluster size is selected as the number of neighbors, and the test set standardized outlier factor is used as the threshold, and points greater than the threshold are considered outliers. The present invention uses a dynamic neighbor size and global threshold solution to solve the problem of low accuracy of fixed outlier factor and neighbor size in the LOF algorithm.

[0087] The present invention further optimizes the clustering results by performing cluster merging and outlier identification on the initial clustering results, and generates a conversation analysis report containing target focus points based on the optimized clustering results.

[0088] In a specific implementation of the present invention, for the detected outliers, the Top5 clusters closest to the outliers are selected, and several samples are randomly selected from each of the Top5 clusters. The large model determines whether the outlier belongs to one or more of the Top5 clusters. If so, the outlier is merged into the corresponding cluster, otherwise the outlier is removed.

[0089] When the big model makes a judgment, the outlier and several samples selected from each of the Top5 clusters are input into the big model, and a prompt word is input to guide the big model to judge whether the outlier is of the same category as the several samples selected from each cluster. If the big model judges that it is, it will be merged into the cluster corresponding to the several samples. Here, the big model may judge that the outlier belongs to multiple of the Top5 clusters. At this time, the outlier is merged into multiple clusters at the same time, indicating that the outlier contains different categories under the same focus point.

[0090] A conversation analysis report is generated based on the optimized clustering results. The conversation analysis report contains focus information and the distribution of different categories under each focus point. This is objective data generated by analyzing the focus of the conversation. The conversation analysis results can be used to guide manual customer service work or configure AI customer service scripts, configure product activity intensity, etc. to improve customer satisfaction, help improve work efficiency in this business scenario, and provide customers with a more intelligent, accurate and efficient conversation interaction experience.

[0091] In order to verify the effect of the present invention, the conversation texts between customer service and customers in a certain period of time are analyzed to analyze the distribution of each category under multiple focus points such as customer ratings, reasons for customer dissatisfaction, reasons for customer satisfaction, etc. A total of 54,032 training data are used for testing, with a total of 263 categories.

[0092] This embodiment tests the effects of pre-trained encoder, pre-trained encoder + first stage training, pre-trained encoder + two-stage training, pre-trained encoder + two-stage training + outlier recognition, pre-trained encoder + two-stage training + cluster merging, pre-trained encoder + two-stage training + cluster merging + outlier recognition, pre-trained encoder + DBSCAN clustering model, and pre-trained encoder + HDBSCAN clustering model. Taking NMI, ACC, and SIL as evaluation indicators, the test results are shown in Table 1.

[0093] Model Name NMI ACC SIL bge-base-zh 82.4 60.66 9.3439 bge-base-zh-pretrained 88.67 75.76 13.3848 bge-base-zh-tuned 90.64 78.86 29.9871 bge-base-zh-tuned+clustering merge 92.52 80.81 None bge-base-zh-tuned+outlier identification 93.71 82.69 None bge-base-zh-tuned+clustering merging+outlier identification (this invention) 95.15 88.35 None bge-base-zh+DBSCAN 41.11 35.88 4.7565 bge-base-zh+HDBSCAN 44.09 38.92 5.0814

[0094] In Table 1 above, bge-base-zh represents the pre-trained encoder, pretrained represents the first stage of training, and tuned represents the two-stage training. It can be seen that the two-stage training each brings different degrees of improvement to the model performance.

[0095] Regarding the indicators used:

[0096] 1.NMI (Normalized Mutual Information) is a metric used to measure the similarity between two partitions (for example, the clustering result obtained by the algorithm and the true label). It is based on the concept of information entropy and measures the amount of information one partition contains about another partition.

[0097] 2. SIL (Silhouette Coefficient) is one of the commonly used indicators to measure the effectiveness of unsupervised clustering. It focuses on the fact that data points in the same cluster should be as close as possible to each other, and different clusters should be as far apart as possible. This evaluation indicator is not applicable to the solution of performing post-processing optimization on the initial clustering.

[0098] 3. ACC (accuracy), in clustering tasks, the overlap between clustering results and true categories is measured by the "optimal mapping" method. That is, first try to match a true label for each cluster label so that the classification accuracy is the highest after matching, and then calculate the accuracy.

[0099] As can be seen from Table 1, the present invention has the best effect.

[0100] In this embodiment, a conversation analysis system based on a large model for implementing the above analysis method is also provided, including:

[0101] Input module, which is used for dialogue text and target focus in a given business scenario;

[0102] The big model analysis model is used to input part of the dialogue text, the corresponding target focus points and prompt words into the big model, and the big model analyzes the key points of each round of dialogue, and outputs the result description of the specific category of the dialogue under the target focus point; the label of each round of dialogue text is standardized according to the result description to obtain part of the labeled dialogue text;

[0103] The encoder training module is used to perform two-stage training on the pre-trained encoder by combining the labeled dialogue text and the unlabeled dialogue text. In the first stage of training, data enhancement is performed on the labeled dialogue text and the unlabeled dialogue text, and the cross entropy loss of the labeled dialogue text and the self-contrastive learning loss of all the dialogue texts are calculated. In the second stage of training, the self-contrastive learning loss is calculated at the beginning of each training iteration, and then all the dialogue texts are clustered to generate pseudo labels for the unlabeled dialogue text. Finally, the cross entropy loss is calculated by mixing the labeled dialogue text and the pseudo-labeled dialogue text.

[0104] The clustering analysis module is used to input a set of dialogue texts to be analyzed, the corresponding target focus points and prompt words into the large model, and remove the dialogue texts that do not contain the target focus points according to the result description output by the large model. The remaining dialogue texts are extracted features and initially clustered using the trained encoder, and cluster merging and outlier point identification are performed based on the initial clustering results, the clustering results are optimized, and a conversation analysis report containing the target focus points is generated.

[0105] For the system embodiment, since it basically corresponds to the method embodiment, the relevant parts can refer to the partial description of the method embodiment, and the implementation methods of the remaining modules will not be repeated here. The system embodiment described above is only schematic, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of the present invention. Ordinary technicians in this field can understand and implement it without paying creative work.

[0106] The embodiments of the system of the present invention can be applied to any device with data processing capabilities, and the device with data processing capabilities can be a device or apparatus such as a computer. The system embodiments can be implemented by software, or by hardware or a combination of software and hardware. Taking software implementation as an example, as a device in a logical sense, the corresponding computer program instructions in the non-volatile memory are read into the memory by the processor of any device with data processing capabilities and run.

[0107] The above-mentioned embodiments only express several implementation modes of the present invention, and the description is relatively specific and detailed, but it cannot be understood as limiting the scope of the present invention. For those of ordinary skill in the art, several modifications and improvements can be made without departing from the concept of the present invention, which all belong to the protection scope of the present invention.

Claims

1. A conversation analysis method based on a large model, characterized in that: include: Conversation text and target focus in a given business scenario; Input part of the conversation text, the corresponding target focus points and prompt words into the big model, and let the big model analyze the key points of each round of conversation and output the result description of the specific category of the conversation under the target focus points; standardize the label of each round of conversation text according to the result description to obtain part of the labeled conversation text; The pre-trained encoder is trained in two stages by combining labeled and unlabeled conversation texts. In the first stage of training, data augmentation is performed on labeled and unlabeled conversation texts, and the cross entropy loss of labeled conversation texts and the self-contrastive learning loss of all conversation texts are calculated. In the second stage of training, the self-contrastive learning loss is calculated at the beginning of each training iteration, and then all conversation texts are clustered to generate pseudo labels for unlabeled conversation texts. Finally, the cross entropy loss is calculated by mixing labeled conversation texts and pseudo-labeled conversation texts. A set of conversation texts to be analyzed, the corresponding target focus points and prompt words are input into the big model. The conversation texts that do not contain the target focus points are eliminated according to the result description output by the big model. The remaining conversation texts are extracted features and initially clustered using the trained encoder. Cluster merging and outlier identification are performed based on the initial clustering results, the clustering results are optimized, and a conversation analysis report containing the target focus points is generated.

2. The conversation analysis method based on a large model according to claim 1, characterized in that: The target focus refers to the information that you want to focus on based on the conversation text between the service representative and the customer in the business scenario, including the reasons for customer satisfaction, customer dissatisfaction, customer rejection, customer ratings, and reasons why the customer thinks the service representative's answer is irrelevant. Each focus corresponds to several categories.

3. The conversation analysis method based on a large model according to claim 1, characterized in that: The standardization of the labels of each round of dialogue text according to the result description refers to converting the result description of a specific category under the target focus output by the large model into the category label under the target focus.

4. The conversation analysis method based on a large model according to claim 1, characterized in that: The pre-trained encoder adopts a BERT encoder.

5. The conversation analysis method based on a large model according to claim 1 or 4, characterized in that: The first stage of training a pre-trained encoder involves: Each training step takes a set of labeled dialogue texts and a set of unlabeled dialogue texts; The labeled conversation text and the unlabeled conversation text are input into the pre-trained encoder twice respectively. Each conversation text corresponds to a pair of features (z, z) output by the pre-trained encoder twice. ′ ), where z is the feature generated by the first input pre-trained encoder, z ′ It is the enhanced feature generated by the second input encoder; For a set of labeled conversation texts, calculate the cross entropy loss before and after enhancement, and take the average as the total cross entropy loss; Take the features of each dialogue text and its corresponding enhanced features as positive samples, and the features of each dialogue text and the enhanced features of another dialogue text as negative samples, and calculate the self-contrastive learning loss; Combining the total cross entropy loss and self-contrastive learning loss, several training steps are iterated to complete the first stage training of the pre-trained encoder.

6. The conversation analysis method based on a large model according to claim 5 is characterized in that: The process of performing the second phase of training on the pre-trained encoder includes: Each training step takes a set of labeled dialogue texts and a set of unlabeled dialogue texts, and performs the following three steps in sequence: In the first step, the self-contrastive learning loss is calculated using the first-stage method; The second step is to cluster the features of labeled and unlabeled conversation texts to generate pseudo labels for unlabeled data. In the third step, the pseudo-labeled dialogue text is regarded as labeled data, and all dialogue texts are mixed and the total cross entropy loss is calculated using the method of the first stage; Combining the self-contrastive learning loss and the total cross entropy loss, several training steps are iterated to complete the second stage training of the pre-trained encoder.

7. The conversation analysis method based on a large model according to claim 1 is characterized in that: The cluster merging comprises: For the small sample cluster in the initial clustering results, select the Top5 cluster closest to it, randomly select several samples from each of the Top5 clusters, and use the large model to determine whether the small sample cluster belongs to one or more of the Top5 clusters. If so, merge the small sample cluster into the corresponding cluster, otherwise do not merge it.

8. The conversation analysis method based on a large model according to claim 1, characterized in that: The outlier identification includes: The local outlier factor is used to detect outliers, half of the size of each cluster is selected as the number of neighbors, and the auxiliary test set standardized outlier factor is used as the threshold. Points greater than the threshold are regarded as outliers. For the detected outliers, the Top5 clusters closest to the outliers are selected, and several samples are randomly selected from each of the Top5 clusters. The large model determines whether the outlier belongs to one or more of the Top5 clusters. If so, the outlier is merged into the corresponding cluster, otherwise the outlier is removed.

9. The conversation analysis method based on a large model according to claim 1, characterized in that: The conversation analysis report includes focus point information and the distribution of different categories under each focus point.

10. A conversation analysis system based on a large model, characterized in that: include: Input module, which is used for dialogue text and target focus in a given business scenario; The big model analysis model is used to input part of the dialogue text, the corresponding target focus points and prompt words into the big model, and the big model analyzes the key points of each round of dialogue, and outputs the result description of the specific category of the dialogue under the target focus point; the label of each round of dialogue text is standardized according to the result description to obtain part of the labeled dialogue text; The encoder training module is used to perform two-stage training on the pre-trained encoder by combining the labeled dialogue text and the unlabeled dialogue text. In the first stage of training, data enhancement is performed on the labeled dialogue text and the unlabeled dialogue text, and the cross entropy loss of the labeled dialogue text and the self-contrastive learning loss of all the dialogue texts are calculated. In the second stage of training, the self-contrastive learning loss is calculated at the beginning of each training iteration, and then all the dialogue texts are clustered to generate pseudo labels for the unlabeled dialogue text. Finally, the cross entropy loss is calculated by mixing the labeled dialogue text and the pseudo-labeled dialogue text. The clustering analysis module is used to input a set of dialogue texts to be analyzed, the corresponding target focus points and prompt words into the large model, and remove the dialogue texts that do not contain the target focus points according to the result description output by the large model. The remaining dialogue texts are extracted features and initially clustered using the trained encoder, and cluster merging and outlier point identification are performed based on the initial clustering results, the clustering results are optimized, and a conversation analysis report containing the target focus points is generated.

Images