A method, apparatus, device, medium and product for attributing session satisfaction

Abstract

This invention discloses an attribution method, apparatus, device, medium, and product for session satisfaction. The method includes: determining the session satisfaction of the session data to be tested based on the session data to be tested and a pre-trained session satisfaction model; determining the influence weight of each sentence in the session data to be tested on the session satisfaction based on the session satisfaction model; filtering the session data to be tested based on the influence weight of each sentence to obtain at least two filtered sentences; determining a target satisfaction factor for each filtered sentence based on the at least two filtered sentences and a pre-trained satisfaction factor model; and determining the attribution data corresponding to the session data to be tested based on the influence weight of the at least two filtered sentences and the target satisfaction factor. This method solves the problem of unclear guidance direction of attribution data leading to scattered optimization resources and improves the optimization efficiency of session service quality.

Description

Technical Field

[0001] This invention relates to the field of Internet technology, and in particular to a method, apparatus, device, medium and product for attributing conversation satisfaction. Background Technology

[0002] A conversational system is a software system that enables users to communicate with human customer service representatives or with intelligent customer service representatives through a specific medium. Conversational satisfaction refers to the subjective evaluation of the user's overall satisfaction with the conversation throughout the entire communication process.

[0003] Attribution of conversation satisfaction aims to analyze and extract sentences from conversation content that lead to user satisfaction or dissatisfaction. The category label corresponding to these sentences is called the satisfaction factor.

[0004] In the process of realizing this invention, at least the following technical problems were found in the prior art:

[0005] Satisfaction factors can be used to guide the optimization of session service quality. Current attribution methods usually extract multiple satisfaction factors for a single session, which leads to unclear guidance and scattered optimization resources, resulting in low efficiency in optimizing session service quality. Summary of the Invention

[0006] This invention provides an attribution method, apparatus, device, medium, and product for session satisfaction, to address the problem of scattered optimization resources caused by unclear guidance from attribution data, and to improve the optimization efficiency of session service quality.

[0007] According to one embodiment of the present invention, an attribution method for conversation satisfaction is provided, the method comprising:

[0008] The session satisfaction level of the session data to be tested is determined based on the session data to be tested and the pre-trained session satisfaction model.

[0009] Based on the conversation satisfaction model, determine the influence weight of each sentence in the conversation data to be tested on the conversation satisfaction.

[0010] Based on the influence weight of each sentence to be tested, the test session data is filtered to obtain at least two filtered sentences;

[0011] Based on the at least two screening sentences and the pre-trained satisfaction factor model, determine the target satisfaction factor for each screening sentence;

[0012] Based on the influence weights of the at least two selected sentences and the target satisfaction factor, the attribution data corresponding to the test session data is determined.

[0013] According to another embodiment of the present invention, an attribution device for conversation satisfaction is provided, the device comprising:

[0014] The conversation satisfaction determination module is used to determine the conversation satisfaction of the conversation data to be tested based on the conversation data to be tested and the pre-trained conversation satisfaction model.

[0015] The influence weight determination module is used to determine the influence weight of each sentence in the test conversation data on the conversation satisfaction based on the conversation satisfaction model.

[0016] The test conversation data filtering module is used to filter the test conversation data according to the influence weight of each test sentence to obtain at least two filtered sentences;

[0017] The target satisfaction factor determination module is used to determine the target satisfaction factor for each screening sentence based on the at least two screening sentences and the pre-trained satisfaction factor model.

[0018] The attribution data determination module is used to determine the attribution data corresponding to the test session data based on the influence weights of the at least two selected sentences and the target satisfaction factor.

[0019] According to another embodiment of the present invention, an electronic device is provided, the electronic device comprising:

[0020] At least one processor; and

[0021] A memory communicatively connected to the at least one processor; wherein,

[0022] The memory stores a computer program that can be executed by the at least one processor, the computer program being executed by the at least one processor to enable the at least one processor to perform the session satisfaction attribution method according to any embodiment of the present invention.

[0023] According to another embodiment of the present invention, a computer-readable storage medium is provided, the computer-readable storage medium storing computer instructions for causing a processor to execute and implement the session satisfaction attribution method described in any embodiment of the present invention.

[0024] According to another embodiment of the present invention, a computer program product is provided, including a computer program that, when executed by a processor, implements the session satisfaction attribution method described in any embodiment of the present invention.

[0025] The technical solution of this invention combines with the task of evaluating conversation satisfaction. While determining conversation satisfaction through a conversation satisfaction model, it also determines the influence weight of each sentence in the conversation data to be tested on conversation satisfaction based on the conversation satisfaction model. The influence weight is used as the screening criterion for the sentences to be tested, and the influence weight is used to represent the importance of the satisfaction factor corresponding to the screened sentences. This solves the problem of unclear guidance direction of attribution data leading to the dispersion of optimization resources and improves the optimization efficiency of conversation service quality.

[0026] It should be understood that the description in this section is not intended to identify key or essential features of the embodiments of the present invention, nor is it intended to limit the scope of the invention. Other features of the invention will become readily apparent from the following description. Attached Figure Description

[0027] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0028] Figure 1 A flowchart illustrating an attribution method for conversation satisfaction provided in an embodiment of the present invention;

[0029] Figure 2 A flowchart illustrating another attribution method for session satisfaction provided in an embodiment of the present invention;

[0030] Figure 3 A model architecture diagram of a specific example of a conversation satisfaction model provided in an embodiment of the present invention;

[0031] Figure 4 This is a schematic diagram of the structure of an attribution device for conversation satisfaction provided in an embodiment of the present invention;

[0032] Figure 5 This is a schematic diagram of the structure of an electronic device provided in one embodiment of the present invention. Detailed Implementation

[0033] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.

[0034] It should be noted that the terms "to be tested," "test," "reference," "target," etc., used in the specification, claims, and accompanying drawings of this invention are used to distinguish similar objects and are not necessarily used to describe a specific order or sequence. It should be understood that such data can be interchanged where appropriate so that the embodiments of the invention described herein can be implemented in orders other than those illustrated or described herein. Furthermore, the terms "comprising" and "having," and any variations thereof, are intended to cover non-exclusive inclusion; for example, a process, method, system, product, or apparatus that comprises a series of steps or units is not necessarily limited to those steps or units explicitly listed, but may include other steps or units not explicitly listed or inherent to such processes, methods, products, or apparatus.

[0035] Figure 1 This is a flowchart of a conversation satisfaction attribution method provided in an embodiment of the present invention. This embodiment is applicable to the situation of attributing the satisfaction of a conversation. The method can be executed by a conversation satisfaction attribution device, which can be implemented in hardware and / or software and can be configured in a terminal device.

[0036] like Figure 1 As shown, the method includes:

[0037] S110. Determine the session satisfaction level of the session data to be tested based on the session data to be tested and the pre-trained session satisfaction model.

[0038] Specifically, the test session data represents the data set generated by two or more participants completing a full communication process through a certain communication method. For example, the communication method includes, but is not limited to, chat applications or email, and the participants can be a user and a human customer service representative, a user and an intelligent customer service representative, a user, a human customer service representative and an intelligent customer service representative, or a user and two human customer service representatives, etc. There are no restrictions on the communication method or participants corresponding to the test session data here.

[0039] In this embodiment, the conversation data to be tested includes at least two sentences to be tested. Specifically, a sentence, as the basic linguistic unit constituting conversation text, represents a relatively complete semantic unit expressing an idea, question, request, answer, etc., by the participants in the conversation throughout the communication process. For example, the sentences to be tested can be marked with punctuation marks such as periods, question marks, and exclamation marks.

[0040] In one specific embodiment, the test session data also includes a timestamp for each sentence to be tested and / or identity information data corresponding to the session participants. The identity information data may include user account information, session device information, geolocation information, and social attribute information, etc.

[0041] Specifically, the conversation satisfaction model is a model used to measure the user's satisfaction with the entire communication process. For example, the conversation satisfaction model can be a traditional machine learning model, a recurrent neural network, a long short-term memory network, or a Transformer network. This embodiment does not limit the conversation satisfaction model.

[0042] In one specific embodiment, the method further includes: inputting training session data into an untrained session satisfaction model to obtain the output predicted satisfaction; determining a satisfaction loss function based on the predicted satisfaction and the actual satisfaction of the training session data; adjusting the model parameters of the untrained session satisfaction model according to the satisfaction loss function until the training termination condition is met, and then using the session satisfaction model in the current iteration as the trained session satisfaction model.

[0043] For example, the actual satisfaction level can come from user feedback questionnaires, and the satisfaction loss function can be the squared loss function, logarithmic loss function, exponential loss function, mean squared error loss function, logistic regression loss function, Huber loss function, or cross-entropy loss function, etc. The training termination condition can be the convergence of the satisfaction loss function and / or the number of iterations reaching a threshold, but it is not limited to the above example.

[0044] For example, conversation satisfaction can be a continuous numerical satisfaction probability or a discrete numerical classification result. For instance, conversation satisfaction can be satisfied or dissatisfied, or dissatisfied, moderately satisfied, and very satisfied. There is no limit to the number of conversation satisfaction categories here, and the specific settings can be customized according to actual needs.

[0045] S120. Based on the conversation satisfaction model, determine the influence weight of each sentence in the conversation data on conversation satisfaction.

[0046] In one specific embodiment, the influence weight of each sentence to be tested on the conversation satisfaction in the conversation data to be tested is determined according to the conversation satisfaction model, including: obtaining the influence weight of each sentence to be tested on the conversation satisfaction in the conversation data to be tested from the model parameter data of the conversation satisfaction model; wherein, the conversation satisfaction model is a deep learning model based on the attention mechanism, a deep learning model combined with an attention module, or a pre-trained language model.

[0047] For example, deep learning models based on attention mechanisms can be Transformer networks or Hierarchical Attention Networks (HAN), deep learning models that combine attention modules can be convolutional neural networks or recurrent neural networks that combine attention modules, and pre-trained language models can be BERT pre-trained language models or GTP pre-trained language models.

[0048] Specifically, since the classification prediction of the above conversation satisfaction model depends on the influence weight of each sentence in the conversation data, the conversation satisfaction model can not only classify the conversation satisfaction, but also obtain the influence weight of each sentence on the conversation satisfaction from the model parameter data of the conversation satisfaction model.

[0049] In another specific embodiment, the influence weight of each sentence to be tested on the session satisfaction in the session data to be tested is determined according to the session satisfaction model, including: using a model interpretable algorithm to determine the influence weight of each sentence to be tested on the session satisfaction in the session data to be tested according to the session satisfaction model.

[0050] For example, model interpretability algorithms can be LIME (Local Interpretable Model-Agnostic Explanations) or SHAP (SHapley Additive exPlanations).

[0051] The core idea of ​​the LIME algorithm is to approximate a conversation satisfaction model within a local scope using an interpretable model (such as a simple model like linear regression or decision trees). Specifically, for each test sentence, perturbation sentence samples are generated, such as by randomly deleting, replacing, or inserting words into the test sentence. Then, the conversation satisfaction model is used to predict the conversation satisfaction of these perturbation sentence samples. Based on the perturbation sentence samples and their corresponding prediction results, an interpretable model is selected for fitting or training to obtain a locally interpretable model. For example, for a linear regression locally interpretable model, feature coefficients can be used to reflect the weight of the test sentence's influence on conversation satisfaction. For a decision tree locally interpretable model, the position of the test sentence in the decision path and whether it belongs to a split condition can be used to reflect the weight of the test sentence's influence on conversation satisfaction.

[0052] The core idea of ​​the SHAP algorithm is based on the concept of Shapley value in game theory. In a game scenario, the Shapley value is used to allocate the payoff of a coalition (a group of multiple participants). Applying this concept to the interpretation of the conversation satisfaction model, the output of the model is considered the "payoff," and the sentence being tested is considered a "participant." The SHAP algorithm enumerates all possible sentence combinations (coalitions) and measures the marginal contribution of adding or removing a sentence from each combination to the output of the conversation satisfaction model. The Shapley value of the sentence being tested is a weighted average of its marginal contributions across all possible coalition orders. In this embodiment, the Shapley value of the sentence being tested is used as the weight of its influence on conversation satisfaction.

[0053] S130. Based on the influence weight of each sentence to be tested, filter the conversation data to be tested to obtain at least two filtered sentences.

[0054] Specifically, the sentences selected are those with higher influence weights in the test conversation data.

[0055] In one specific embodiment, at least two filter sentences are obtained by filtering the test session data according to the influence weight of each sentence to be tested, including: sorting the sentences to be tested in the test session data according to the influence weight of each sentence to be tested to obtain a sentence sorting result; and determining at least two filter sentences according to the sentence sorting result and preset filtering conditions.

[0056] For example, when the sorting algorithm is ascending order, the preset filtering condition is to select a preset number or preset proportion of sentences from right to left; when the sorting algorithm is descending order, the preset filtering condition is to select a preset number or preset proportion of sentences from left to right. The preset number can be 5 sentences, and the preset proportion can be 20%. There are no limitations on the preset number or proportion; they can be customized according to actual needs.

[0057] In another specific embodiment, based on the influence weight of each sentence to be tested, the test session data is filtered to obtain at least two filtered sentences, including: selecting sentences in the test session data whose influence weight is greater than a weight threshold as filtered sentences. For example, the weight threshold can be 0.3. The weight threshold is not limited here and can be determined based on multiple statistical measures of influence weights (such as the median or average). For example, the weight threshold can be 1.5 times the statistical measure.

[0058] S140. Based on at least two screening sentences and a pre-trained satisfaction factor model, determine the target satisfaction factor for each screening sentence.

[0059] Specifically, the satisfaction factor model is a model used to predict the cause label of sentences that have an impact on conversation satisfaction. For example, the satisfaction factor model can be a convolutional neural network, a recurrent neural network, or an NLP (Natural Language Processing) multi-classification model, but is not limited to the example scenario.

[0060] For example, satisfaction factors corresponding to "very satisfied" could include a clear solution, personalized service, positive or professional language expression, etc.; satisfaction factors corresponding to "unsatisfied" could include unresolved issues, improper handling of issues, communication barriers, unfulfilled promises, or slow processing time, etc.; and satisfaction factors corresponding to "generally satisfied" could include a basically resolved issue, normal language expression, or service meeting expectations, etc. There are no restrictions on the settings for satisfaction factors here; they can be customized according to the specific conversation scenario.

[0061] In one specific embodiment, the method further includes: inputting training sentences from the training dataset into an untrained satisfaction model factor to obtain the output predicted satisfaction factor; determining a factor loss function based on the predicted satisfaction factor and the labeled satisfaction factor corresponding to the training sentence in the training dataset; adjusting the model parameters of the untrained satisfaction model factor model based on the factor loss function until the training termination condition is met, and then using the satisfaction model factor model in the current iteration as the trained satisfaction model factor model.

[0062] For example, the factor loss function can be the squared loss function, logarithmic loss function, exponential loss function, mean squared error loss function, logistic regression loss function, Huber loss function, or cross-entropy loss function, etc. The training termination condition can be the convergence of the factor loss function and / or the number of iterations reaching a threshold, but it is not limited to the above example.

[0063] Based on the above embodiments, specifically, the method further includes: determining the test satisfaction and influence weight set for each test session data in the test session dataset according to the test session dataset and the session satisfaction model; wherein, the influence weight set includes the influence weight of each test sentence in the test session data on the test satisfaction; for each test session data, filtering the test session data according to the influence weight set of the test session data to obtain a reference sentence set; performing clustering processing on the sentence set composed of the reference sentence sets corresponding to all test session data to obtain at least two clustered sentence sets; constructing a training dataset according to the at least two clustered sentence sets and the labeled satisfaction factors corresponding to the at least two clustered sentence sets respectively; wherein, the training dataset represents the dataset used to train the satisfaction factor model.

[0064] Specifically, the test session dataset contains multiple test session data, and the test session data contains multiple test sentences. The filtering strategy for the test session data can be the same as or different from the filtering strategy for the test session data in the above embodiment.

[0065] In one specific embodiment, a reference sentence set is obtained by filtering the test session data according to the influence weight set of the test session data, including adding the test sentence with the highest influence weight in the test session data as a reference sentence to the reference sentence set. In this embodiment, the reference sentence set contains one reference sentence.

[0066] For example, the clustering algorithms used in the clustering process include, but are not limited to, K-means clustering, Mean-Shift clustering, density-based noise spatial clustering, and Gaussian mixture model expectation-maximization clustering, etc. The clustering algorithm used here is not limited, and can be defined and set according to actual needs.

[0067] Specifically, the annotation satisfaction factor is the same for each cluster of sentence sets.

[0068] The advantage of this setup is that it solves the problem of large labeled data volume in the training dataset of the satisfaction factor model, reduces the labeling difficulty of the training dataset, and thus improves the efficiency of training dataset construction.

[0069] S150. Based on the influence weights of at least two selected sentences and the target satisfaction factor, determine the attribution data corresponding to the test session data.

[0070] In one specific embodiment, attribution data corresponding to the test session data is determined based on the influence weights of at least two selected sentences and the target satisfaction factor, including: using at least two influence weights as attribution weights of at least two target satisfaction factors respectively; wherein, the influence weights and attribution weights correspond one-to-one with the selected sentences; and constructing attribution data corresponding to the test session data based on at least two target satisfaction factors and the attribution weight of each target satisfaction factor.

[0071] In another specific embodiment, attribution data corresponding to the test session data is determined based on the influence weights of at least two selected sentences and the target satisfaction factor, including: normalizing the at least two influence weights to obtain the attribution weights corresponding to the at least two target satisfaction factors respectively; and constructing the attribution data corresponding to the test session data based on the at least two target satisfaction factors and the attribution weight of each target satisfaction factor.

[0072] For example, if the influence weights of the screening sentences corresponding to the three target satisfaction factors are 0.03, 0.09 and 0.1 respectively, then the attribution weights of the three target satisfaction factors in the attribution data are 0, 0.8571 and 1 respectively.

[0073] The technical solution of this embodiment, by combining with the task of evaluating conversation satisfaction, determines the conversation satisfaction through the conversation satisfaction model, and at the same time determines the influence weight of each sentence in the conversation data to be tested on the conversation satisfaction based on the conversation satisfaction model. The influence weight is used as the screening basis for the sentences to be tested, and the influence weight is used to represent the importance of the satisfaction factor corresponding to the screened sentences. This solves the problem of unclear guidance direction of attribution data leading to the dispersion of optimization resources, and improves the optimization efficiency of conversation service quality.

[0074] Figure 2 This is a flowchart illustrating another method for attributing conversation satisfaction according to an embodiment of the present invention. This embodiment further refines the step of "determining the conversation satisfaction of the conversation data to be tested based on the conversation data to be tested and a pre-trained conversation satisfaction model" in the above embodiment. In this embodiment, the conversation satisfaction model includes a word encoding layer, a word attention layer, a sentence encoding layer, a sentence attention layer, and a classifier.

[0075] like Figure 2 As shown, the method includes:

[0076] S210. Through the word encoding layer in the conversation satisfaction model, for each sentence to be tested, word embedding operation is performed on the sentence to be tested to obtain at least one word vector, and feature extraction is performed on the at least one word vector to obtain the word feature representation of the sentence to be tested.

[0077] Specifically, the test session data is represented as U, and the k-th test sentence in the test session data U is represented as U. k Then the session data to be tested is U = {U1, U2, ..., U...} k , ..., U n}, where n represents the number of sentences to be tested in the session data U.

[0078] With the sentence to be tested U k For example, suppose the sentence to be tested is U k Includes T k Each word represents a sentence to be tested. Each word in the algorithm is mapped to a word vector through a word embedding operation. For example, the word vector model corresponding to the word embedding operation can be the Word2Vec model, the GloVe model, etc.

[0079] In one specific embodiment, the word encoding module in the word encoding layer uses a text convolutional network, a BiGRU (Bidirectional Gated Recurrent Unit), or a bidirectional recurrent neural network.

[0080] Specifically, the word feature representation is a sequence of feature representations obtained by concatenating the word representations corresponding to at least one word in the sentence to be tested.

[0081] S220. Through the word attention layer in the conversation satisfaction model, attention processing is performed on the word feature representation of each sentence to be tested to obtain the sentence vector of the sentence to be tested.

[0082] In one specific embodiment, the word attention layer employs an in-sentence attention mechanism.

[0083] S230. Through the sentence encoding layer in the conversation satisfaction model, for each sentence to be tested, the sentence vector of the sentence to be tested is used to extract features to obtain the sentence feature representation of the sentence to be tested.

[0084] In one specific embodiment, the sentence encoding layer includes a sentence encoding module, which employs a text convolutional network, BiGRU, or a bidirectional recurrent neural network.

[0085] In another specific embodiment, the sentence encoding module further includes a multi-head attention module and a layer normalization module. The sentence encoding module is used to perform feature encoding on the sentence vector of each test sentence and output the sentence encoding features of the test sentence. The multi-head attention module is used to perform multi-head attention processing based on the sentence encoding features of multiple test sentences to obtain the attention features of each test sentence. The layer normalization module is used to perform layer normalization processing based on the attention features of multiple test sentences to determine the sentence feature representation of each test sentence.

[0086] The advantages of setting up a multi-head attention module and a layer normalization module are that the multi-head attention module can extract more sentence-level contextual semantic features, thereby improving the accuracy of conversation satisfaction, while the layer normalization module can ensure that the conversation satisfaction model can converge quickly during training and improve the stability of the conversation satisfaction model.

[0087] S240. Through the sentence attention layer in the conversation satisfaction model, attention processing is performed based on the sentence feature representations of multiple sentences to be tested to obtain the conversation vector of the conversation data to be tested.

[0088] In one specific embodiment, the sentence attention layer employs an in-sentence attention mechanism.

[0089] In one specific embodiment, the sentence attention layer includes a nonlinear module, an importance module, an activation function module, and an output module. The nonlinear module performs nonlinear processing on the sentence feature representation of each sentence to obtain nonlinear features. The importance module obtains the transposed features corresponding to the nonlinear features of each sentence to obtain an importance score by performing a dot product between the transposed features and the sentence-level contextual attention vector in the conversation satisfaction model. The activation function module normalizes the importance scores of multiple sentences to obtain the sentence weight of each sentence. The output module performs a weighted summation of the sentence feature representations and sentence weights of multiple sentences to obtain the conversation vector of the conversation data.

[0090] Specifically, the sentence to be tested, U k The nonlinear characteristic is represented by u k u k Satisfy the following formula:

[0091] u k =tanh(W s h k +b s )

[0092] Among them, W s and b s The parameter h represents the in-statement attention mechanism. k The sentence to be tested is U. k Sentence features representation.

[0093] Specifically, the activation function module uses the softmax function, and the sentence weight is represented by α. k α k Satisfy the following formula:

[0094]

[0095] in, Indicates the transpose feature, u s Represents sentence-level context attention vectors. Indicates the importance score.

[0096] Specifically, the session vector is represented as v, which satisfies the following formula:

[0097]

[0098] S250. Using the classifier in the conversation satisfaction model, output the conversation satisfaction of the conversation data to be tested based on the conversation vector.

[0099] In one specific embodiment, the classifier includes a global dialogue representation module and an activation function module. For example, the activation function module uses the softmax function.

[0100] Figure 3 This is a model architecture diagram illustrating a specific example of a conversation satisfaction model provided in an embodiment of the present invention. Specifically, Figure 3 The four blank boxes in the text represent word feature representations. Both the word encoding layer and the sentence encoding layer use BiGRU encoding modules. The classifier includes a global dialogue representation module and a softmax module.

[0101] S260. Based on the conversation satisfaction model, determine the influence weight of each sentence in the conversation data on conversation satisfaction.

[0102] In one specific embodiment, the influence weight of each sentence to be tested on the session satisfaction in the session data is determined according to the session satisfaction model, including: obtaining the sentence weights corresponding to the multiple sentences to be tested output by the activation function module in the session satisfaction model; and for each sentence to be tested, the sentence weight of the sentence to be tested is used as the influence weight of the sentence to be tested on the session satisfaction.

[0103] S270. Based on the influence weight of each sentence to be tested, filter the conversation data to be tested to obtain at least two filter sentences.

[0104] S280. Based on at least two screening sentences and a pre-trained satisfaction factor model, determine the target satisfaction factor for each screening sentence.

[0105] S290. Based on the influence weights of at least two selected sentences and the target satisfaction factor, determine the attribution data corresponding to the test session data.

[0106] In this embodiment, S270-S290 correspond to the same or similar S130-S150 in the above embodiment, and will not be described again in this embodiment.

[0107] The technical solution of this embodiment sets the conversation satisfaction model to include a word encoding layer, a word attention layer, a sentence encoding layer, a sentence attention layer, and a classifier. This enables the conversation satisfaction model to extract the influence weight of the sentence to be tested on the conversation satisfaction. While ensuring the accuracy of conversation satisfaction, it reduces the difficulty of obtaining the influence weight, thereby improving the attribution efficiency of conversation satisfaction.

[0108] It should be noted that the collection, use, storage, sharing and transfer of user personal information involved in the technical solution of the present invention all comply with the provisions of relevant laws and regulations, and require notification to users and obtaining their consent or authorization. When applicable, user personal information is subjected to de-identification and / or anonymization and / or encryption technical processing.

[0109] The following are embodiments of the conversation satisfaction attribution device provided in this invention. This device and the conversation satisfaction attribution method in the above embodiments belong to the same inventive concept. For details not described in detail in the embodiments of the conversation satisfaction attribution device, please refer to the content of the conversation satisfaction attribution method in the above embodiments.

[0110] Figure 4 This is a schematic diagram of the structure of an attribution device for conversation satisfaction provided in one embodiment of the present invention. Figure 4 As shown, the device includes: a conversation satisfaction determination module 310, an influence weight determination module 320, a conversation data screening module 330, a target satisfaction factor determination module 340, and an attribution data determination module 350.

[0111] Among them, the conversation satisfaction determination module 310 is used to determine the conversation satisfaction of the conversation data to be tested based on the conversation data to be tested and the pre-trained conversation satisfaction model.

[0112] The influence weight determination module 320 is used to determine the influence weight of each sentence in the test conversation data on the conversation satisfaction based on the conversation satisfaction model.

[0113] The test session data filtering module 330 is used to filter the test session data according to the influence weight of each test sentence to obtain at least two filtered sentences;

[0114] The target satisfaction factor determination module 340 is used to determine the target satisfaction factor for each screening sentence based on at least two screening sentences and a pre-trained satisfaction factor model.

[0115] The attribution data determination module 350 is used to determine the attribution data corresponding to the test session data based on the influence weights of at least two selected sentences and the target satisfaction factor.

[0116] The technical solution of this embodiment, by combining with the task of evaluating conversation satisfaction, determines the conversation satisfaction through the conversation satisfaction model, and at the same time determines the influence weight of each sentence in the conversation data to be tested on the conversation satisfaction based on the conversation satisfaction model. The influence weight is used as the screening basis for the sentences to be tested, and the influence weight is used to represent the importance of the satisfaction factor corresponding to the screened sentences. This solves the problem of unclear guidance direction of attribution data leading to the dispersion of optimization resources, and improves the optimization efficiency of conversation service quality.

[0117] In one specific embodiment, the session satisfaction determination module 310 is specifically used for:

[0118] Through the word encoding layer in the conversation satisfaction model, for each sentence to be tested, word embedding operation is performed on the sentence to be tested to obtain at least one word vector, and feature extraction is performed on each of the at least one word vector to obtain the word feature representation of the sentence to be tested;

[0119] By using the word attention layer in the conversation satisfaction model, attention processing is performed on the word feature representation of each sentence to be tested to obtain the sentence vector of the sentence to be tested.

[0120] By using the sentence encoding layer in the conversation satisfaction model, for each sentence to be tested, the sentence vector of the sentence to be tested is used to extract features, and the sentence feature representation of the sentence to be tested is obtained.

[0121] By using the sentence attention layer in the conversation satisfaction model, attention processing is performed based on the sentence feature representations of multiple sentences to be tested, and the conversation vector of the conversation data to be tested is obtained.

[0122] The classifier in the conversation satisfaction model outputs the conversation satisfaction of the test conversation data based on the conversation vector.

[0123] In one specific embodiment, the sentence attention layer includes a non-linear module, an importance module, an activation function module, and an output module;

[0124] The system comprises the following modules: a nonlinear module, which performs nonlinear processing on the sentence feature representation of each sentence to obtain nonlinear features; an importance module, which obtains the transposed features corresponding to the nonlinear features of each sentence to obtain an importance score by performing a dot product between the transposed features and the sentence-level contextual attention vector in the conversation satisfaction model; an activation function module, which normalizes the importance scores of multiple sentences to obtain the sentence weight of each sentence; and an output module, which performs a weighted summation of the sentence feature representations and sentence weights of multiple sentences to obtain the conversation vector of the conversation data.

[0125] In one specific embodiment, the influence weight determination module 320 is specifically used for:

[0126] Obtain the sentence weights corresponding to the multiple test sentences output by the activation function module in the conversation satisfaction model;

[0127] For each sentence to be tested, the sentence weight of the sentence to be tested is used as the weight of the impact of the sentence to be tested on the satisfaction of the conversation.

[0128] In one specific embodiment, the sentence encoding layer includes a sentence encoding module, a multi-head attention module, and a layer normalization module;

[0129] The system includes a sentence encoding module, which encodes the sentence vector of each test sentence to output the sentence encoding features; a multi-head attention module, which performs multi-head attention processing based on the sentence encoding features of multiple test sentences to obtain the attention features of each test sentence; and a layer normalization module, which performs layer normalization processing based on the attention features of multiple test sentences to determine the sentence feature representation of each test sentence.

[0130] In one specific embodiment, the method further includes:

[0131] The training dataset construction module is used to determine the test satisfaction and influence weight set for each test session data in the test session dataset based on the test session dataset and the session satisfaction model; wherein, the influence weight set contains the influence weight of each test sentence in the test session data on the test satisfaction.

[0132] For each test session data, a reference sentence set is obtained by filtering the test session data according to the influence weight set of the test session data;

[0133] Cluster the sentence set consisting of the reference sentence set corresponding to all test session data to obtain at least two clustered sentence sets;

[0134] A training dataset is constructed based on at least two clustered sentence sets and the labeled satisfaction factors corresponding to each of the at least two clustered sentence sets; wherein, the training dataset represents the dataset used to train the satisfaction factor model.

[0135] The conversation satisfaction attribution device provided in this embodiment of the invention can execute the conversation satisfaction attribution method provided in any embodiment of the invention, and has the corresponding functional modules and beneficial effects of the execution method.

[0136] Figure 5This is a schematic diagram of an electronic device provided according to one embodiment of the present invention. The electronic device 10 is intended to represent various forms of digital computers, such as laptop computers, desktop computers, workbenches, servers, blade servers, mainframe computers, and other suitable computers. The electronic device may also represent various forms of mobile devices, such as personal digital assistants, cellular phones, smartphones, wearable devices (e.g., helmets, glasses, watches, etc.), and other similar computing devices. The components shown herein, their connections and relationships, and their functions are merely illustrative and are not intended to limit the implementation of the invention described and / or claimed herein.

[0137] like Figure 5 As shown, the electronic device 10 includes at least one processor 11 and a memory, such as a read-only memory (ROM) 12 or a random access memory (RAM) 13, communicatively connected to the at least one processor 11. The memory stores computer programs executable by the at least one processor 11. The processor 11 can perform various appropriate actions and processes based on the computer program stored in the ROM 12 or loaded from storage unit 18 into the RAM 13. The RAM 13 can also store various programs and data required for the operation of the electronic device 10. The processor 11, ROM 12, and RAM 13 are interconnected via a bus 14. An input / output (I / O) interface 15 is also connected to the bus 14.

[0138] Multiple components in electronic device 10 are connected to I / O interface 15, including: input unit 16, such as keyboard, mouse, etc.; output unit 17, such as various types of displays, speakers, etc.; storage unit 18, such as disk, optical disk, etc.; and communication unit 19, such as network card, modem, wireless transceiver, etc. Communication unit 19 allows electronic device 10 to exchange information or data with other devices through computer networks such as the Internet and / or various telecommunications networks.

[0139] Processor 11 can be a variety of general-purpose and / or special-purpose processing components with processing and computing capabilities. Some examples of processor 11 include, but are not limited to, a central processing unit (CPU), a graphics processing unit (GPU), various special-purpose artificial intelligence (AI) computing chips, various processors running machine learning model algorithms, digital signal processors (DSPs), and any suitable processor, controller, microcontroller, etc. Processor 11 performs the various methods and processes described above, such as the session satisfaction attribution method provided in the above embodiments.

[0140] In some embodiments, the session satisfaction attribution method provided in the above embodiments can be implemented as a computer program tangibly contained in a computer-readable storage medium, such as storage unit 18. In some embodiments, part or all of the computer program can be loaded and / or installed on electronic device 10 via ROM 12 and / or communication unit 19. When the computer program is loaded into RAM 13 and executed by processor 11, one or more steps of the session satisfaction attribution method described above can be performed. Alternatively, in other embodiments, processor 11 can be configured to perform the session satisfaction attribution method by any other suitable means (e.g., by means of firmware).

[0141] Various embodiments of the systems and techniques described above herein can be implemented in the following systems or combinations thereof: digital electronic circuit systems, integrated circuit systems, field programmable gate arrays (FPGAs), application-specific integrated circuits (ASICs), application-specific standard parts (ASSPs), systems on chips (SOCs), complex programmable logic devices (CPLDs), computer hardware, firmware, software, and / or combinations thereof. These various embodiments may include implementations in one or more computer programs that can be executed and / or interpreted on a programmable system including at least one programmable processor, which may be a dedicated or general-purpose programmable processor, capable of receiving data and instructions from a storage system, at least one input device, and at least one output device, and transmitting data and instructions to the storage system, the at least one input device, and the at least one output device.

[0142] Computer programs used to implement the attribution method for session satisfaction of the present invention can be written in any combination of one or more programming languages. These computer programs can be provided to a processor of a general-purpose computer, a special-purpose computer, or other programmable data processing device, such that when executed by the processor, the computer programs cause the functions / operations specified in the flowcharts and / or block diagrams to be implemented. The computer programs can be executed entirely on the machine, partially on the machine, as a standalone software package partially on the machine and partially on a remote machine, or entirely on a remote machine or server.

[0143] In the context of this application, a computer-readable storage medium can be a tangible medium that may contain or store a computer program for use by or in conjunction with an instruction execution system, apparatus, or device. A computer-readable storage medium can be, but is not limited to, electronic, magnetic, optical, electromagnetic, infrared, or semiconductor systems, apparatus, or devices, or any suitable combination of the foregoing. Alternatively, a computer-readable storage medium can be a machine-readable storage medium. Examples of machine-readable storage media include, based on an electrical connection of at least one wire, a portable computer disk, a hard disk, random access memory (RAM), read-only memory (ROM), erasable programmable read-only memory (EPROM), flash memory, optical fiber, compact disc read-only memory (CD-ROM), optical storage devices, magnetic storage devices, or any suitable combination of the foregoing.

[0144] To provide interaction with a user, the systems and techniques described herein can be implemented on a terminal device having: a display device for displaying information to the user (e.g., a cathode-ray tube (CRT) or liquid crystal display (LCD) monitor); and a keyboard and pointing device (e.g., a mouse or trackball) through which the user provides input to the terminal device. Other types of devices can also provide interaction with the user; for example, feedback provided to the user can be any form of sensory feedback (e.g., visual feedback, auditory feedback, or tactile feedback); and input from the user can be received in any form (including sound input, voice input, or tactile input).

[0145] The systems and technologies described herein can be implemented in computing systems that include backend components (e.g., as data servers), or computing systems that include middleware components (e.g., application servers), or computing systems that include frontend components (e.g., user computers with graphical user interfaces or web browsers through which users can interact with implementations of the systems and technologies described herein), or any combination of such backend, middleware, or frontend components. The components of the system can be interconnected via digital data communication (e.g., communication networks) of any form or medium. Examples of communication networks include local area networks (LANs), wide area networks (WANs), blockchain networks, and the Internet.

[0146] A computing system can include clients and servers. Clients and servers are generally located far apart and typically interact through communication networks. The client-server relationship is created by computer programs running on the respective computers and having a client-server relationship with each other. The server can be a cloud server, also known as a cloud computing server or cloud host, which is a hosting product within the cloud computing service system. It addresses the shortcomings of traditional physical hosts and Virtual Private Server (VPS) services, such as high management difficulty and weak business scalability.

[0147] It should be understood that the various forms of processes shown above can be used, with steps reordered, added, or deleted. For example, the steps described in this invention can be executed in parallel, sequentially, or in different orders, as long as the desired result of the technical solution of this invention can be achieved, and this is not limited herein.

[0148] The specific embodiments described above do not constitute a limitation on the scope of protection of this invention. Those skilled in the art should understand that various modifications, combinations, sub-combinations, and substitutions can be made according to design requirements and other factors. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of this invention should be included within the scope of protection of this invention.

Claims

1. An attribution method for conversation satisfaction, characterized in that, include: The session satisfaction level of the session data to be tested is determined based on the session data to be tested and the pre-trained session satisfaction model. Based on the conversation satisfaction model, determine the influence weight of each sentence in the conversation data to be tested on the conversation satisfaction. Based on the influence weight of each sentence to be tested, the test session data is filtered to obtain at least two filtered sentences; Based on the at least two screening sentences and the pre-trained satisfaction factor model, determine the target satisfaction factor for each screening sentence; Based on the influence weights of the at least two selected sentences and the target satisfaction factor, the attribution data corresponding to the test session data is determined.

2. The method according to claim 1, characterized in that, The step of determining the session satisfaction of the session data to be tested based on the session data to be tested and the pre-trained session satisfaction model includes: Through the word encoding layer in the conversation satisfaction model, for each sentence to be tested, word embedding operation is performed on the sentence to be tested to obtain at least one word vector, and feature extraction is performed on the at least one word vector to obtain the word feature representation of the sentence to be tested; Through the word attention layer in the conversation satisfaction model, attention processing is performed on the word feature representation of each sentence to be tested to obtain the sentence vector of the sentence to be tested. Through the sentence encoding layer in the conversation satisfaction model, for each sentence to be tested, the sentence vector of the sentence to be tested is used to extract features to obtain the sentence feature representation of the sentence to be tested; The conversation vector of the conversation data to be tested is obtained by performing attention processing based on the sentence feature representation of multiple sentences to be tested through the sentence attention layer in the conversation satisfaction model. The classifier in the conversation satisfaction model outputs the conversation satisfaction of the test conversation data based on the conversation vector.

3. The method according to claim 2, characterized in that, The sentence attention layer includes a non-linear module, an importance module, an activation function module, and an output module; The nonlinear module is used to perform nonlinear processing on the sentence feature representation of each sentence to be tested to obtain nonlinear features; the importance module is used to obtain the transposed features corresponding to the nonlinear features of each sentence to be tested, and perform dot product processing on the transposed features and the sentence-level context attention vector in the conversation satisfaction model to obtain the importance score of the sentence to be tested; the activation function module is used to normalize the importance scores of the multiple sentences to be tested to obtain the sentence weight of each sentence to be tested; the output module is used to perform weighted summation processing on the sentence feature representations and sentence weights of the multiple sentences to be tested to obtain the conversation vector of the conversation data to be tested.

4. The method according to claim 3, characterized in that, The step of determining the influence weight of each sentence in the test conversation data on the conversation satisfaction based on the conversation satisfaction model includes: Obtain the sentence weights corresponding to the multiple test sentences output by the activation function module in the conversation satisfaction model; For each sentence to be tested, the sentence weight of the sentence to be tested is used as the influence weight of the sentence to be tested on the satisfaction of the conversation.

5. The method according to claim 2, characterized in that, The sentence encoding layer includes a sentence encoding module, a multi-head attention module, and a layer normalization module; The sentence encoding module is used to encode the sentence vector of each sentence to be tested and output the sentence encoding features of the sentence to be tested; the multi-head attention module is used to perform multi-head attention processing based on the sentence encoding features of the multiple sentences to be tested to obtain the attention features of each sentence to be tested; and the layer normalization module is used to perform layer normalization processing based on the attention features of the multiple sentences to be tested to determine the sentence feature representation of each sentence to be tested.

6. The method according to any one of claims 1-5, characterized in that, The method further includes: Based on the test session dataset and the session satisfaction model, determine the test satisfaction and influence weight set for each test session data in the test session dataset; wherein, the influence weight set includes the influence weight of each test sentence in the test session data on the test satisfaction. For each test session data, a reference sentence set is obtained by filtering the test session data according to the influence weight set of the test session data; Cluster the sentence set consisting of the reference sentence set corresponding to all test session data to obtain at least two clustered sentence sets; A training dataset is constructed based on the at least two clustered sentence sets and the labeled satisfaction factors corresponding to the at least two clustered sentence sets respectively; wherein, the training dataset represents the dataset used to train the satisfaction factor model.

7. An attribution device for conversation satisfaction, characterized in that, include: The conversation satisfaction determination module is used to determine the conversation satisfaction of the conversation data to be tested based on the conversation data to be tested and the pre-trained conversation satisfaction model. The influence weight determination module is used to determine the influence weight of each sentence in the test conversation data on the conversation satisfaction based on the conversation satisfaction model. The test conversation data filtering module is used to filter the test conversation data according to the influence weight of each test sentence to obtain at least two filtered sentences; The target satisfaction factor determination module is used to determine the target satisfaction factor for each screening sentence based on the at least two screening sentences and the pre-trained satisfaction factor model. The attribution data determination module is used to determine the attribution data corresponding to the test session data based on the influence weights of the at least two selected sentences and the target satisfaction factor.

8. An electronic device, characterized in that, The electronic device includes: At least one processor; and A memory communicatively connected to the at least one processor; wherein, The memory stores a computer program that can be executed by the at least one processor to enable the at least one processor to perform the attribution method for session satisfaction as described in any one of claims 1-6.

9. A computer-readable storage medium, characterized in that, The computer-readable storage medium stores computer instructions that, when executed by a processor, implement the attribution method for session satisfaction as described in any one of claims 1-6.

10. A computer program product comprising a computer program that, when executed by a processor, implements the attribution method for session satisfaction according to any one of claims 1-6.

Images