Service Satisfaction Determination Method, Apparatus, Storage Medium, and Computer Device

Abstract

The present invention discloses a method, apparatus, storage medium and computer device for determining service satisfaction. The method includes: obtaining each customer utterance and each merchant utterance during the conversation between the customer and the merchant; vectorizing each customer utterance to obtain a vector representation of each customer utterance, and vectorizing each merchant utterance to obtain a vector representation of each merchant utterance; for each customer utterance, matching a set of merchant utterances corresponding to the customer utterance, and obtaining a vector representation of the set of merchant utterances, using the vector representation of the set of merchant utterances to enhance the vector representation of the customer utterance, and obtaining an enhanced vector representation of each customer utterance; determining the emotion type of each customer utterance according to the enhanced vector representation of each customer utterance; determining the customer's satisfaction with the conversation at least according to the emotion type of each customer utterance.

Description

Technical Field

[0001] The present invention relates to the field of computers, and in particular, to a method, apparatus, storage medium, and computer device for determining service satisfaction. Background Art

[0002] Current e-commerce platforms have become comprehensive and promising ecosystems. They not only penetrate into traditional industries, such as payment and logistics, but also greatly change all aspects of the retail industry. Taking customer service as an example, third-party retailers answer various doubts and questions of potential customers before, during, and after sales through instant messaging software provided by the platform. The content of the conversation may cover all aspects of the product, such as product performance information, return and exchange services, logistics query problems, etc. Therefore, the customer experience can be obtained from the content of the conversation. For example, whether the customer is satisfied. However, in the related art, obtaining the service satisfaction of customers is extremely complex and cumbersome. For example, it is necessary for merchants to view the conversation and manually analyze the conversation, which is not only too costly but also too inefficient.

[0003] In view of the above problems, no effective solution has been proposed yet. Summary of the Invention

[0004] Embodiments of the present invention provide a method, apparatus, storage medium, and computer device for determining service satisfaction, so as to at least solve the technical problems in the related art that obtaining the service satisfaction of customers is not only costly but also inefficient.

[0005] According to one aspect of the embodiments of the present invention, a method for determining service satisfaction is provided, including: obtaining each customer utterance and each merchant utterance in the process of the conversation between the customer and the merchant; vectorizing each customer utterance to obtain a vector representation of each customer utterance, and vectorizing each merchant utterance to obtain a vector representation of each merchant utterance; for each customer utterance, matching a set of merchant utterances corresponding to the customer utterance, and obtaining a vector representation of the set of merchant utterances, and using the vector representation of the set of merchant utterances to enhance the vector representation of the customer utterance to obtain an enhanced vector representation of each customer utterance; determining the emotion type of each customer utterance according to the enhanced vector representation of each customer utterance; and determining the satisfaction of the customer with the conversation at least according to the emotion type of each customer utterance.

[0006] According to another aspect of the embodiments of the present invention, there is also provided a service satisfaction determination device, including: an acquisition module, configured to acquire each customer utterance and each merchant utterance during the conversation between the customer and the merchant; a vectorization module, configured to vectorize each customer utterance to obtain a vector representation of each customer utterance, and vectorize each merchant utterance to obtain a vector representation of each merchant utterance; a matching module, configured to, for each customer utterance, match a set of merchant utterances corresponding to the customer utterance, and obtain a vector representation of the set of merchant utterances, and use the vector representation of the set of merchant utterances to enhance the vector representation of the customer utterance to obtain an enhanced vector representation of each customer utterance; a first determination module, configured to determine the emotion type of each customer utterance according to the enhanced vector representation of each customer utterance; and a second determination module, configured to determine the satisfaction of the customer with the conversation at least according to the emotion type of each customer utterance.

[0007] According to still another aspect of the embodiments of the present invention, there is also provided a storage medium storing a program, wherein when the program is run by a processor, the processor is controlled to execute the service satisfaction determination method described in any one of the above.

[0008] According to yet another aspect of the embodiments of the present invention, there is also provided a processor for running a program, wherein when the program runs, it executes the service satisfaction determination method described in any one of the above.

[0009] According to still another aspect of the embodiments of the present invention, there is also provided a computer device, including: a memory and a processor, the memory storing a computer program; the processor being configured to execute the computer program stored in the memory, and when the computer program runs, the processor is caused to execute the service satisfaction determination method described in any one of the above.

[0010] In the embodiments of the present invention, by vectorizing the customer utterances and the merchant utterances to obtain vector representations of the customer utterances and the merchant utterances, and using the vector representation of the matched set of merchant utterances to enhance the vector representation of the customer utterances to obtain enhanced vector representations of the customer utterances, determining the emotion type of the customer utterances based on the obtained enhanced vector representations of the customer utterances, and finally determining the satisfaction of the customer with the conversation at least according to the emotion type of the customer utterances, by taking the customer utterances as the basis, using the merchant utterances for context connection and information supplementation, the purpose of intelligently and accurately obtaining the service satisfaction of the customer is achieved, thereby realizing the technical effects of reducing costs and improving efficiency, and further solving the technical problems in the related art that to obtain the service satisfaction of the customer, there are not only high costs but also low efficiency. BRIEF DESCRIPTION OF THE DRAWINGS

[0011] The accompanying drawings described herein are used to provide a further understanding of the present invention and form a part of this application. The schematic embodiments of the present invention and their descriptions are used to explain the present invention and do not constitute an improper limitation to the present invention. In the drawings:

[0012] Figure 1 is a hardware structure block diagram of a computer terminal for implementing a service satisfaction determination method according to an embodiment of the present invention;

[0013] Figure 2 is a flowchart of a service satisfaction determination method according to Embodiment 1 of the present invention;

[0014] Figure 3 is a service dialogue according to a preferred embodiment of the present invention and an emotional analysis diagram of the service dialogue;

[0015] Figure 4 is a schematic structural diagram of a service satisfaction analysis model according to a preferred embodiment of the present invention;

[0016] Figure 5 is a schematic structural diagram of a service satisfaction analysis system according to a preferred embodiment of the present invention;

[0017] Figure 6 is a block diagram of the structure of a service satisfaction determination device according to an embodiment of the present invention. Detailed Embodiments

[0018] In order to enable those skilled in the art of the present technology to better understand the solution of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings in the embodiments of the present invention. Obviously, the described embodiments are only a part of the embodiments of the present invention, rather than all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those of ordinary skill in the art without making creative efforts shall fall within the protection scope of the present invention.

[0019] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned accompanying drawings are used to distinguish similar objects and do not necessarily need to describe a specific order or sequence. It should be understood that such data can be interchanged under appropriate circumstances so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. In addition, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion. For example, a process, method, system, product, or device that includes a series of steps or units does not necessarily have to be limited to those steps or units clearly listed, but may include other steps or units not clearly listed or inherent to these processes, methods, products, or devices.

[0020] First, some nouns or terms that appear during the description of the embodiments of the present application are applicable to the following explanations:

[0021] Long Short-Term Memory (LSTM) model: A type of recurrent neural network model suitable for processing and predicting important events with relatively long intervals and delays in time series. Systems based on LSTM can learn to translate languages, control robots, perform image analysis, speech recognition, image recognition, speech recognition, etc.

[0022] Softmax function: Also known as the normalized exponential function, it is a generalization of the logistic function and is the gradient logarithmic normalization of a finite-term discrete probability distribution. It can "compress" a K-dimensional vector containing arbitrary real numbers into another K-dimensional vector, such that the range of each element is between (0, 1), and the sum of all elements is 1.

[0023] Embodiment 1

[0024] According to an embodiment of the present invention, there is also provided a method embodiment of a service satisfaction determination method. It should be noted that the steps shown in the flowchart of the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions. And although the logical order is shown in the flowchart, in some cases, the steps shown or described can be executed in a different order than here.

[0025] The method embodiment provided by the first embodiment of the present application can be executed on a mobile terminal, a computer terminal, or a similar computing device. Figure 1 It is a hardware structure block diagram of a computer terminal (or mobile device) for implementing a service satisfaction determination method according to an embodiment of the present invention. As Figure 1 shown, the computer terminal 10 (or mobile device 10) may include one or more (shown as 102a, 102b,..., 102n in the figure) processors 102 (the processor 102 may include, but is not limited to, a processing device such as a microprocessor MCU or a programmable logic device FPGA), and a memory 104 for storing data. In addition, it may further include: a transmission module, a display, an input / output interface (I / O interface), a universal serial bus (USB) port (which can be included as one of the ports of the I / O interface), a network interface, a power supply, and / or a camera. Those of ordinary skill in the art can understand that Figure 1 the structure shown is only schematic and does not limit the structure of the above-mentioned electronic device. For example, the computer terminal 10 may further include more or fewer components than Figure 1 shown, or have a different configuration from Figure 1 shown.

[0026] It should be noted that one or more of the above-mentioned processors 102 and / or other data processing circuits can generally be referred to as "data processing circuits" herein. The data processing circuit can be embodied in whole or in part as software, hardware, firmware, or any combination thereof. In addition, the data processing circuit can be a single independent processing module, or be incorporated in whole or in part into any one of other elements in the computer terminal 10 (or mobile device). As involved in the embodiments of the present application, the data processing circuit is a kind of processor control (such as the selection of a variable resistance terminal path connected to an interface).

[0027] The memory 104 can be used to store software programs and modules of application software, such as the program instructions / data storage device corresponding to the service satisfaction determination method in the embodiments of the present invention. The processor 102 executes various functional applications and data processing by running the software programs and modules stored in the memory 104, that is, implements the vulnerability detection method of the above-mentioned application program. The memory 104 can include a high-speed random access memory, and can also include a non-volatile memory, such as one or more magnetic storage devices, flash memory, or other non-volatile solid-state memories. In some instances, the memory 104 can further include a memory remotely disposed relative to the processor 102, and these remote memories can be connected to the computer terminal 10 through a network. Examples of the above-mentioned network include but are not limited to the Internet, enterprise intranet, local area network, mobile communication network, and combinations thereof.

[0028] The above-mentioned transmission module is used to receive or send data via a network. Specific examples of the above-mentioned network can include the wireless network provided by the communication provider of the computer terminal 10. In one instance, the transmission module includes a network adapter (Network Interface Controller, NIC), which can be connected to other network devices through a base station and thus can communicate with the Internet. In one instance, the transmission module can be a radio frequency (RF) module, which is used to communicate with the Internet wirelessly.

[0029] The display can be, for example, a touch-screen liquid crystal display (LCD), which enables the user to interact with the user interface of the computer terminal 10 (or mobile device).

[0030] It should be noted here that in some alternative embodiments, the above-mentioned Figure 1 shown computer device (or mobile device) can include hardware elements (including circuits), software elements (including computer code stored on a computer-readable medium), or a combination of both hardware elements and software elements. It should be pointed out that Figure 1This is just an example of a specific concrete instance and is intended to show the types of components that can exist in the above computer device (or mobile device).

[0031] Under the above operating environment, the present application provides a service satisfaction determination method as Figure 2 shown. Figure 2 It is a flowchart of the service satisfaction determination method according to Embodiment 1 of the present invention, as Figure 2 shown, and this process includes the following steps:

[0032] Step S202, obtain each customer utterance and each merchant utterance in the process of the customer's conversation with the merchant;

[0033] As an optional embodiment, the execution subject of this process can be, for example, the server of an e-commerce platform, or a network device for implementing this function. Among them, the server can be of a certain scale or a small machine device with the above processing function, such as a computer terminal, a notebook, a mobile phone, etc.

[0034] As an optional embodiment, the conversation between the customer and the merchant can be a conversation between the customer and the customer service entrusted by the merchant. That is, the merchant utterance can be represented by the customer service's customer service utterance. Specifically, it can be flexibly selected according to the scenario requirements.

[0035] As an optional embodiment, when obtaining each customer utterance and each merchant utterance in the process of the customer's conversation with the merchant, it can be obtained in various ways. For example, the conversation records between the customer and the merchant are stored in the customer service system. Therefore, each customer utterance and each merchant utterance in the process of the customer's conversation with the merchant can be obtained from the conversation records between the customer and the merchant.

[0036] Step S204, vectorize each customer utterance to obtain a vector representation of each customer utterance, and vectorize each merchant utterance to obtain a vector representation of each merchant utterance;

[0037] As an optional embodiment, when vectorizing each customer utterance to obtain a vector representation of each customer utterance, and vectorizing each merchant utterance to obtain a vector representation of each merchant utterance, various methods can be used. For example, a relatively simple single-layer unidirectional LSTM (Long Short-Term Memory Network) model can be used to learn the vector representation of each utterance.

[0038] For example, a conversation D can be divided into a customer utterance sequence and a merchant utterance sequence Where N and L represent the number of merchant utterances and the number of customer utterances in the conversation respectively. To obtain the vector representation of each utterance, a standard single-layer unidirectional LSTM (Long Short-Term Memory Network) model is used to learn the vector representation of each utterance, so each utterance can be represented as a fixed-length vector and This representation learning process can be formalized as:

[0039] Step S206, for each customer utterance, match the set of merchant utterances corresponding to the customer utterance, and obtain the vector representation of the set of merchant utterances. Use the vector representation of the set of merchant utterances to enhance the vector representation of the customer utterance, and obtain the enhanced vector representation of each customer utterance;

[0040] As an optional embodiment, for each customer utterance, match the set of merchant utterances corresponding to the customer utterance. Here, the set of merchant utterances can be the set of all merchant utterances in a conversation. For example, for each customer utterance, the merchant utterances before the customer utterance can be used to represent the correlation with the customer utterance, such as representing the basis and background of the customer utterance. And the merchant utterances after the customer utterance can be used to supplement, illustrate or explain the content of the customer utterance. The accuracy of the customer utterance representing the customer's emotion can be effectively improved by the merchant utterances before and after the customer utterance.

[0041] As an optional embodiment, there are various ways to match the set of merchant utterances corresponding to the customer utterance and obtain the vector representation of the set of merchant utterances. Since the set of merchant utterances matched corresponding to the customer utterance is used to enhance the customer vector representation, that is, to supplement or enrich the customer utterance expressing the customer's emotion, the influence degree of different merchant utterances on the customer utterance is different.

[0042] As an optional embodiment, the influence degree of different merchant utterances on the customer utterance considers the distance between the customer utterance and the merchant utterance. For example, according to the distance between the customer utterance and each merchant utterance, determine the position attention weight between the customer utterance and the merchant utterance. Among them, the closer the distance between the customer utterance and the merchant utterance, the greater the position attention weight between the customer utterance and the merchant utterance.

[0043] As an alternative embodiment, various methods can be adopted to determine the positional attention weights between the customer's utterance and each merchant's utterance based on the distance between the customer's utterance and each merchant's utterance. For example, determining the positional attention weights between the customer's utterance and each merchant's utterance based on the distance between the customer's utterance and each merchant's utterance includes: respectively determining the positional information of each customer's utterance and each merchant's utterance in the conversation; based on the positional information, determining the distance between each customer's utterance and each merchant's utterance; based on the distance and the number of utterances included in the customer-merchant conversation, determining the positional attention weights between the customer's utterance and each merchant's utterance. Specifically, the following method can be adopted to determine the positional attention weights between the customer's utterance and each merchant's utterance based on the distance between the customer's utterance and each merchant's utterance: wherein, and represent the respective positional information of the i-th customer's utterance and the j-th merchant's utterance in the conversation, is the distance between the i-th customer's utterance and the j-th merchant's utterance , and |D| is the number of utterances included in the customer-merchant conversation.

[0044] As an alternative embodiment, the influence degree of different merchant's utterances on the customer's utterance considers the relevance between the customer's utterance and the merchant's utterance. For example, the utterance attention weights between the customer's utterance and each merchant's utterance can be determined based on the relevance between the customer's utterance and each merchant's utterance. Among them, the greater the relevance between the customer's utterance and the merchant's utterance, the greater the utterance attention weights between the customer's utterance and the merchant's utterance.

[0045] As an alternative embodiment, various methods can also be adopted to determine the utterance attention weights between the customer's utterance and each merchant's utterance based on the relevance between the customer's utterance and each merchant's utterance. For example, the following method can be adopted to determine the utterance attention weights between the customer's utterance and each merchant's utterance based on the relevance between the customer's utterance and each merchant's utterance: Determine a neural network model for characterizing the relevance between the customer's utterance and each merchant's utterance; based on the neural network model, determine the utterance attention weights between the customer's utterance and each merchant's utterance. Specifically, the following method can be adopted to determine the utterance attention weights between the customer's utterance and each merchant's utterance based on the relevance between the customer's utterance and each merchant's utterance: wherein, W c , V, and b c are the model parameters of the neural network model, represents repeating the connection operation for the i-th customer's utterance a total of N times, and e N represents a column vector of all 1s.

[0046] As an alternative embodiment, considering the distance between the customer's utterance and the merchant's utterance and the relevance between the customer's utterance and the merchant's utterance as described above, a set of merchant's utterances corresponding to the customer's utterance can be determined according to the position attention weight and the utterance attention weight, and a vector representation of the set of merchant's utterances can be obtained.

[0047] As an alternative embodiment, various methods can be used to determine a set of merchant's utterances corresponding to the customer's utterance according to the position attention weight and the utterance attention weight, and obtain a vector representation of the set of merchant's utterances. For example, the following method can be used to determine a set of merchant's utterances corresponding to the customer's utterance according to the position attention weight and the utterance attention weight, and obtain a vector representation of the set of merchant's utterances: Obtain the vector representation of the merchant's utterance; Determine the vector representation of the set of merchant's utterances according to the position attention weight, the utterance attention weight, and the vector representation of the merchant's utterance. Specifically, the following method can be used to determine a set of merchant's utterances corresponding to the customer's utterance according to the position attention weight and the utterance attention weight, and obtain a vector representation of the set of merchant's utterances:

[0048]

[0049] where represents the vector representation of the j-th merchant's utterance.

[0050] Optionally, the vector representation of the set of merchant's utterances can be used in various ways to enhance the vector representation of the customer's utterance, and an enhanced vector representation of each customer's utterance can be obtained. For example, the following method can be used to enhance the vector representation of the customer's utterance using the vector representation of the set of merchant's utterances, and obtain an enhanced vector representation of each customer's utterance: Determine a predetermined model for enhancing the vector representation of the customer's utterance using the vector representation of the set of merchant's utterances; According to the predetermined model, obtain an enhanced vector representation of each customer's utterance. Specifically, the following method can be used to enhance the vector representation of the customer's utterance using the vector representation of the set of merchant's utterances, and obtain an enhanced vector representation of each customer's utterance: where h i represents the enhanced vector representation of the i-th customer's utterance.

[0051] Step S208, determine the sentiment type of each customer's utterance according to the enhanced vector representation of each customer's utterance;

[0052] As an alternative embodiment, when determining the sentiment type of each customer utterance based on the enhanced vector representation of each customer utterance, multiple methods can also be adopted. For example, a sentiment classification layer can be used to determine the sentiment type of each customer utterance according to the enhanced vector representation of each customer utterance. Among them, the sentiment classification layer includes a linear layer and a function classification layer, and the sentiment type is represented by a sentiment probability distribution. Specifically, using the sentiment classification layer to determine the sentiment type of each customer utterance according to the enhanced vector representation of each customer utterance includes: p i = softmax(W s h s + bs), where the sentiment classification layer includes a linear layer and a softmax function classification layer, Ws and bs are the parameters trained by the linear layer, and p i represents the sentiment probability distribution of the i-th customer utterance. There may be a certain correspondence between the sentiment probability of the customer utterance and the sentiment type of the customer. For example, assuming that the sentiment types of the customer utterances include three categories (positive, neutral, negative), it can be considered positive when the sentiment probability is greater than the first threshold (70%), neutral when the sentiment probability is between the first threshold (70%) and the second threshold (30%), and for the sentiment probability less than the second threshold, it can be considered that the sentiment expressed by the customer utterance is negative.

[0053] Step S210, determine the customer's satisfaction with the conversation based on at least the sentiment type of each customer utterance.

[0054] As an alternative embodiment, there are multiple ways to determine the customer's satisfaction with the conversation based on at least the sentiment type of each customer utterance. For example, determining the customer's satisfaction with the conversation based on at least the sentiment type of each customer utterance includes: determining the sentiment type weight of each customer utterance; determining the customer's satisfaction with the conversation based on at least the sentiment type of each customer utterance and the corresponding sentiment type weight. Specifically, the following method can be adopted to determine the customer's satisfaction with the conversation according to the sentiment type of each customer utterance:

[0055]

[0056] where p D represents the customer's satisfaction with the conversation, v i = tanh(W d h i + b d ), W d and b d represent trainable parameters, and the v d vector is a fixed query representation.

[0057] As an alternative embodiment, the above service satisfaction method can also display any object in the above steps. For example, at least one of the following objects can be displayed: display the conversation record between the customer and the merchant; display the customer's satisfaction with the conversation, as well as the process results included in the process of determining the customer's satisfaction with the conversation; display the setting and modification of the model training parameters, as well as the feedback and correction of the merchant on the customer satisfaction.

[0058] Combined with the above embodiments and preferred embodiments, a preferred embodiment of the present invention will be described below.

[0059] Before describing the preferred embodiment of the present invention, an example of a service conversation on an ordinary e-commerce platform will be used for illustration. Figure 3 is a service conversation according to the preferred embodiment of the present invention and an emotional analysis diagram of the service conversation, as Figure 3 shown. In Figure 3 the upper half, the content of the service conversation is described, that is, the multi-round conversation content between the customer and the merchant. Among them, C represents the customer, and S represents the seller; in Figure 3 the lower half, the emotional analysis of the customer's words is described. In this conversation, the customer requests the merchant to pay the mailing cost for the return. The evidence shows that the customer is not satisfied with the service.

[0060] Automatically analyzing the customer's satisfaction with the service provided by the merchant is very important. For retailers, it can quickly locate low-quality service conversations, find the reasons and take actions; it is also very important for e-commerce platforms, which can help the platform formulate clear rules, such as "not fitting is not a quality problem, and the buyer should bear the shipping cost".

[0061] Therefore, in the preferred embodiment of the present invention, the concept of the service satisfaction analysis task is given: given a conversation between a customer and a merchant (actually a service agent), this task aims to predict the customer's satisfaction, that is, whether the customer is satisfied with the merchant's response. Figure 3 The lower part gives the analysis result of dissatisfaction. In addition, the customer's emotion has a strong connection with the customer's final satisfaction, that is to say, the satisfaction can be aggregated by the emotion of each customer's sentence. For example, Figure 3 the lower part also gives the emotion of each customer's sentence. The customer gradually changes from the initial positive emotion to the neutral emotion and then turns to the negative emotion after being rejected, and it can be concluded that the customer is not satisfied.

[0062] Based on the above analysis and the given task concept, a service satisfaction analysis model is proposed in the preferred embodiment of the present invention. Figure 4 is a schematic structural diagram of the service satisfaction analysis model according to the preferred embodiment of the present invention, asFigure 4 As shown in Figure 4 , the model includes an input representation layer, a question-answer matching mechanism, and a core classification model. The input representation layer vectorizes each customer / merchant utterance in the conversation; the question-answer matching mechanism module (including a position attention layer and an utterance attention layer) is to find the set of merchant utterances most relevant to each customer statement and use it to enhance the vector representation of each customer utterance; the core classification model takes the enhanced customer statements as input, first performs sentiment classification on the utterances (categories include: positive, neutral, negative), and generates service satisfaction labels (categories include satisfied, average, dissatisfied) based on the sentiment classification results through an attention mechanism.

[0063] The following separately explains each part included in the service satisfaction analysis model.

[0064] Input Representation Layer

[0065] The input of this input representation layer is a conversation D, which can be divided into a customer utterance sequence and a merchant utterance sequence where N and L respectively represent the number of merchant utterances and the number of customer utterances in the conversation. To obtain the vector representation of each utterance, a standard single-layer unidirectional LSTM (Long Short-Term Memory Network) model is used to learn the vector representation of each utterance, so each utterance can be represented as a fixed-length vector and This representation learning process can be formalized as:

[0066]

[0067] Question-Answer Matching Mechanism

[0068] Drawing on the question-answer matching concept in machine reading comprehension, in the service satisfaction analysis task, the customer's utterance is regarded as a question, and the merchant's response is regarded as an answer. Note that although not all customer utterances are questions, there may also be complaints, etc., and a good merchant also needs to respond, which has many similarities but also differences from the traditional understanding of questions and answers. For the sake of easy understanding, the original concept of questions and answers is borrowed. The role of this module is to match the most relevant answers from the context through each customer question and enhance the vector representation of the customer utterance based on these statements, so as to obtain a better representation (simply modeling limited utterances through LSTM is obviously insufficient because the information contained in the utterances is limited and the context is not considered). Note that these matched merchant statements have two functions: providing context clues that arouse the customer's emotions in terms of sentiment; providing more semantic content information supplementation for the customer's utterance in terms of content. This question-answer matching mechanism includes two attention mechanism layers: position attention and utterance attention.

[0069] (1) Location Attention Layer: From observations in conversations and daily experience, it can be seen that questions and answers are usually adjacent in location, that is, for a question, the answer most relevant to it is near the question. Therefore, the purpose of the location attention layer is to use prior knowledge to guide matching and give higher weights or attention to answers (i.e., customer service utterances) closer to the question (i.e., customer utterances). Therefore, first define the location attention function g as follows:

[0070]

[0071] Where and represent the corresponding location information of utterances and in the conversation, is the distance between the two utterances, and |D| is the length of the conversation, that is, the number of utterances. Intuitively, the farther the two utterances are, the smaller the calculated weight value, and the closer they are, the larger the weight value. The output result of this layer is as follows:

[0072]

[0073] means multiplying the vector representation of each merchant utterance by the location attention weight. The output results of all merchant statements after passing through this layer are: O = [o1; o2;...; o N .

[0074] (2) Utterance Attention Layer: Given a customer question, each answer utterance of the merchant has a different degree of relevance. The purpose of this layer is to automatically find a more important set of merchant utterances and generate a vector representation by weighting the merchant utterances as information supplementation for the corresponding customer question vector. This process can be expressed by the following formula:

[0075]

[0076] Where α j represents the weight value calculated through the attention mechanism of this layer, and this value can be calculated through the following forward neural network:

[0077]

[0078] Where W c , V and b c are trainable model parameters, represents the repeated concatenation operation N times, that is And e N represents a column vector of all 1s.

[0079] Core Classification Model (or Core Classification Module)

[0080] The purpose of this core classification model is to obtain the sentiment analysis results for each customer utterance based on the vector representation of the input customer utterance and the vector generated from the matched merchant utterance and combine them into the satisfaction result of the merchant service

[0081] (1) Sentiment classification: Concatenate the representation vector generated from the matched merchant utterance to the original customer vector to enhance its vector representation (referred to as segment representation), and use it as the input of a single-layer unidirectional LSTM model. The gate mechanism of the LSTM unit can be utilized to capture the characteristics of long-term dependencies between inputs, and obtain the enhanced semantic vector representation h i for each customer statement. Specifically, it can be written as the following formula:

[0082]

[0083] To obtain the sentiment distribution for each statement, input h i into the sentiment classification layer, which includes a linear layer and a softmax function classification layer. The specific form is as follows:

[0084] p i = softmax(W s h s + b s )

[0085] where W s and b s are the trainable parameters of the linear layer, and p i represents the sentiment probability distribution of the i-th customer utterance

[0086] (2) Satisfaction classification: To judge the impact of the sentiment of different customer statements on the final satisfaction, this layer designs a segment attention mechanism to calculate different weight values for each segment of the customer statement. The specific form is as follows:

[0087] v i = tanh(W d h i + b d )

[0088]

[0089] where W d and b d represent the trainable parameters, and the v d vector is similar to a fixed query representation, that is, "what kind of customer sentiment is important". Finally, the satisfaction distribution of the dialogue D can be obtained by weighting the sentiment probability distributions of all customer utterances as follows:

[0090]

[0091] It should be noted that the training of each of the above models is based on the dialogue data with satisfaction labels annotated. Note that the sentiment labels of the utterances are not required to train the model, and the model can automatically deduce the sentiment labels of the utterances based on the satisfaction labels. The model uses the cross-entropy loss function as the objective function to estimate the parameter estimation of the model, and uses the Adam optimization technique to optimize the objective function.

[0092] In this preferred embodiment, based on the service satisfaction analysis model, an online service system prototype can be constructed. That is, in this preferred embodiment, a service satisfaction analysis system is provided. Figure 5 It is a schematic structural diagram of the service satisfaction analysis system according to the preferred embodiment of the present invention, as Figure 5 shown. The system may include the following interfaces: service dialogue record display, service satisfaction analysis, and system background management interface. The following is a brief introduction:

[0093] (1) Service dialogue record interface. This interface shows the historical dialogue record information of the customer service agent to the merchant, and provides three display methods respectively: display by the agent of the customer service, display by satisfaction category, and display the dialogue record information in chronological order.

[0094] (2) Service satisfaction analysis interface. This interface shows the analysis results of the algorithm to the merchant and displays them in a graph. This function includes: satisfaction trend analysis and display of the analysis results in a pie chart. The trend analysis can show the fluctuation of the service satisfaction of the customer service over time within the query time window. The smaller the fluctuation, the more stable the service satisfaction, and the higher the score, the better the quality of the customer service.

[0095] (3) System background management interface. On the one hand, this interface facilitates the system administrator to provide interfaces for model parameter setting, incremental training of the model, and data management. On the other hand, it provides the merchant with feedback and correction of the prediction results to help improve the system performance and the prediction effect.

[0096] It should be noted that for the foregoing method embodiments, for the sake of simple description, they are all expressed as a series of action combinations. However, those skilled in the art should know that the present invention is not limited by the described action sequence, because according to the present invention, certain steps can be performed in other sequences or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions and modules involved are not necessarily essential to the present invention.

[0097] Through the description of the above embodiments, those skilled in the art can clearly understand that the method according to the above embodiments can be implemented by means of software plus a necessary general hardware platform. Of course, it can also be implemented by hardware, but in many cases the former is a better implementation. Based on such an understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art can be embodied in the form of a software product. This computer software product is stored in a storage medium (such as ROM / RAM, magnetic disk, optical disk) and includes several instructions for causing a terminal device (which can be a mobile phone, computer, server, or network device, etc.) to execute the methods of the various embodiments of the present invention.

[0098] Embodiment 2

[0099] According to an embodiment of the present invention, there is also provided a device for implementing the above service satisfaction determination. Figure 6 It is a structural block diagram of a service satisfaction determination device according to an embodiment of the present invention, as Figure 6 shown. The device includes: an acquisition module 60, a vectorization module 62, a matching module 64, a first determination module 66, and a second determination module 68. The device will be described below.

[0100] The acquisition module 60 is used to acquire each customer's speech and each merchant's speech during the conversation between the customer and the merchant; the vectorization module 62 is connected to the above acquisition module 60 and is used to vectorize each customer's speech to obtain a vector representation of each customer's speech, and vectorize each merchant's speech to obtain a vector representation of each merchant's speech; the matching module 64 is connected to the above vectorization module 62 and is used to match, for each customer's speech, a set of merchant's speeches corresponding to the customer's speech, and obtain a vector representation of the set of merchant's speeches, and use the vector representation of the set of merchant's speeches to enhance the vector representation of the customer's speech to obtain an enhanced vector representation of each customer's speech; the first determination module 66 is connected to the above matching module 64 and is used to determine the emotion type of each customer's speech according to the enhanced vector representation of each customer's speech; the second determination module 68 is connected to the above first determination module 66 and is used to determine the customer's satisfaction with the conversation at least according to the emotion type of each customer's speech.

[0101] It should be noted here that the above acquisition module 60, vectorization module 62, matching module 64, first determination module 66, and second determination module 68 correspond to steps S202 to S210 in Embodiment 1. The instances and application scenarios implemented by each module and the corresponding steps are the same, but are not limited to the content disclosed in the above Embodiment 1. It should be noted that the above modules, as part of the device, can run in the computer terminal 10 provided in Embodiment 1.

[0102] As an alternative embodiment, the above-mentioned matching module 64 may include: a position attention layer unit configured to determine the position attention weights between the customer utterance and each merchant utterance according to the distance between the customer utterance and each merchant utterance, wherein the closer the distance between the customer utterance and the merchant utterance, the greater the position attention weight between the customer utterance and the merchant utterance; a discourse attention layer unit configured to determine the discourse attention weights between the customer utterance and each merchant utterance according to the relevance between the customer utterance and each merchant utterance, wherein the greater the relevance between the customer utterance and the merchant utterance, the greater the discourse attention weight between the customer utterance and the merchant utterance; a matching unit configured to determine a set of merchant utterances corresponding to the customer utterance according to the position attention weights and the discourse attention weights, and obtain a vector representation of the set of merchant utterances.

[0103] As an alternative embodiment, the above-mentioned matching module 64 may further include: a segment attention layer unit configured to determine a predetermined model for enhancing the vector representation of the customer utterance by using the vector representation of the set of merchant utterances; and according to the predetermined model, obtain an enhanced vector representation of each customer utterance.

[0104] Embodiment 3

[0105] An embodiment of the present invention may provide a computer terminal, which may be any computer terminal device in a group of computer terminals. Optionally, in this embodiment, the above-mentioned computer terminal may also be replaced with a terminal device such as a mobile terminal.

[0106] Optionally, in this embodiment, the above-mentioned computer terminal may be located in at least one of a plurality of network devices in a computer network.

[0107] Optionally, in this embodiment, the above-mentioned computer terminal may include: one or more (only one is shown in the figure) processors and a memory.

[0108] Wherein, the memory may be used to store software programs and modules, such as program instructions / modules corresponding to the security vulnerability detection method and device in the embodiment of the present invention. The processor executes various functional applications and data processing by running the software programs and modules stored in the memory, that is, implements the above-mentioned service satisfaction determination method. The memory may include a high-speed random access memory, and may further include a non-volatile memory, such as one or more magnetic storage devices, a flash memory, or other non-volatile solid-state memories. In some instances, the memory may further include a memory remotely provided with respect to the processor, and these remote memories may be connected to the above-mentioned computer terminal through a network. Examples of the above-mentioned network include but are not limited to the Internet, an enterprise internal network, a local area network, a mobile communication network, and combinations thereof.

[0109] The processor can call the information and application programs stored in the memory through the transmission device to execute the following steps: obtain each customer utterance and each merchant utterance during the conversation between the customer and the merchant; vectorize each customer utterance to obtain the vector representation of each customer utterance, and vectorize each merchant utterance to obtain the vector representation of each merchant utterance; for each customer utterance, match the corresponding set of merchant utterances, obtain the vector representation of the set of merchant utterances, and use the vector representation of the set of merchant utterances to enhance the vector representation of the customer utterance to obtain the enhanced vector representation of each customer utterance; determine the sentiment type of each customer utterance according to the enhanced vector representation of each customer utterance; determine the customer's satisfaction with the conversation at least according to the sentiment type of each customer utterance.

[0110] Optionally, the above-mentioned processor can also execute the program code of the following steps: for each customer utterance, the steps for matching the corresponding set of merchant utterances include: determining the position attention weight between the customer utterance and the merchant utterance according to the distance between the customer utterance and each merchant utterance, where the closer the distance between the customer utterance and the merchant utterance, the greater the position attention weight between the customer utterance and the merchant utterance; determining the discourse attention weight between the customer utterance and the merchant utterance according to the relevance between the customer utterance and each merchant utterance, where the greater the relevance between the customer utterance and the merchant utterance, the greater the discourse attention weight between the customer utterance and the merchant utterance; determining the set of merchant utterances corresponding to the customer utterance according to the position attention weight and the discourse attention weight, and obtaining the vector representation of the set of merchant utterances.

[0111] Optionally, the above-mentioned processor can also execute the program code of the following steps: determining the position attention weight between the customer utterance and the merchant utterance according to the distance between the customer utterance and each merchant utterance includes: respectively determining the position information of each customer utterance and each merchant utterance in the conversation; determining the distance between each customer utterance and each merchant utterance according to the position information; determining the position attention weight between the customer utterance and the merchant utterance according to the distance and the number of utterances included in the conversation between the customer and the merchant; determining the discourse attention weight between the customer utterance and the merchant utterance according to the relevance between the customer utterance and each merchant utterance includes: determining the neural network model for characterizing the relevance between the customer utterance and each merchant utterance; determining the discourse attention weight between the customer utterance and the merchant utterance according to the neural network model; determining the set of merchant utterances corresponding to the customer utterance according to the position attention weight and the discourse attention weight, and obtaining the vector representation of the set of merchant utterances includes: obtaining the vector representation of the merchant utterance; determining the vector representation of the set of merchant utterances according to the position attention weight, the discourse attention weight, and the vector representation of the merchant utterance.

[0112] Optionally, the above-mentioned processor may also execute the program code of the following steps: enhancing the vector representation of the customer's utterance with the vector representation of the merchant's utterance set, and obtaining the enhanced vector representation of each customer's utterance, including: determining a predetermined model for enhancing the vector representation of the customer's utterance with the vector representation of the merchant's utterance set; and obtaining the enhanced vector representation of each customer's utterance according to the predetermined model.

[0113] Optionally, the above-mentioned processor may also execute the program code of the following steps: determining the sentiment type of each customer's utterance according to the enhanced vector representation of each customer's utterance, including: using a sentiment classification layer to determine the sentiment type of each customer's utterance according to the enhanced vector representation of each customer's utterance, where the sentiment classification layer includes a linear layer and a function classification layer, and the sentiment type is represented by a sentiment probability distribution.

[0114] Optionally, the above-mentioned processor may also execute the program code of the following steps: determining the customer's satisfaction with the conversation at least according to the sentiment type of each customer's utterance, including: determining the sentiment type weight of the sentiment type of each customer's utterance; and determining the customer's satisfaction with the conversation at least according to the sentiment type of each customer's utterance and the corresponding sentiment type weight.

[0115] Optionally, the above-mentioned processor may also execute the program code of the following steps: further including at least one of the following: displaying the conversation record between the customer and the merchant; displaying the customer's satisfaction with the conversation and the process results included in the process of determining the customer's satisfaction with the conversation; and displaying the setting and modification of the model training parameters, as well as the feedback and correction of the merchant on the customer's satisfaction.

[0116] Through the embodiments of the present invention, by vectorizing the customer's utterance and the merchant's utterance, obtaining the vector representation of the customer's utterance and the vector representation of the merchant's utterance, and enhancing the vector representation of the customer's utterance with the vector representation of the matching merchant's utterance set to obtain the enhanced vector representation of the customer's utterance, determining the sentiment type of the customer's utterance based on the obtained enhanced vector representation of the customer's utterance, and finally determining the customer's satisfaction with the conversation at least according to the sentiment type of the customer's utterance, by using the customer's utterance as the basis, making context connection with the merchant's utterance and supplementing information, the purpose of intelligently and accurately obtaining the customer's service satisfaction is achieved, thereby realizing the technical effects of reducing costs and improving efficiency, and further solving the technical problems in the related art that to obtain the customer's service satisfaction, there are not only high costs but also low efficiency.

[0117] Those of ordinary skill in the art can understand that the computer terminal may also be a smart phone (such as an Android phone, an iOS phone, etc.), a tablet computer, a palm computer, and terminal devices such as Mobile Internet Devices (MID), PAD, etc.

[0118] Those of ordinary skill in the art can understand that all or part of the steps in the various methods of the above embodiments can be completed by instructing the hardware related to the terminal device through a program, and this program can be stored in a computer-readable storage medium. The storage medium can include: a flash drive, a read-only memory (ROM), a random access memory (RAM), a magnetic disk, an optical disk, etc.

[0119] Embodiment 4

[0120] An embodiment of the present invention also provides a storage medium. Optionally, in this embodiment, the above storage medium can be used to store the program code executed by the service satisfaction determination method provided in the first embodiment above.

[0121] Optionally, in this embodiment, the above storage medium can be located in any one of the computer terminals in the computer terminal group in the computer network, or in any one of the mobile terminals in the mobile terminal group.

[0122] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: obtaining each customer utterance and each merchant utterance in the process of the customer-merchant conversation; vectorizing each customer utterance to obtain a vector representation of each customer utterance, and vectorizing each merchant utterance to obtain a vector representation of each merchant utterance; for each customer utterance, matching a set of merchant utterances corresponding to the customer utterance, and obtaining a vector representation of the set of merchant utterances, and enhancing the vector representation of the customer utterance by using the vector representation of the set of merchant utterances to obtain an enhanced vector representation of each customer utterance; determining the emotion type of each customer utterance according to the enhanced vector representation of each customer utterance; determining the customer's satisfaction with the conversation at least according to the emotion type of each customer utterance.

[0123] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: for each customer utterance, the set of merchant utterances corresponding to the customer utterance is matched to include: determining the position attention weight between the customer utterance and the merchant utterance according to the distance between the customer utterance and each merchant utterance, where the closer the distance between the customer utterance and the merchant utterance, the greater the position attention weight between the customer utterance and the merchant utterance; determining the discourse attention weight between the customer utterance and the merchant utterance according to the relevance between the customer utterance and each merchant utterance, where the greater the relevance between the customer utterance and the merchant utterance, the greater the discourse attention weight between the customer utterance and the merchant utterance; determining the set of merchant utterances corresponding to the customer utterance according to the position attention weight and the discourse attention weight, and obtaining a vector representation of the set of merchant utterances.

[0124] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: determining the positional attention weights between the customer utterances and the merchant utterances according to the distances between the customer utterances and each merchant utterance includes: respectively determining the positional information of each customer utterance and each merchant utterance in the conversation; according to the positional information, determining the distances between each customer utterance and each merchant utterance; according to the distances and the number of utterances included in the customer-merchant conversation, determining the positional attention weights between the customer utterances and the merchant utterances; determining the utterance attention weights between the customer utterances and the merchant utterances according to the relevance between the customer utterances and each merchant utterance includes: determining a neural network model for characterizing the relevance between the customer utterances and each merchant utterance; according to the neural network model, determining the utterance attention weights between the customer utterances and the merchant utterances; determining a set of merchant utterances corresponding to the customer utterances according to the positional attention weights and the utterance attention weights, and obtaining a vector representation of the set of merchant utterances includes: obtaining the vector representation of the merchant utterances; according to the positional attention weights, the utterance attention weights, and the vector representation of the merchant utterances, determining the vector representation of the set of merchant utterances.

[0125] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: enhancing the vector representation of the customer utterances by using the vector representation of the set of merchant utterances to obtain an enhanced vector representation of each customer utterance includes: determining a predetermined model for enhancing the vector representation of the customer utterances by using the vector representation of the set of merchant utterances; according to the predetermined model, obtaining the enhanced vector representation of each customer utterance.

[0126] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: determining the sentiment type of each customer utterance according to the enhanced vector representation of each customer utterance includes: using a sentiment classification layer to determine the sentiment type of each customer utterance according to the enhanced vector representation of each customer utterance, where the sentiment classification layer includes a linear layer and a function classification layer, and the sentiment type is represented by a sentiment probability distribution.

[0127] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: determining the customer's satisfaction with the conversation at least according to the sentiment type of each customer utterance includes: determining the sentiment type weight of the sentiment type of each customer utterance; determining the customer's satisfaction with the conversation at least according to the sentiment type of each customer utterance and the corresponding sentiment type weight.

[0128] Optionally, in this embodiment, the storage medium is further configured to store program code for performing the following steps: It further includes at least one of the following: displaying the conversation record between the customer and the merchant; displaying the customer's satisfaction with the conversation, and the process results included in the process of determining the customer's satisfaction with the conversation; displaying the setting and modification of the model training parameters, as well as the feedback and correction of the merchant on the customer satisfaction.

[0129] The serial numbers of the above embodiments of the present invention are only for description and do not represent the advantages or disadvantages of the embodiments.

[0130] In the above embodiments of the present invention, the descriptions of the respective embodiments have their own emphases. For the parts not detailed in a certain embodiment, reference may be made to the relevant descriptions of other embodiments.

[0131] In the several embodiments provided by the present application, it should be understood that the disclosed technical content can be implemented in other ways. Among them, the device embodiments described above are only illustrative. For example, the division of units is only a logical function division. In actual implementation, there may be other division methods. For example, multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the displayed or discussed coupling or direct coupling or communication connection between each other can be through some interfaces. The indirect coupling or communication connection of units or modules can be in electrical or other forms.

[0132] The units described as separate components may or may not be physically separated. The components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed to multiple network units. Some or all of the units can be selected according to actual needs to achieve the purpose of the solution of this embodiment.

[0133] In addition, the functional units in the various embodiments of the present invention can be integrated in a processing unit, or each unit can exist physically alone, or two or more units can be integrated in one unit. The above integrated unit can be implemented in the form of hardware or in the form of a software functional unit.

[0134] When the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, in essence, or the part that contributes to the prior art, or all or part of this technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions for causing a computer device (which can be a personal computer, a server, or a network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes: various media that can store program codes, such as USB flash drives, read-only memories (ROMs), random access memories (RAMs), mobile hard disks, magnetic disks, or optical discs.

[0135] The above are only the preferred embodiments of the present invention. It should be noted that for those of ordinary skill in the art, without departing from the principle of the present invention, several improvements and modifications can be made, and these improvements and modifications should also be regarded as the protection scope of the present invention.

Claims

1. A method for determining service satisfaction, characterized in that, Including: Obtain each customer utterance and each merchant utterance in the process of the customer-merchant conversation; Vectorize each customer utterance to obtain the vector representation of each customer utterance, and vectorize each merchant utterance to obtain the vector representation of each merchant utterance; For each customer utterance, match the corresponding set of merchant utterances, obtain the vector representation of the set of merchant utterances, and use the vector representation of the set of merchant utterances to enhance the vector representation of the customer utterance to obtain the enhanced vector representation of each customer utterance; Determine the sentiment type of each customer utterance according to the enhanced vector representation of each customer utterance; Determine the customer's satisfaction with the conversation based at least on the sentiment type of each customer utterance; Among them, for each customer utterance, matching the corresponding set of merchant utterances and obtaining the vector representation of the set of merchant utterances includes: determining the corresponding set of merchant utterances according to the positional attention weight and the utterance attention weight, and obtaining the vector representation of the set of merchant utterances, where the closer the distance between the customer utterance and the merchant utterance, the greater the positional attention weight between the customer utterance and the merchant utterance, and the greater the relevance between the customer utterance and the merchant utterance, the greater the utterance attention weight between the customer utterance and the merchant utterance.

2. The method according to claim 1, wherein For each customer utterance, matching the corresponding set of merchant utterances and obtaining the vector representation of the set of merchant utterances includes: Determine the positional attention weight between the customer utterance and the merchant utterance according to the distance between the customer utterance and each merchant utterance; Determine the utterance attention weight between the customer utterance and the merchant utterance according to the relevance between the customer utterance and each merchant utterance; Determine the set of merchant utterances corresponding to the customer utterance according to the positional attention weight and the utterance attention weight, and obtain the vector representation of the set of merchant utterances.

3. The method according to claim 2, wherein Determining the positional attention weight between the customer utterance and the merchant utterance according to the distance between the customer utterance and each merchant utterance includes: respectively determining the position information of each customer utterance and each merchant utterance in the conversation; according to the position information, determining the distance between each customer utterance and each merchant utterance; according to the distance and the number of utterances included in the customer-merchant conversation, determining the positional attention weight between the customer utterance and the merchant utterance; Determining the utterance attention weight between the customer utterance and the merchant utterance according to the relevance between the customer utterance and each merchant utterance includes: determining a neural network model for characterizing the relevance between the customer utterance and each merchant utterance; according to the neural network model, determining the utterance attention weight between the customer utterance and the merchant utterance; Determining the set of merchant utterances corresponding to the customer utterance according to the positional attention weight and the utterance attention weight, and obtaining the vector representation of the set of merchant utterances includes: obtaining the vector representation of the merchant utterance; according to the positional attention weight, the utterance attention weight, and the vector representation of the merchant utterance, determining the vector representation of the set of merchant utterances.

4. The method according to claim 3, wherein Enhancing the vector representation of the customer utterance with the vector representation of the merchant utterance set to obtain the enhanced vector representation of each customer utterance includes: determining a predetermined model for enhancing the vector representation of the customer utterance with the vector representation of the merchant utterance set; and obtaining the enhanced vector representation of each customer utterance according to the predetermined model.

5. The method according to claim 4, wherein Determining the sentiment type of each customer utterance according to the enhanced vector representation of each customer utterance includes: Using a sentiment classification layer to determine the sentiment type of each customer utterance according to the enhanced vector representation of each customer utterance, where the sentiment classification layer includes a linear layer and a function classification layer, and the sentiment type is represented by a sentiment probability distribution.

6. The method according to claim 5, characterized in that, Determining the customer's satisfaction with the conversation based on at least the sentiment type of each customer utterance includes: Determining the sentiment type weight of the sentiment type of each customer utterance; Determining the customer's satisfaction with the conversation based on at least the sentiment type of each customer utterance and the corresponding sentiment type weight.

7. The method according to any one of claims 1 to 6, characterized in that, It further includes at least one of the following: Displaying the conversation record between the customer and the merchant; Displaying the customer's satisfaction with the conversation and the process results included in the process of determining the customer's satisfaction with the conversation; Displaying the setting and modification of the model training parameters, as well as the feedback and correction of the merchant on the customer satisfaction.

8. A service satisfaction determination device, characterized in that, It includes: An acquisition module for acquiring each customer utterance and each merchant utterance during the conversation between the customer and the merchant; A vectorization module for vectorizing each customer utterance to obtain the vector representation of each customer utterance, and vectorizing each merchant utterance to obtain the vector representation of each merchant utterance; A matching module for, for each customer utterance, matching the set of merchant utterances corresponding to the customer utterance, obtaining the vector representation of the merchant utterance set, and enhancing the vector representation of the customer utterance with the vector representation of the merchant utterance set to obtain the enhanced vector representation of each customer utterance; A first determination module for determining the sentiment type of each customer utterance according to the enhanced vector representation of each customer utterance; A second determination module for determining the customer's satisfaction with the conversation based on at least the sentiment type of each customer utterance; Wherein, the matching module is further configured to determine the set of merchant utterances corresponding to the customer utterance according to the position attention weight and the utterance attention weight, and obtain the vector representation of the merchant utterance set, where the closer the distance between the customer utterance and the merchant utterance, the greater the position attention weight between the customer utterance and the merchant utterance, and the greater the relevance between the customer utterance and the merchant utterance, the greater the utterance attention weight between the customer utterance and the merchant utterance.

9. The device according to claim 8, characterized in that, The matching module includes: A position attention layer unit for determining the position attention weight between the customer utterance and the merchant utterance according to the distance between the customer utterance and each merchant utterance; An utterance attention layer unit for determining the utterance attention weight between the customer utterance and the merchant utterance according to the relevance between the customer utterance and each merchant utterance; A matching unit for determining the set of merchant utterances corresponding to the customer utterance according to the position attention weight and the utterance attention weight, and obtaining the vector representation of the merchant utterance set.

10. The device according to claim 9, characterized in that The matching module further includes: A segmented attention layer unit, configured to determine a predetermined model for enhancing the vector representation of the customer utterance by using the vector representation of the merchant utterance set; and obtain the enhanced vector representation of each customer utterance according to the predetermined model.

11. A storage medium, characterized in that, The storage medium stores a program, wherein when the program is run by a processor, the processor is controlled to execute the service satisfaction determination method according to any one of claims 1 to 7.

12. A processor, characterized in that, The processor is configured to run a program, wherein when the program runs, the service satisfaction determination method according to any one of claims 1 to 7 is executed.

13. A computer device, characterized in that, Comprising: A memory and a processor, The memory stores a computer program; The processor is configured to execute the computer program stored in the memory, and when the computer program runs, the processor is caused to execute the service satisfaction determination method according to any one of claims 1 to 7.

Images