Complaint prediction method and model building method, device and related equipment

Abstract

The present invention provides a complaint prediction method and a model establishment method, device and related equipment thereof, the method comprising: setting a predicted label set of a complaint prediction model; obtaining historical customer service dialogue texts and labels of historical customer service dialogue texts; using an LTP language processing tool to segment historical customer service dialogue texts; using a BERT word segmenter to generate a word token id sequence, a word position id sequence, a word fragment id sequence and a word mask sequence; inputting a model encoding layer to obtain a word feature sequence; obtaining a word feature sequence; inputting the word feature sequence into a graph neural network of a grammatical layer to perceive grammatical features; outputting a sentence representation of the historical customer service dialogue text; performing affine transformation and normalization processing on the sentence representation to obtain a label probability distribution; inputting the sentence representation into a label confusion layer to calculate a pseudo-label probability distribution; calculating a loss function; and iteratively training a complaint prediction model. The present invention realizes complaint prediction of online travel agencies.

Description

Background Art

[0002] With the rapid development of deep learning related technologies, the status of neural network models in the field of natural language processing (NLP) has become increasingly important. Compared with traditional machine learning algorithms such as naive Bayes, support vector machines, and N-gram, neural networks have great advantages in automatically extracting features and constructing high-level abstractions, which can effectively overcome the limitations of manual feature engineering, which is time-consuming and labor-intensive and relies on expert experience.

[0003] Identifying customer complaints in the online travel agency industry is a major application direction of text classification in the tourism scenario. The difficulty of this task lies in:

[0004] 1. There are a large number of synonyms and polysemy in Chinese, and the informality, ambiguity and diversity of users' expressions in instant messaging (IM) all pose great challenges to classification;

[0005] 2. The amount of labeled data is usually small, the labeling cost is high, and it is difficult to learn a classifier with strong generalization ability;

[0006] 3. The boundaries between warning labels are not necessarily clear. Some content can even be divided into multiple categories based on semantics, resulting in serious annotation noise.

[0007] Compared with computer vision (CV), although the supervised data sets in the field of NLP are often small, which makes deep learning models prone to overfitting, the advantage of the field of NLP is that there are a large number of unsupervised corpora. If such data can be fully utilized for unsupervised and self-supervised learning, then it is possible to improve the performance of the model in downstream tasks. This is the motivation for the emergence of pre-trained language models. Since the BERT model proposed by Google in 2018 refreshed the SOTA of various natural language understanding tasks, the research on pre-trained language models has ushered in a boom. BERT uses the Transformer encoder as the main structure, and replaces the position of recurrent neural networks (RNNs) in sequence processing through parallel computing of multi-head self-attention mechanism, providing a new paradigm for encoding text representation.

[0008] When the WordPiece word segmentation algorithm used by BERT is applied to Chinese text processing, sentences are usually simply converted into character sequences. This approach ignores word-level information and the dependencies between Chinese words that can be independent sentence components. In addition, because the one-hot encoding method of labels assumes too strong and ignores the overlapping relationship between labels, it is easy to lose a lot of semantic information contained in the labels, causing the model to be unable to handle situations where labels are confused or noisy.

[0009] Therefore, how to capture word-level connections while avoiding losing a large amount of semantic information contained in the labels, which causes the model to be unable to handle label confusion or noise, thereby realizing complaint prediction for online travel agencies, is a technical problem that needs to be solved urgently in this field. Summary of the invention

[0010] In order to overcome the defects of the above-mentioned prior art, the present invention provides a complaint prediction method and its model building method, device and related equipment, which can capture word-level connections while avoiding the loss of a large amount of semantic information contained in the labels, causing the model to be unable to handle label confusion or noise, thereby realizing complaint prediction of online travel agencies.

[0011] According to one aspect of the present invention, a method for establishing a complaint prediction model is provided, comprising:

[0012] Setting a predicted label set of the complaint prediction model;

[0013] Get the historical customer service conversation text and the label of the historical customer service conversation text;

[0014] Using an LTP language processing tool to segment the historical customer service conversation text to generate a word mask sequence and a syntactic dependency graph;

[0015] Using the BERT tokenizer to generate a word token id sequence, a word position id sequence, a word fragment id sequence, and a word mask sequence based on the historical customer service conversation text;

[0016] Input the word token id sequence, word position id sequence, and word fragment id sequence into the model encoding layer to obtain the word feature sequence of the user content identified by the historical customer service conversation text in the character dimension;

[0017] converting the character feature sequence and the character mask sequence into a set size, and obtaining a word feature sequence based on the character feature sequence and the character mask sequence;

[0018] Inputting the word feature sequence into a graph neural network of a grammatical layer to perceive grammatical features of the word feature sequence;

[0019] Outputting a sentence representation of the historical customer service conversation text according to the grammatical features of the word feature sequence and the word mask sequence;

[0020] Performing affine transformation and normalization processing on the sentence representation to obtain a label probability distribution of the label output by the complaint prediction model;

[0021] Input the sentence representation into the label confusion layer to calculate the pseudo-label probability distribution;

[0022] Calculating the loss function of the complaint prediction model according to the pseudo-label probability distribution and the label probability distribution;

[0023] The complaint prediction model is iteratively trained so that the calculated loss function meets the set conditions.

[0024] In some embodiments of the present invention, the obtaining of historical customer service conversation texts and labels of historical customer service conversation texts further includes:

[0025] Set the maximum number of words in the historical customer service conversation text;

[0026] Sets the maximum number of characters in a word in the historical customer service conversation text.

[0027] In some embodiments of the present invention, the character feature sequence is a sequence obtained by using only the first 6 output characters of the BERT word segmenter, and the character feature sequence obtained by inputting the encoding layer.

[0028] In some embodiments of the present invention, converting the character feature sequence and the character mask sequence into a set size, and obtaining a word feature sequence based on the character feature sequence and the character mask sequence includes:

[0029] The word masks of the word mask sequence are used as weights, and the features of each character of the word in the word character dimension are averaged to obtain a word feature sequence.

[0030] In some embodiments of the present invention, the graph neural network of the grammatical layer includes two multi-relational graph neural network layers.

[0031] In some embodiments of the present invention, the loss function of the complaint prediction model is KL divergence.

[0032] According to another aspect of the present invention, a complaint prediction method is also provided, comprising:

[0033] Receive the dialogue text entered by the user;

[0034] According to the dialogue text, generating a word token id sequence, a word position id sequence, a word segment id sequence, a word mask sequence and a syntactic dependency graph;

[0035] Input the generated word token id sequence, word position id sequence, word segment id sequence, word mask sequence and syntactic dependency graph of the conversation text into the complaint prediction model established by the complaint prediction model establishment method as described above;

[0036] Obtaining the label probability distribution of the labels predicted by the complaint prediction model;

[0037] The label with the highest probability in the label probability distribution is used as the predicted label of the dialogue text.

[0038] According to another aspect of the present invention, there is also provided a complaint prediction model establishment device, comprising:

[0039] A setting module, used to set a predicted label set of the complaint prediction model;

[0040] An acquisition module is used to acquire historical customer service conversation texts and labels of historical customer service conversation texts;

[0041] A word segmentation module, used to segment the historical customer service conversation text using an LTP language processing tool to generate a word mask sequence and a syntactic dependency graph;

[0042] An embedding module, for generating a word token id sequence, a word position id sequence, a word fragment id sequence, and a word mask sequence based on the historical customer service conversation text using a BERT tokenizer;

[0043] An encoding module, used for inputting the word token id sequence, word position id sequence, and word fragment id sequence into a model encoding layer to obtain a word feature sequence of the user content identified by the historical customer service conversation text in a character dimension;

[0044] a conversion module, used for converting the character feature sequence and the character mask sequence into a set size, and obtaining a word feature sequence based on the character feature sequence and the character mask sequence;

[0045] A grammar module, used for inputting the word feature sequence into the graph neural network of the grammar layer to perceive the grammatical features of the word feature sequence;

[0046] A sentence representation module, configured to output a sentence representation of the historical customer service conversation text according to the grammatical features of the word feature sequence and the word mask sequence;

[0047] A probability distribution module, used to perform affine transformation and normalization processing on the sentence representation to obtain a label probability distribution of the label output by the complaint prediction model;

[0048] A label confusion module, used for inputting the sentence representation into the label confusion layer and calculating the pseudo-label probability distribution;

[0049] A loss calculation module, used to calculate the loss function of the complaint prediction model according to the pseudo-label probability distribution and the label probability distribution;

[0050] The iterative training module is used to iteratively train the complaint prediction model so that the calculated loss function meets the set conditions.

[0051] According to another aspect of the present invention, there is also provided an electronic device, comprising: a processor; and a storage medium on which a computer program is stored, wherein the computer program, when executed by the processor, executes the steps of the complaint prediction model establishment method as described above.

[0052] According to another aspect of the present invention, a storage medium is provided, on which a computer program is stored. When the computer program is executed by a processor, the steps of the complaint prediction model establishment method as described above are executed.

[0053] Compared with the prior art, the advantages of the present invention are:

[0054] The present invention applies cutting-edge deep learning technology to solve the problem of customer complaint warning in the tourism industry, improves the recognition rate of abnormal messages by the warning system, and reduces manual intervention. Assist staff to promptly discover and locate deficiencies in product services, understand user needs and feedback, so as to make corresponding improvements and improvements in a timely manner and improve user retention rates. Among them, in order to capture word-level connections, the present invention uses a multi-relational graph neural network CompGCN, which learns sentence-level grammatical features by converting syntactic dependency trees into heterogeneous graph structures and performing message passing and reduction on the graph. Furthermore, the present invention introduces a label confusion model (LCM) to model the potential connections between labels and labels, and labels and sentences, and automatically learns a soft distribution for each sample to replace the original one-hot distribution for loss calculation, so as to avoid losing a large amount of semantic information contained in the label, causing the model to be unable to handle label confusion or noise. BRIEF DESCRIPTION OF THE DRAWINGS

[0055] The above and other features and advantages of the present invention will become more apparent by describing in detail exemplary embodiments thereof with reference to the attached drawings.

[0056] Figure 1 A flowchart of a method for establishing a complaint prediction model according to an embodiment of the present invention is shown.

[0057] Figure 2 A schematic diagram of a method for establishing a complaint prediction model according to an embodiment of the present invention is shown.

[0058] Figure 3 A module diagram of a complaint prediction model building device according to an embodiment of the present invention is shown.

[0059] Figure 4 A schematic diagram of a computer-readable storage medium in an exemplary embodiment of the present disclosure is schematically shown.

[0060] Figure 5 A schematic diagram of an electronic device in an exemplary embodiment of the present disclosure is schematically shown. DETAILED DESCRIPTION

[0061] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0062] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0063] In order to address the defects of the prior art, capture word-level connections, and avoid losing a large amount of semantic information contained in the labels, which would cause the model to be unable to handle label confusion or noise, thereby realizing complaint prediction for online travel agencies, the present invention provides a complaint prediction method and its model establishment method, device, and related equipment.

[0064] See first Figure 1 and Figure 2 , Figure 1 A schematic diagram of a method for establishing a complaint prediction model according to an embodiment of the present invention is shown. Figure 2 A schematic diagram of a method for establishing a complaint prediction model according to an embodiment of the present invention is shown. The method for establishing a complaint prediction model comprises the following steps:

[0065] Step S101: setting a predicted label set of the complaint prediction model.

[0066] Specifically, the model output space can be set to a collection of seven labels: non-complaint session, poor customer experience, complaint / exposure, evasion, unresolved issues, four arrival categories (no room at the store, no car / air ticket at the venue, no car at the venue, no ticket at the venue), and personal injury.

[0067] Step S102: Obtain historical customer service conversation texts and labels of historical customer service conversation texts.

[0068] Specifically, the customer service conversation texts within a certain period of time can be obtained from the Hive (data warehouse tool) table, and the labeling personnel can label the user's speech with these categories.

[0069] Furthermore, a single historical customer service conversation text may be limited to contain a maximum of 40 words, and each word may contain a maximum of 4 characters, that is, the maximum length of the word sequence is 160 characters.

[0070] Step S103: Use the LTP language processing tool to segment the historical customer service conversation text to generate a word mask sequence and a syntactic dependency graph.

[0071] Specifically, the LTP language processing tool can be used to perform syntactic dependency analysis on the original text after word segmentation, and the resulting syntactic tree can be viewed as a multi-relationship graph.

[0072] LTP defines 15 types of relationships, including subject-verb relationship (SBV), verb-object relationship (VOB), and indirect-object relationship (IOB). The HEAD relationship of the virtual node root pointing to the core word of the sentence is not considered here. The set of the other 14 types of relationships is denoted as R. For each relationship represented by an edge, the relationship type of its reverse edge is introduced, denoted as R. inv , plus the self-connection relationship Τ that adds self-loops to each node in the graph, there are a total of 29 relationship types.

[0073] Step S104: Use the BERT word segmenter to generate a word token id sequence, a word position id sequence, a word fragment id sequence, and a word mask sequence based on the historical customer service conversation text.

[0074] Specifically, the BERT word segmenter can be used as the embedding layer 201, so that the historical customer service conversation text can be input into the embedding layer 201 as a character sequence. The embedding layer 201 first converts the input character sequence into a corresponding index id sequence in the token vocabulary and performs one-hot encoding to obtain in is the character set of the tth word in the sentence, and its length is s t .x t，i is the unique hot vector representation of the i-th character in the t-th word, after the token embedding matrix After transformation, we have:

[0075]

[0076] Similarly, similar operations can be performed on the word position id sequence and word segment id sequence to obtain the position and segment dense vectors and The output of summing up these three embedded features and performing layer normalization is as follows:

[0077]

[0078] Step S105: Input the character token id sequence, character position id sequence, and character fragment id sequence into the model encoding layer to obtain the character feature sequence of the user content identified by the historical customer service conversation text in the character dimension.

[0079] Specifically, the encoding layer 202 may be the encoding layer in BERT. The basic structure of the encoding layer 202 includes a multi-head self-attention layer and a fully connected network layer. Assume that the sequence feature matrix after the embedding layer 201 is A total of L encoder blocks are used, then for l=1, 2, ..., L:

[0080] In the self-attention layer, N attention heads are used, the dimension of the input vector is h, the dimension of each head is d = h / N, and the weights of query, key, value and projection are

[0081]

[0082]

[0083]

[0084] Among them, Q (l，n) , K (l，n) 、V (l，n) They represent the query, key, and value matrices corresponding to the n-th attention head of the l-th layer respectively, and LayerNorm is the layer normalization operation.

[0085] In the fully connected layer, first use The input is affine transformed, processed by a nonlinear activation function, and then Projected to the original dimension, that is

[0086]

[0087]

[0088] Among them, gelu is the Gaussian error linear unit.

[0089] Specifically, for the sake of model performance considerations, only the first 6 blocks of the BERT pre-trained model can be used in the encoding layer 202, and the embedding vector sequence e of the previous step can be input.t，i , and get the encoded word sequence representation.

[0090] Step S106: converting the character feature sequence and the character mask sequence into a set size, and obtaining a word feature sequence based on the character feature sequence and the character mask sequence.

[0091] Specifically, step S106 can transform the shape of the character feature sequence into a normalized size (batch_size)×40×4×h, transform the shape of the character mask sequence into batch_size×40×4, and average the features in the word character dimension with the character mask as the weight to obtain the word feature sequence.

[0092] For example, the character feature sequence processed by the encoding layer 202 In order to obtain the word feature sequence, the character latent vectors of each word are averaged as the representation of the word, that is,

[0093]

[0094] Step S107: input the word feature sequence into the graph neural network of the grammatical layer 203 to perceive the grammatical features of the word feature sequence.

[0095] Step S108: Outputting the sentence representation of the historical customer service conversation text according to the grammatical features of the word feature sequence and the word mask sequence.

[0096] Specifically, the graph neural network of the grammatical layer includes two multi-relational graph neural network CompGCN layers.

[0097] CompGCN is a graph neural network framework that can consider multiple relational information and learn the representation of nodes and relations at the same time. At the same time, the framework designs a vector decomposition operation to effectively alleviate the parameter overload problem. First, initialize a set of basis vectors {v1, v2, ..., v B} and the coefficient matrix {α b，r}, where α b，r Represents the relationship r in the basis vector v b The coefficients on , so the initial representation of the relationship r is z r is a linear combination of all basis vectors:

[0098]

[0099] In order to perform vector operations on the features of nodes and edges, it is necessary to project the relationship representation from the edge space to the node space, that is:

[0100] h r =W rel z r

[0101] W rel is the projection matrix, the transformed h r It is the relationship representation in the node latent space.

[0102] In addition, the following three convolution kernels can be defined for the direction of the edge:

[0103]

[0104] Here, W O , W I and W S They represent the convolution parameters used when the direction of the edge is forward, reverse, and self-loop respectively.

[0105] At the kth layer, the process of CompGCN aggregating neighborhood features to update node representation is as follows:

[0106]

[0107] Among them, the φ function is used to integrate the joint influence of neighbor node u and edge type r. There are mainly the following selection methods:

[0108] φ(x u , z r )=x u -z r

[0109] φ(x u , z r )=x u *z r

[0110] φ(x u , z r )=x u ★z r

[0111] In the formula, x u and z r is the vector representation of node u and edge type r.

[0112] When stacking multiple layers of CompGCN, basis vector combination only occurs in the first layer of the network, after which the relationship representation is updated using a linear transformation as shown below:

[0113]

[0114] Assume that K layers of graph convolution are used to obtain the word sequence representation as Finally, a readout function is used to aggregate the hidden vectors of all nodes to obtain the entire sentence representation m:

[0115] m=Readout(HK )

[0116] Step S109: performing affine transformation and normalization processing on the sentence representation to obtain the label probability distribution of the label output by the complaint prediction model.

[0117] Specifically, assuming that the label set is C, the sentence vector obtained in the previous layer is projected to |C| dimension, and then softmax normalized as the label probability distribution predicted by the model:

[0118]

[0119] Here W output and b output are the weight and bias of the model output layer, respectively.

[0120] Step S110: input the sentence representation to the label confusion layer 204 to calculate the pseudo-label probability distribution.

[0121] Specifically, the label matrix can be initialized First, connect a DNN feedforward network for learning label representation and map it to the same dimension as the sentence representation m,

[0122] V label =g(E label )

[0123] The label matrix V label Interact with the sentence representation m, and the result is normalized after affine transformation, and the label confusion distribution is as follows:

[0124]

[0125] Here W c and b c Represents the weights and biases of the label confusion layer.

[0126] Assuming the true label of the sample is y, define the pseudo label distribution y (s) for:

[0127] y (s) =softmax(αy+y (c) )

[0128] The smoothing parameter α is used to control the relative importance of the confusion distribution and the true distribution. The smoothing parameter α can be set to 0.8, for example, but the present invention is not limited thereto.

[0129] Step S111: Calculate the loss function of the complaint prediction model according to the pseudo-label probability distribution and the label probability distribution.

[0130] Specifically, the loss function used by the model is the KL divergence of the pseudo label distribution with respect to the predicted label distribution, that is,

[0131]

[0132] Step S112: iteratively training the complaint prediction model so that the calculated loss function meets the set conditions.

[0133] Specifically, the AdamW optimization algorithm can be used to minimize the loss, and the final model can be obtained through continuous iterative training.

[0134] In the OTA customer service scenario, the model proposed in this paper classifies the text of customers in IM from the grammatical and semantic levels to determine whether an early warning is needed, and can achieve a prediction accuracy of 92%. Compared with traditional machine learning methods such as SVM, naive Bayes, and simple CNN, RNN and other network modules, it reduces a lot of feature work while obtaining higher performance.

[0135] In the complaint prediction model establishment method provided by the present invention, the cutting-edge technology of deep learning is applied to solve the problem of customer complaint warning in the tourism industry, improve the recognition rate of abnormal messages by the warning system, and reduce manual intervention. Assist staff to promptly discover and locate the deficiencies of product services, understand user needs and feedback, so as to make corresponding improvements and perfections in a timely manner and improve user retention rate. Among them, in order to capture word-level connections, the present invention uses a multi-relationship graph neural network CompGCN, which learns sentence-level grammatical features by converting the syntactic dependency tree into a heterogeneous graph structure and performing message passing and reduction on the graph. Furthermore, the present invention introduces a label confusion model (LCM) to model the potential connections between labels and labels, and labels and sentences, and automatically learns a soft distribution for each sample to replace the original one-hot distribution for loss calculation, so as to avoid losing a large amount of semantic information contained in the label, causing the model to be unable to handle label confusion or noise.

[0136] Furthermore, the present invention also provides a complaint prediction method, which comprises the following steps:

[0137] Receive the dialogue text entered by the user;

[0138] According to the dialogue text, generating a word token id sequence, a word position id sequence, a word segment id sequence, a word mask sequence and a syntactic dependency graph;

[0139] Input the generated word token id sequence, word position id sequence, word segment id sequence, word mask sequence and syntactic dependency graph from the conversation text into the complaint prediction model established by the complaint prediction model establishment method according to any one of claims 1 to 6;

[0140] Obtaining the label probability distribution of the labels predicted by the complaint prediction model;

[0141] The label with the highest probability in the label probability distribution is used as the predicted label of the dialogue text.

[0142] The inference stage does not need to go through the label confusion layer. The label probability distribution predicted by the model is directly used as the output, and the text is judged as the label type with the highest probability. The system then decides whether human intervention is needed based on the prediction results.

[0143] In the complaint prediction method provided by the present invention, the cutting-edge technology of deep learning is applied to solve the problem of customer complaint warning in the tourism industry, improve the recognition rate of abnormal messages by the warning system, and reduce manual intervention. Assist staff to promptly discover and locate the deficiencies of product services, understand user needs and feedback, so as to make corresponding improvements and perfections in a timely manner and improve user retention rate. Among them, in order to capture word-level connections, the present invention uses a multi-relational graph neural network CompGCN, which learns sentence-level grammatical features by converting the syntactic dependency tree into a heterogeneous graph structure and performing message passing and reduction on the graph. Furthermore, the present invention introduces a label confusion model (LCM) to model the potential connections between labels and labels, and labels and sentences, and automatically learns a soft distribution for each sample to replace the original one-hot distribution for loss calculation, so as to avoid losing a large amount of semantic information contained in the label, causing the model to be unable to handle label confusion or noise.

[0144] The above is merely an illustrative description of multiple implementations of the present invention, and the present invention is not limited thereto.

[0145] The present invention also provides a complaint prediction model establishment device. Figure 3 The schematic diagram of the complaint prediction model establishment device according to an embodiment of the present invention is shown. The complaint prediction model establishment device 300 includes a setting module 301, an acquisition module 302, a word segmentation module 303, an embedding module 304, an encoding module 305, a conversion module 306, a grammar module 307, a sentence representation module 308, a probability distribution module 309, a label confusion module 310, a loss calculation module 311 and an iterative training module 312.

[0146] The setting module 301 is used to set the predicted label set of the complaint prediction model;

[0147] The acquisition module 302 is used to acquire the historical customer service conversation text and the label of the historical customer service conversation text;

[0148] The word segmentation module 303 is used to segment the historical customer service dialogue text using an LTP language processing tool to generate a word mask sequence and a syntactic dependency graph;

[0149] The embedding module 304 is used to generate a word token id sequence, a word position id sequence, a word fragment id sequence and a word mask sequence based on the historical customer service conversation text using a BERT word segmenter;

[0150] The encoding module 305 is used to input the word token id sequence, word position id sequence, and word segment id sequence into the model encoding layer to obtain the word feature sequence of the user content identified by the historical customer service conversation text in the character dimension;

[0151] The conversion module 306 is used to convert the character feature sequence and the character mask sequence into a set size, and obtain a word feature sequence based on the character feature sequence and the character mask sequence;

[0152] The grammar module 307 is used to input the word feature sequence into the graph neural network of the grammar layer to perceive the grammatical features of the word feature sequence;

[0153] The sentence representation module 308 is used to output the sentence representation of the historical customer service dialogue text according to the grammatical features of the word feature sequence and the word mask sequence;

[0154] The probability distribution module 309 is used to perform affine transformation and normalization processing on the sentence representation to obtain the label probability distribution of the label output by the complaint prediction model;

[0155] The label confusion module 310 is used to input the sentence representation into the label confusion layer and calculate the pseudo label probability distribution;

[0156] The loss calculation module 311 is used to calculate the loss function of the complaint prediction model according to the pseudo-label probability distribution and the label probability distribution;

[0157] The iterative training module 312 is used to iteratively train the complaint prediction model so that the calculated loss function meets the set conditions.

[0158] In the complaint prediction model establishment device provided by the present invention, the cutting-edge technology of deep learning is applied to solve the problem of customer complaint warning in the tourism industry, improve the recognition rate of abnormal messages by the warning system, and reduce manual intervention. Assist staff to promptly discover and locate the deficiencies of product services, understand user needs and feedback, so as to make corresponding improvements and perfections in a timely manner and improve user retention rate. Among them, in order to capture word-level connections, the present invention uses a multi-relationship graph neural network CompGCN, which learns sentence-level grammatical features by converting the syntactic dependency tree into a heterogeneous graph structure and performing message passing and reduction on the graph. Furthermore, the present invention introduces a label confusion model (LCM) to model the potential connections between labels and labels, and labels and sentences, and automatically learns a soft distribution for each sample to replace the original one-hot distribution for loss calculation, so as to avoid losing a large amount of semantic information contained in the label, causing the model to be unable to handle label confusion or noise.

[0159] Figure 3 The complaint prediction model establishment device provided by the present invention is only schematically shown. Without violating the concept of the present invention, the splitting, merging, and adding of modules are all within the protection scope of the present invention. The complaint prediction model establishment device provided by the present invention can be implemented by software, hardware, firmware, plug-ins, and any combination thereof, and the present invention is not limited thereto.

[0160] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a computer program is stored, and when the program is executed by, for example, a processor, it can implement the steps of the complaint prediction method and / or the prediction model establishment method described in any of the above embodiments. In some possible implementations, various aspects of the present invention can also be implemented in the form of a program product, which includes a program code. When the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps of various exemplary embodiments of the present invention described in the above complaint prediction method and / or prediction model establishment method section of this specification.

[0161] refer to Figure 4 As shown, a program product 400 for implementing the above method according to an embodiment of the present invention is described, which can adopt a portable compact disk read-only memory (CD-ROM) and include program code, and can be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto. In this document, a readable storage medium can be any tangible medium containing or storing a program, which can be used by or in combination with an instruction execution system, an apparatus or a device.

[0162] The program product may use any combination of one or more readable media. The readable medium may be a readable signal medium or a readable storage medium. The readable storage medium may be, for example, but not limited to, an electrical, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or device, or any combination of the above. More specific examples (non-exhaustive list) of readable storage media include: an electrical connection with one or more wires, a portable disk, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM or flash memory), an optical fiber, a portable compact disk read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination of the above.

[0163] The computer readable storage medium may include a data signal propagated in a baseband or as part of a carrier wave, wherein a readable program code is carried. This propagated data signal may take a variety of forms, including but not limited to an electromagnetic signal, an optical signal, or any suitable combination of the above. The readable storage medium may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by an instruction execution system, an apparatus, or a device or used in combination with it. The program code contained on the readable storage medium may be transmitted with any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the above.

[0164] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the tenant computing device, partially on the tenant device, as a separate software package, partially on the tenant computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving remote computing devices, the remote computing device may be connected to the tenant computing device via any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., via the Internet using an Internet service provider).

[0165] In an exemplary embodiment of the present disclosure, an electronic device is also provided, which may include a processor and a memory for storing executable instructions of the processor. The processor is configured to execute the steps of the complaint prediction method and / or prediction model establishment method described in any of the above embodiments by executing the executable instructions.

[0166] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0167] Refer to the following Figure 5 The electronic device 600 according to this embodiment of the present invention is described. Figure 5 The electronic device 600 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0168] like Figure 5 As shown, the electronic device 600 is in the form of a general computing device. The components of the electronic device 600 may include but are not limited to: at least one processing unit 610, at least one storage unit 620, a bus 630 connecting different system components (including the storage unit 620 and the processing unit 610), a display unit 640, etc.

[0169] The storage unit stores program codes, which can be executed by the processing unit 610, so that the processing unit 610 executes the steps of various exemplary embodiments of the present invention described in the complaint prediction method and / or prediction model establishment method section of this specification. For example, the processing unit 610 can execute the following steps: Figures 1 to 3 Follow the steps shown in .

[0170] The storage unit 620 may include a readable medium in the form of a volatile storage unit, such as a random access memory unit (RAM) 6201 and / or a cache memory unit 6202 , and may further include a read-only memory unit (ROM) 6203 .

[0171] The storage unit 620 may also include a program / utility 6204 having a set (at least one) of program modules 6205, such program modules 6205 including but not limited to: an operating system, one or more application programs, other program modules and program data, each of which or some combination may include the implementation of a network environment.

[0172] Bus 630 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0173] The electronic device 600 may also communicate with one or more external devices 700 (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable tenants to interact with the electronic device 600, and / or communicate with any device that enables the electronic device 600 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed through an input / output (I / O) interface 650. In addition, the electronic device 600 may also communicate with one or more networks (e.g., a local area network (LAN), a wide area network (WAN), and / or a public network, such as the Internet) through a network adapter 660. The network adapter 660 may communicate with other modules of the electronic device 600 through a bus 630. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device 600, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems.

[0174] Through the description of the above implementation methods, it is easy for those skilled in the art to understand that the example implementation methods described here can be implemented by software or by combining software with necessary hardware. Therefore, the technical solution according to the implementation methods of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, and includes a number of instructions to enable a computing device (which can be a personal computer, a server, or a network device, etc.) to execute the above-mentioned complaint prediction method and / or prediction model establishment method according to the implementation methods of the present disclosure.

[0175] Compared with the prior art, the advantages of the present invention are:

[0176] The present invention applies cutting-edge deep learning technology to solve the problem of customer complaint warning in the tourism industry, improves the recognition rate of abnormal messages by the warning system, and reduces manual intervention. Assist staff to promptly discover and locate deficiencies in product services, understand user needs and feedback, so as to make corresponding improvements and improvements in a timely manner and improve user retention rates. Among them, in order to capture word-level connections, the present invention uses a multi-relational graph neural network CompGCN, which learns sentence-level grammatical features by converting syntactic dependency trees into heterogeneous graph structures and performing message passing and reduction on the graph. Furthermore, the present invention introduces a label confusion model (LCM) to model the potential connections between labels and labels, and labels and sentences, and automatically learns a soft distribution for each sample to replace the original one-hot distribution for loss calculation, so as to avoid losing a large amount of semantic information contained in the label, causing the model to be unable to handle label confusion or noise.

[0177] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A method for establishing a complaint prediction model, characterized in that: include: Setting a predicted label set of the complaint prediction model; Get the historical customer service conversation text and the label of the historical customer service conversation text; Using an LTP language processing tool to segment the historical customer service conversation text to generate a word mask sequence and a syntactic dependency graph; Using the BERT tokenizer to generate a word token id sequence, a word position id sequence, a word fragment id sequence, and a word mask sequence based on the historical customer service conversation text; Input the word token id sequence, word position id sequence, and word fragment id sequence into the model encoding layer to obtain the word feature sequence of the user content identified by the historical customer service conversation text in the character dimension; converting the character feature sequence and the character mask sequence into a set size, and obtaining a word feature sequence based on the character feature sequence and the character mask sequence; Inputting the word feature sequence into a graph neural network of a grammatical layer to perceive grammatical features of the word feature sequence; Outputting a sentence representation of the historical customer service conversation text according to the grammatical features of the word feature sequence and the word mask sequence; Performing affine transformation and normalization processing on the sentence representation to obtain a label probability distribution of the label output by the complaint prediction model; Input the sentence representation into the label confusion layer to calculate the pseudo-label probability distribution; Calculating the loss function of the complaint prediction model according to the pseudo-label probability distribution and the label probability distribution; The complaint prediction model is iteratively trained so that the calculated loss function meets the set conditions.

2. The method for establishing a complaint prediction model according to claim 1, characterized in that: The method of obtaining the historical customer service conversation text and the tag of the historical customer service conversation text also includes: Set the maximum number of words in the historical customer service conversation text; Sets the maximum number of characters in a word in the historical customer service conversation text.

3. The method for establishing a complaint prediction model according to claim 1, wherein: The character feature sequence is a sequence obtained by using only the first 6 output characters of the BERT word segmenter, and the character feature sequence obtained by inputting the encoding layer.

4. The method for establishing a complaint prediction model according to claim 1, wherein: The converting the character feature sequence and the character mask sequence into a set size, and obtaining a word feature sequence based on the character feature sequence and the character mask sequence comprises: The word masks of the word mask sequence are used as weights, and the features of each character of the word are averaged in the word character dimension to obtain a word feature sequence.

5. The method for establishing a complaint prediction model according to claim 1, wherein: The graph neural network of the grammatical layer includes two multi-relational graph neural network layers.

6. The method for establishing a complaint prediction model according to claim 5, characterized in that: The loss function of the complaint prediction model is KL divergence.

7. A complaint prediction method, characterized in that: include: Receive the dialogue text entered by the user; According to the dialogue text, generating a word token id sequence, a word position id sequence, a word segment id sequence, a word mask sequence and a syntactic dependency graph; Input the generated word token id sequence, word position id sequence, word segment id sequence, word mask sequence and syntactic dependency graph from the conversation text into the complaint prediction model established by the complaint prediction model establishment method according to any one of claims 1 to 6; Obtaining the label probability distribution of the labels predicted by the complaint prediction model; The label with the highest probability in the label probability distribution is used as the predicted label of the dialogue text.

8. A complaint prediction model building device, characterized in that: include: A setting module, used to set a predicted label set of the complaint prediction model; An acquisition module is used to acquire historical customer service conversation texts and labels of historical customer service conversation texts; A word segmentation module, used to segment the historical customer service conversation text using an LTP language processing tool to generate a word mask sequence and a syntactic dependency graph; An embedding module, for generating a word token id sequence, a word position id sequence, a word fragment id sequence, and a word mask sequence based on the historical customer service conversation text using a BERT tokenizer; An encoding module, used for inputting the word token id sequence, word position id sequence, and word fragment id sequence into a model encoding layer to obtain a word feature sequence of the user content identified by the historical customer service conversation text in a character dimension; a conversion module, used for converting the character feature sequence and the character mask sequence into a set size, and obtaining a word feature sequence based on the character feature sequence and the character mask sequence; A grammar module, used for inputting the word feature sequence into the graph neural network of the grammar layer to perceive the grammatical features of the word feature sequence; A sentence representation module, configured to output a sentence representation of the historical customer service conversation text according to the grammatical features of the word feature sequence and the word mask sequence; A probability distribution module, used to perform affine transformation and normalization processing on the sentence representation to obtain a label probability distribution of the label output by the complaint prediction model; A label confusion module, used for inputting the sentence representation into the label confusion layer and calculating the pseudo-label probability distribution; A loss calculation module, used to calculate the loss function of the complaint prediction model according to the pseudo-label probability distribution and the label probability distribution; The iterative training module is used to iteratively train the complaint prediction model so that the calculated loss function meets the set conditions.

9. An electronic device, characterized in that: The electronic device comprises: processor; A storage medium having a computer program stored thereon, wherein the computer program, when executed by the processor, executes the complaint prediction model establishment method as described in any one of claims 1 to 7.

10. A storage medium, characterized in that: The storage medium stores a computer program, and when the computer program is executed by the processor, the method for establishing a complaint prediction model as described in any one of claims 1 to 7 is executed.

Images