Text data processing method, device, electronic device and readable storage medium

Abstract

The present disclosure provides a text data processing method, device, electronic device and computer-readable storage medium, which relate to the field of computer technology. Among them, the text data processing method includes: obtaining intent conversion features associated with the target intent in the dialogue text to be identified; extracting text embedding features related to the target intent in the dialogue text; classifying the intent conversion features and the text embedding features based on a classification model to obtain a first intent classification or a second intent classification. Through the technical solution of the present disclosure, classification is performed based on the intent conversion features and the text embedding features and using a classification model to obtain a first classification intent or a second classification intent with higher accuracy. By introducing the intent flow in the dialogue and the embedded information in the text, the accuracy of the classification result is improved.

Description

Background Art

[0002] With the development of big data and artificial intelligence technology, a large number of user semantic tags can be obtained by text processing of user conversations, and the intention classification of user conversations can be realized based on user semantics.

[0003] In the related art, text mining of user conversations is performed based on a bag-of-words model or a word-embedding model, and a classification model is used to classify the mining results to achieve user semantic intent classification. However, since the bag-of-words model uses quantitative numerical expressions and the word-embedding model is applicable to a limited length of text, the accuracy of intent classification is low.

[0004] It should be noted that the information disclosed in the above background technology section is only used to enhance the understanding of the background of the present disclosure, and therefore may include information that does not constitute the prior art known to ordinary technicians in the field. Summary of the invention

[0005] The purpose of the present disclosure is to provide a text data processing method, device, electronic device and computer-readable storage medium, which can at least to some extent overcome the problem of low accuracy of intent classification obtained by classification models in related technologies.

[0006] Other features and advantages of the present disclosure will become apparent from the following detailed description, or may be learned in part by the practice of the present disclosure.

[0007] According to one aspect of the present disclosure, a text data processing method is provided, comprising: obtaining intent conversion features associated with a target intent in a conversation text to be identified; extracting text embedding features related to the target intent in the conversation text; and classifying the intent conversion features and the text embedding features based on a classification model to obtain a first intent classification or a second intent classification.

[0008] In one embodiment of the present disclosure, the step of obtaining the intent conversion feature associated with the target intent in the conversation text to be identified includes: constructing a directed graph of the intent flow process in the conversation based on the conversation text; and extracting the intent conversion feature associated with the target intent from the directed graph.

[0009] In one embodiment of the present disclosure, constructing a directed graph of the intention flow process in the conversation based on the conversation text includes: determining the intention label corresponding to each sentence in the conversation text; converting the intention label into an intention node; and generating a directed graph of the intention flow process based on the intention node.

[0010] In one embodiment of the present disclosure, the directed graph of the intention flow process generated based on the intention nodes includes: determining the flow direction according to the word order between adjacent intention nodes representing different intentions; generating directed edges between adjacent intention nodes based on the flow direction; and constructing the directed graph based on the multiple intention nodes and the corresponding directed edges.

[0011] In one embodiment of the present disclosure, extracting the intent conversion feature associated with the target intent from the directed graph includes: extracting the intent conversion feature associated with the target intent from the directed graph by graph embedding.

[0012] In one embodiment of the present disclosure, the method of extracting the intent conversion feature associated with the target intent from the directed graph by graph embedding includes: configuring the multiple intent nodes into a node sequence by random walk; performing neural network training on unsupervised tasks of other nodes in the node sequence other than the node of the target intent to obtain the intent conversion feature in vector form.

[0013] In one embodiment of the present disclosure, the performing of neural network training on the unsupervised tasks of other nodes in the node sequence other than the node of the target intent to obtain the intent conversion feature in vector form includes: performing the neural network training on the unsupervised tasks of the other nodes based on a skip-word model; and based on the result of the neural network training, obtaining a graph embedding feature representation of the intent node from a hidden layer of the neural network as the intent conversion feature.

[0014] In one embodiment of the present disclosure, the extracting text embedding features related to the target intent in the conversation text includes: extracting the text embedding features related to the target intent from the conversation text using a pre-trained model.

[0015] In one embodiment of the present disclosure, the pre-trained model is a BERT pre-trained model; the extracting the text embedding features related to the target intent from the conversation text using the pre-trained model includes: based on the target intent, cutting the conversation text into multiple sub-texts; splicing the sub-texts to obtain a spliced ​​text; and inputting the spliced ​​text into the BERT pre-trained model to output the text embedding features.

[0016] In one embodiment of the present disclosure, the dividing the dialogue text into multiple sub-texts based on the target intent includes: extracting a first sentence including the target intent, a second sentence before the first sentence, and a third sentence after the first sentence in the dialogue text; treating the first sentence, the second sentence, and the third sentence as one sub-text, and performing a dividing operation to obtain the multiple sub-texts.

[0017] In one embodiment of the present disclosure, the step of inputting the concatenated text into the BERT pre-trained model to output the text embedding feature includes: inputting the concatenated text into the BERT pre-trained model, the BERT pre-trained model including multiple Transformer layers, using the embedding vector output by the previous Transformer layer as the input vector of the next Transformer layer, and using the hidden state output by the last Transformer layer as the text embedding feature; wherein the text embedding feature includes at least one of semantic features, tone features and voice features.

[0018] In one embodiment of the present disclosure, the classification of the intent conversion feature and the text embedding feature based on the classification model to obtain the first intent classification or the second intent classification includes: acquiring attribute information of the dialogue object; fusing the attribute information, the intent conversion feature and the text embedding feature to generate a feature vector; inputting the feature vector into the classification model to classify the feature vector based on the classification boundary of the classification model to output the first intent classification or the second intent classification.

[0019] In one embodiment of the present disclosure, the attribute information includes risk attribute information of the dialogue object.

[0020] In one embodiment of the present disclosure, the text data processing method also includes: generating training samples based on historical feature vectors; training a logistic regression model based on a minimization processing rule for a loss function of the classification model and the training samples to generate the classification boundary; and generating the classification model based on the classification boundary.

[0021] According to another aspect of the present disclosure, a text data processing device is provided, including: an acquisition module, used to acquire intent conversion features associated with a target intent in a conversation text to be identified; an extraction module, used to extract text embedding features related to the target intent in the conversation text; and a classification module, used to classify the intent conversion features and the text embedding features based on a classification model to obtain a first intent classification or a second intent classification.

[0022] According to another aspect of the present disclosure, an electronic device is provided, comprising: a processor; and a memory for storing executable instructions of the processor; wherein the processor is configured to perform any one of the above-mentioned text data processing methods by executing the executable instructions.

[0023] According to another aspect of the present disclosure, a computer-readable storage medium is provided, on which a computer program is stored, and when the computer program is executed by a processor, any one of the above-mentioned text data processing methods is implemented.

[0024] The text data processing solution provided by the embodiments of the present disclosure mines the intention conversion features and text embedding features in the dialogue text to reflect the intention flow in the dialogue text in the intention conversion features. The text embedding features can not only reflect the semantic understanding of the dialogue text, but also reflect the tone and voice. Therefore, based on the above-mentioned intention conversion features and text embedding features, a classification model is used for classification to obtain a first classification intention or a second classification intention with higher accuracy. By introducing the intention flow in the dialogue and the embedded information in the text, the accuracy of the classification result is improved.

[0025] Furthermore, by mining the intention conversion features in the conversation text, the correlation between the contexts of the conversation text is reflected based on the intention conversion features, thereby improving the recall rate of the classification results.

[0026] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the present disclosure. BRIEF DESCRIPTION OF THE DRAWINGS

[0027] The accompanying drawings herein are incorporated into the specification and constitute a part of the specification, illustrate embodiments consistent with the present disclosure, and together with the specification are used to explain the principles of the present disclosure. Obviously, the accompanying drawings described below are only some embodiments of the present disclosure, and for ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without creative work.

[0028] Figure 1 A schematic diagram showing the structure of a text data processing system in an embodiment of the present disclosure;

[0029] Figure 2 A flowchart of a text data processing method according to an embodiment of the present disclosure is shown;

[0030] Figure 3 A flowchart showing another text data processing method in an embodiment of the present disclosure;

[0031] Figure 4 A schematic diagram showing a directed graph of intent conversion used in a text data processing process in an embodiment of the present disclosure;

[0032] Figure 5 A flowchart showing another text data processing method in an embodiment of the present disclosure;

[0033] Figure 6 A schematic diagram showing text embedding feature extraction used in the text data processing process in an embodiment of the present disclosure;

[0034] Figure 7 A schematic diagram showing a text data processing process in an embodiment of the present disclosure;

[0035] Figure 8 A schematic diagram showing a text data processing device in an embodiment of the present disclosure;

[0036] Fig. 9 A schematic diagram of an electronic device in an embodiment of the present disclosure is shown. DETAILED DESCRIPTION

[0037] Example embodiments will now be described more fully with reference to the accompanying drawings. However, example embodiments can be implemented in a variety of forms and should not be construed as limited to the examples set forth herein; rather, these embodiments are provided so that the disclosure will be more comprehensive and complete and to fully convey the concepts of the example embodiments to those skilled in the art. The described features, structures, or characteristics may be combined in any suitable manner in one or more embodiments.

[0038] In addition, the accompanying drawings are only schematic illustrations of the present disclosure and are not necessarily drawn to scale. The same reference numerals in the figures represent the same or similar parts, and their repeated description will be omitted. Some of the block diagrams shown in the accompanying drawings are functional entities and do not necessarily correspond to physically or logically independent entities. These functional entities can be implemented in software form, or implemented in one or more hardware modules or integrated circuits, or implemented in different networks and / or processor devices and / or microcontroller devices.

[0039] The solution provided in the present application mines the intention conversion features and text embedding features in the dialogue text to reflect the intention flow in the dialogue text in the intention conversion features. The text embedding features can not only reflect the semantic understanding of the dialogue text, but also reflect the tone and voice. Therefore, based on the above-mentioned intention conversion features and text embedding features, a classification model is used for classification to obtain a first classification intention or a second classification intention with higher accuracy. By introducing the intention flow in the dialogue and the embedded information in the text, the accuracy of the classification result is improved.

[0040] To facilitate understanding, several terms involved in this application are first explained below.

[0041] Intention Recognition: It is a common natural language understanding task and a text classification problem. It uses rules, machine learning, deep learning and other methods to label speech into standardized user intent categories.

[0042] Text embedding (Word2Vec): A technique for converting words represented in natural language into vectors or matrices that can be understood by computers. This technique can take into account factors such as the semantics of words (synonyms), the relationship between words in the corpus (context), and the dimensionality of the vector (processing complexity).

[0043] The solution provided in the embodiments of the present application involves computer vision technology such as graphics processing and image recognition, which is specifically explained through the following embodiments.

[0044] Figure 1 A structural schematic diagram of a text data processing system in an embodiment of the present disclosure is shown, including multiple terminals 120 and a server cluster 140 .

[0045] The terminal 120 may be a mobile terminal such as a mobile phone, a game console, a tablet computer, an e-book reader, smart glasses, an MP4 (Moving Picture Experts Group Audio Layer IV) player, a smart home device, an AR (Augmented Reality) device, a VR (Virtual Reality) device, or the terminal 120 may also be a personal computer (PC), such as a laptop computer and a desktop computer.

[0046] The terminal 120 may be installed with an application for providing text data processing.

[0047] The terminal 120 is connected to the server cluster 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0048] The server cluster 140 is a server, or is composed of several servers, or is a virtualization platform, or is a cloud computing service center. The server cluster 140 is used to provide background services for applications that provide text data processing. Optionally, the server cluster 140 undertakes the main computing work, and the terminal 120 undertakes the secondary computing work; or, the server cluster 140 undertakes the secondary computing work, and the terminal 120 undertakes the main computing work; or, the terminal 120 and the server cluster 140 adopt a distributed computing architecture for collaborative computing.

[0049] In some optional embodiments, the server cluster 140 is used to store text data processing information, such as images to be detected, reference image libraries, and images that have completed detection.

[0050] Optionally, the client terminals of the application programs installed in different terminals 120 are the same, or the client terminals of the application programs installed on the two terminals 120 are client terminals of the same type of application programs on different control system platforms. Based on the different terminal platforms, the specific forms of the client terminals of the application programs may also be different, for example, the client terminals of the application programs may be mobile phone client terminals, PC client terminals, or World Wide Web (Web) client terminals.

[0051] Those skilled in the art will appreciate that the number of the terminals 120 may be more or less. For example, there may be only one terminal, or there may be dozens or hundreds of terminals, or a greater number. The embodiment of the present application does not limit the number and device type of the terminals.

[0052] Optionally, the system may also include a management device ( Figure 1 (not shown), the management device is connected to the server cluster 140 via a communication network. Optionally, the communication network is a wired network or a wireless network.

[0053] Optionally, the above-mentioned wireless network or wired network uses standard communication technology and / or protocol. The network is usually the Internet, but it can also be any network, including but not limited to a local area network (LAN), a metropolitan area network (MAN), a wide area network (WAN), a mobile, wired or wireless network, a dedicated network or any combination of a virtual private network). In some embodiments, the data exchanged through the network is represented by technologies and / or formats including Hyper Text Mark-up Language (HTML), Extensible Markup Language (XML), etc. In addition, conventional encryption technologies such as Secure Socket Layer (SSL), Transport Layer Security (TLS), Virtual Private Network (VPN), Internet Protocol Security (IPsec) can also be used to encrypt all or some links. In other embodiments, customized and / or dedicated data communication technologies can also be used to replace or supplement the above-mentioned data communication technologies.

[0054] Below, each step of the text data processing method in this example implementation will be described in more detail with reference to the accompanying drawings and embodiments.

[0055] Figure 2 The method provided by the embodiment of the present disclosure can be executed by any electronic device with computing and processing capabilities, such as Figure 1 The terminal 120 and / or the server cluster 140 in the embodiment. In the following examples, the terminal 120 is used as the execution subject for example description.

[0056] like Figure 2 As shown, the terminal 120 executes the text data processing method, including the following steps:

[0057] Step S202: Acquire the intent conversion feature associated with the target intent in the dialogue text to be recognized.

[0058] Among them, the target intent can be understood as the intent that can be directly extracted from the dialogue text, that is, the intention embodied on the surface of the target text. The target intent can be the first intent classification or the second intent classification in the following step S206. If the target intent is consistent with the following classification result, it indicates that the target intent is a true intention. If the target intent is inconsistent with the following classification result, it indicates that the target intent is a false intention.

[0059] By parsing the contextual relationships in the conversation text, the intention conversion between contexts is determined and the intention conversion features are generated.

[0060] Step S204: extract text embedding features related to the target intent in the conversation text.

[0061] Text embedding features include but are not limited to semantic features, tone features, and voice features.

[0062] Step S206: classify the intent conversion features and the text embedding features based on the classification model to obtain a first intent classification or a second intent classification.

[0063] Among them, the classification model can be generated based on training of a logistic regression model, a support vector machine model, a K nearest neighbor classification model, a Gaussian naive Bayes model, a classification decision tree model, a random forest classification model or a gradient boosting classification tree model.

[0064] In addition, the classification model can be a binary classification model or a multi-classification model with more than two classifications.

[0065] In this embodiment, the intention conversion features and text embedding features in the dialogue text are mined to reflect the intention flow in the dialogue text in the intention conversion features. The text embedding features can not only reflect the semantic understanding of the dialogue text, but also reflect the tone and voice. Therefore, based on the above-mentioned intention conversion features and text embedding features, classification is performed using a classification model to obtain a first classification intention or a second classification intention with higher accuracy. By introducing the intention flow in the dialogue and the embedded information in the text, the accuracy of the classification result is improved.

[0066] In one embodiment of the present disclosure, step S202, obtaining the intent conversion feature associated with the target intent in the dialogue text to be recognized, includes:

[0067] Construct a directed graph of the intent flow process in the conversation based on the conversation text.

[0068] Here, by constructing a directed graph, the user conversation is split into sentences or words and then represented in the form of a graph.

[0069] Intent conversion features associated with the target intent are extracted from the directed graph.

[0070] Among them, a specific implementation method of extracting intent conversion features associated with the target intent from a directed graph includes: extracting intent conversion features associated with the target intent from a directed graph by adopting a graph embedding method.

[0071] Graph embedding is a process of mapping graph data (usually high-dimensional dense matrices) into low-dimensional dense vectors. Graph embedding needs to capture the topological structure of the graph, the relationship between vertices, and other information (such as subgraphs, edges, etc.). Using graph embedding processing can represent more conversation information, so that downstream tasks can achieve better performance.

[0072] After the directed graph is constructed, the "destination intention" node and the intention flow features of the upstream and downstream of the destination node are extracted through graph embedding, so that the intention flow information before and after the node is vectorized.

[0073] In this embodiment, through the construction of a directed graph, the semantic association between user conversation contexts can be reflected, and then the target intent in the conversation process and the transformation factors related to the target intent can be captured based on the associated semantics. Combined with the graph embedding method, the graph data of the directed graph is mapped to vector data to generate intent conversion features in vector form. The data to be classified including the intent conversion features is input into the classification model for classification. The diagram not only improves the accuracy of intent classification, but also helps to improve the recall rate and generalization ability of intent classification.

[0074] like Figure 3 As shown, in one embodiment of the present disclosure, step S202, obtaining the intent conversion feature associated with the target intent in the dialogue text to be recognized specifically includes:

[0075] Step S302, determining the intention label corresponding to each sentence in the dialogue text.

[0076] Among them, automatic speech recognition (Automatic Speech Recognition) can be used to convert the vocabulary content in the user conversation into computer-readable input, such as binary code or character sequence, and generate intent labels based on the binary code or character sequence.

[0077] Step S304, converting the intent label into an intent node.

[0078] Generate a directed graph of the intent flow process based on the intent node, including:

[0079] Step S306, determining the flow direction according to the word order between adjacent intention nodes representing different intentions.

[0080] Step S308, generating directed edges between adjacent intention nodes based on the flow direction.

[0081] Step S310, constructing a directed graph based on multiple intention nodes and corresponding directed edges.

[0082] Among them, since the changes in the user's intentions in the previous and next sentences are related during the conversation, and the changes in intentions may occur repeatedly, a graph is used to represent the flow of user intentions in the entire conversation.

[0083] A directed graph is defined as a set of nodes and edges G = (V, E), where V is a node and E is an edge. In the process of graph construction, the intention label is represented in the form of a node to obtain an intention node. If the user intention is detected to change from A to B during the conversation, there is a directed edge A→B between the intention node A and the intention node B. In this way, the flow process of the user intention in the conversation is represented in the form of a directed graph, such as Figure 4 shown.

[0084] Step S312, configuring multiple intention nodes into a node sequence by random walk.

[0085] Specifically, the first step of the graph embedding process is to generate a node sequence by random walk. Specifically, a biased random walk is used to adjust the weights of breadth-first search (BFS) and depth-first search (DFS) in the sequence generation process through hyperparameters p and q, so as to capture the local features of the intent flow and the deep features of long links as needed.

[0086] Step S314, performing neural network training on the unsupervised tasks of other nodes in the node sequence except the node of the target intent, to obtain the intent conversion feature in the form of a vector.

[0087] Among them, step S314 performs neural network training on unsupervised tasks of other nodes in the node sequence other than the node of the target intent to obtain an implementation method of the intent conversion feature in vector form, including:

[0088] Based on the skip-gram model, neural network training is performed on unsupervised tasks of other nodes. Based on the results of the neural network training, the graph embedding feature representation of the intent node is obtained from the hidden layer of the neural network as the intent conversion feature.

[0089] Specifically, the second step of the graph embedding process is to combine the node sequence generated in the previous step with the skip-gram model (i.e., the skip-gram model, which is a type of Word2Vec model), and use the unsupervised task of predicting other nodes in a sequence with a certain node to perform neural network training. The training goal is to maximize the probability of each node predicting the previous and next nodes in the sequence, and the objective function is:

[0090]

[0091] where u i is the one-hot vector of node i, N s (u i ) is to help u with strategy s i The set of other node vectors in the found sequence.

[0092] Through this unsupervised training, the graph embedding feature representation of the nodes can be obtained from the hidden layer of the neural network.

[0093] In this embodiment, when a conversation has multiple rounds and generates a large amount of text information, the multi-round conversation text is abstracted into different intents through graph embedding, and the flow of intent is further transformed into a directed graph to capture the key information and intent changes in the entire process of the user conversation, so as to determine whether the user's actual intent is consistent with the target intent based on the intent change. Figure 1To improve the accuracy of intent classification.

[0094] In one embodiment of the present disclosure, step S204, extracting text embedding features related to the target intent in the conversation text includes: extracting text embedding features related to the target intent from the conversation text using a pre-trained model.

[0095] The pre-trained model is specifically a general language understanding model that is pre-trained with a large amount of text corpus.

[0096] The following uses the BERT pre-training model as the pre-training model to specifically describe the extraction of text embedding features.

[0097] Specifically, Figure 5 As shown in FIG. 1 , a method for extracting text embedding features related to target intent from conversation text using a pre-trained model includes:

[0098] Step S502: based on the target intent, the dialogue text is divided into a plurality of sub-texts.

[0099] Among them, based on the target intent, the specific implementation method of cutting the dialogue text into multiple sub-texts includes: extracting the first sentence including the target intent, the second sentence before the first sentence and the third sentence after the first sentence in the dialogue text; treating the first sentence, the second sentence and the third sentence as a sub-text, performing a cutting operation to obtain multiple sub-texts.

[0100] like Figure 6 As shown in the figure, [CLS] stands for classification. The BERT model inserts a [CLS] symbol before the text and uses the output vector corresponding to the symbol as the semantic representation of the entire text for text classification. It can be understood as follows: compared with other characters / words already in the text, this symbol without obvious semantic information will more "fairly" integrate the semantic information of each character / word in the text. In addition, a [SEP] symbol is used to separate the two input sentences, and two different text vectors are attached to the two sentences for distinction.

[0101] Step S504, concatenating the sub-texts to obtain a concatenated text.

[0102] Step S506: input the concatenated text into the BERT pre-trained model to output text embedding features.

[0103] A specific implementation method of inputting the concatenated text into the BERT pre-trained model to output text embedding features includes:

[0104] Specifically, in step S506, the spliced text is input into the BERT pre-trained model to output text embedding features, including: inputting the spliced text into the BERT pre-trained model. The BERT pre-trained model includes multiple Transformer layers. The embedding vector output by the previous Transformer layer is used as the input vector of the next Transformer layer, and the hidden state output by the last Transformer layer is used as the text embedding feature.

[0105] Among them, the text embedding features include at least one of semantic features, tone features, and voice features.

[0106] In this embodiment, the text embedding (Word Embedding) features of the statement to be judged are extracted through the BERT (Bidirectional Encoder Representation from Transformers) model. Specifically, since BERT has a limit on the text length and is more suitable for short text classification tasks, the current sentence, the previous sentence, and the next sentence including the target intention can be spliced and input into the BERT model. As Figure 6 shown, the BERT model includes the first layer 602, the second layer 604, and the Nth layer 606, etc. Each layer is a Transformer layer. Through multiple Transformer layers, each layer receives a list of embeddings of a group of word vectors output by the previous layer, generates the same number of embeddings as the output, and continues to be input into the next layer. Finally, the multi-dimensional embedding vector corresponding to [CLS] of the last layer is obtained, and then through the classifier 608 and the prediction module 610, the text embedding feature extraction work is completed.

[0107] For example, 12 Transformer layers can correspondingly output a 768-dimensional embedding vector.

[0108] Adopting this text embedding feature extraction scheme can retain all texts, including various modal particles and conjunctions such as "um", "ah", "ba", etc., so that the model may learn more delicate tone and voice information, and improve the classification accuracy of the model by enhancing the text embedding features.

[0109] In this embodiment, a large-scale pre-trained language model is used to achieve semantic understanding in the user's speech. With the Transformer network as the basic structure of the model, pre-training is achieved through the masked language model and the next sentence prediction task on a large-scale unsupervised corpus. Compared with the bag-of-words model in the related technology, the text embedding features generated in this way can comprehensively and accurately reflect the actual intention of the user.

[0110] In one embodiment of the present disclosure, classifying the intent conversion feature and the text embedding feature based on the classification model to obtain the first intent classification or the second intent classification includes:

[0111] Get the property information of the dialog object.

[0112] Among them, the attribute information of the dialogue object mainly includes the risk attributes of the dialogue object, including but not limited to age, gender, consumption behavior profile, historical credit behavior and user risk rating.

[0113] Attribute information, intent conversion features and text embedding features are fused to generate a feature vector.

[0114] The feature vector is input into the classification model to classify the feature vector based on the classification boundary of the classification model to output the first intent classification or the second intent classification

[0115] In this embodiment, by introducing the user's attribute information, the difference in emotions of different users when expressing the same intention can be effectively fitted based on the user's attribute information, so as to take into account both the text features and user features of the conversation in classification and recognition. Based on business experience, in the post-loan collection scenario, users show some false intentions for the purpose of disguise, perfunctory, deception, etc., mainly because they do not have the ability or willingness to repay. Adding user characteristics that can characterize their repayment ability and willingness, such as consumption capacity, risk rating and other information, has a significant benefit in identifying false intentions.

[0116] like Figure 7 As shown, the directed graph 702 is transformed into an intent conversion vector 708 through vector conversion, and an intent conversion feature 712 related to the target intent is extracted.

[0117] The text feature 704 is input into the BERT model 710 and the text embedding feature 714 is output.

[0118] The intention conversion feature 712 is combined with the text embedding feature 714 and the user attribute information 706, and a binary classification process is performed through a connection module 716 to obtain a classification result 718.

[0119] In one embodiment of the present disclosure, the text data processing method also includes: generating training samples based on historical feature vectors; fitting a logistic regression model with the training samples based on a minimization processing rule for a loss function of the classification model to generate classification boundaries; and generating the classification model based on the classification boundaries.

[0120] In this example, the classification model is essentially a supervised binary classification. Based on the above steps, we obtain the graph embedding features of the user intention flow of each sample, the text embedding features of the user's speech, and some attribute features of the user itself. The above features are input into the logistic regression model for fitting.

[0121] The text data processing method described in this disclosure does not require excessive semantic analysis and expert experience, and realizes a fine-grained tone / voice recognition. Compared with the traditional expert rule method, this method does not require manual reading of a large amount of corpus and grammatical analysis, and is no longer limited to the text features that can be captured by manual experience, and can capture more implicit information.

[0122] The data processing scheme of the present disclosure is described in detail below, with "promising to repay is a true intention" as the first intention classification and "promising to repay is a false intention" as the second intention classification.

[0123] For this technical solution, the experiment takes the distinction between the user's true "promise to repay" intention and the false "promise to repay" intention as an example. Generally speaking, if the "promise to repay" intention is true, the user will complete the repayment within one week after the commitment; if the "promise to repay" intention is false, the user will not complete the repayment within one week after the commitment, or even completely not repay the loan for a longer period of time.

[0124] Taking a multi-round conversation of a user as a sample, the dataset contains the information shown in Table 1.

[0125] Table 1

[0126]

[0127] Each sentence has a sentence intent label, as shown in Table 2.

[0128] Table 2

[0129]

[0130] 1. Model effect evaluation

[0131] (1) Evaluation indicators

[0132] For this type of supervised binary classification problem, the following evaluation indicators are used:

[0133] AUC: AUC refers to the area under the ROC curve, which is obtained based on the true category and predicted probability of the sample. The x-axis is the true positive rate and the y-axis is the false positive rate. It is suitable for binary classification problems. The AUC value range is between 0.5 and 1. The closer the AUC is to 1, the higher the authenticity of the detection method.

[0134] KS: KS is an indicator used to measure the risk differentiation ability of the model. It calculates the difference between the cumulative distribution of good and bad samples. The greater the cumulative difference between good and bad samples, the greater the KS indicator, and the stronger the risk differentiation ability of the model.

[0135] (2) The evaluation results are shown in Table 3.

[0136] Table 3

[0137]

[0138] Experiments have shown that this model has good effect.

[0139] It should be noted that the above figures are only schematic illustrations of the processes included in the method according to an exemplary embodiment of the present invention, and are not intended to be limiting. It is easy to understand that the processes shown in the above figures do not indicate or limit the time sequence of these processes. In addition, it is also easy to understand that these processes can be performed synchronously or asynchronously, for example, in multiple modules.

[0140] It will be appreciated by those skilled in the art that various aspects of the present invention may be implemented as a system, method or program product. Therefore, various aspects of the present invention may be specifically implemented in the following forms, namely: a complete hardware implementation, a complete software implementation (including firmware, microcode, etc.), or a combination of hardware and software, which may be collectively referred to herein as a "circuit", "module" or "system".

[0141] Refer to the following Figure 8 The text data processing apparatus 800 according to this embodiment of the present invention will be described. Figure 8 The text data processing device 800 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0142] The text data processing device 800 is in the form of a hardware module. The components of the text data processing device 800 may include but are not limited to: an acquisition module 802 for acquiring an intent conversion feature associated with a target intent in a dialogue text to be recognized; an extraction module 804 for extracting text embedding features related to the target intent in the dialogue text; and a classification module 806 for classifying the intent conversion feature and the text embedding feature based on a classification model to obtain a first intent classification or a second intent classification.

[0143] Refer to the following Fig. 9 An electronic device 900 according to this embodiment of the present invention will be described. Fig. 9 The electronic device 900 shown is only an example and should not bring any limitation to the functions and scope of use of the embodiments of the present invention.

[0144] like Fig. 9 As shown, the electronic device 900 is in the form of a general computing device. The components of the electronic device 900 may include but are not limited to: at least one processing unit 910, at least one storage unit 920, and a bus 930 connecting different system components (including the storage unit 920 and the processing unit 910).

[0145] The storage unit stores program codes, which can be executed by the processing unit 910, so that the processing unit 910 performs the steps of various exemplary embodiments of the present invention described in the above “Exemplary Method” section of this specification. For example, the processing unit 910 can perform the following steps: Figure 1 Steps S202, S204 and S206 shown in, and other steps defined in the text data processing method of the present disclosure.

[0146] The storage unit 920 may include a readable medium in the form of a volatile storage unit, such as a random access storage unit (RAM) 9201 and / or a cache storage unit 9202 , and may further include a read-only storage unit (ROM) 9203 .

[0147] The storage unit 920 may also include a program / utility 9204 having a set (at least one) of program modules 9205, such program modules 9205 including but not limited to: an operating system, one or more application programs, other program modules, and program data, each of which or some combination may include an implementation of a network environment.

[0148] Bus 930 may represent one or more of several types of bus structures, including a memory unit bus or memory unit controller, a peripheral bus, an accelerated graphics port, a processing unit, or a local bus using any of a variety of bus architectures.

[0149] The electronic device 900 may also communicate with one or more external devices (e.g., keyboards, pointing devices, Bluetooth devices, etc.), may also communicate with one or more devices that enable a user to interact with the electronic device, and / or may communicate with any device that enables the electronic device 900 to communicate with one or more other computing devices (e.g., routers, modems, etc.). Such communication may be performed via an input / output (I / O) interface 950. Furthermore, the electronic device 900 may also communicate with one or more networks (e.g., local area networks (LANs), wide area networks (WANs), and / or public networks, such as the Internet) via a network adapter 940. As shown, the network adapter 940 communicates with other modules of the electronic device 900 via a bus 930. It should be understood that, although not shown in the figure, other hardware and / or software modules may be used in conjunction with the electronic device, including but not limited to: microcode, device drivers, redundant processing units, external disk drive arrays, RAID systems, tape drives, and data backup storage systems, etc.

[0150] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a terminal device, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0151] In an exemplary embodiment of the present disclosure, a computer-readable storage medium is also provided, on which a program product capable of implementing the above method of the present specification is stored. In some possible implementations, various aspects of the present invention may also be implemented in the form of a program product, which includes a program code, and when the program product is run on a terminal device, the program code is used to enable the terminal device to execute the steps according to various exemplary embodiments of the present invention described in the above "Exemplary Method" section of the present specification.

[0152] The program product for implementing the above method according to an embodiment of the present invention may adopt a portable compact disk read-only memory (CD-ROM) and include program code, and may be run on a terminal device, such as a personal computer. However, the program product of the present invention is not limited thereto, and in this document, a readable storage medium may be any tangible medium containing or storing a program, which may be used by or in combination with an instruction execution system, apparatus, or device.

[0153] Computer readable signal media may include data signals propagated in baseband or as part of a carrier wave, in which readable program code is carried. Such propagated data signals may take a variety of forms, including but not limited to electromagnetic signals, optical signals, or any suitable combination of the above. Readable signal media may also be any readable medium other than a readable storage medium, which may send, propagate, or transmit a program for use by or in conjunction with an instruction execution system, apparatus, or device.

[0154] The program code embodied on the readable medium may be transmitted using any appropriate medium, including but not limited to wireless, wired, optical cable, RF, etc., or any suitable combination of the foregoing.

[0155] Program code for performing the operations of the present invention may be written in any combination of one or more programming languages, including object-oriented programming languages ​​such as Java, C++, etc., and conventional procedural programming languages ​​such as "C" or similar programming languages. The program code may be executed entirely on the user computing device, partially on the user device, as a separate software package, partially on the user computing device and partially on a remote computing device, or entirely on a remote computing device or server. In cases involving a remote computing device, the remote computing device may be connected to the user computing device through any type of network, including a local area network (LAN) or a wide area network (WAN), or may be connected to an external computing device (e.g., through the Internet using an Internet service provider).

[0156] It should be noted that, although several modules or units of the device for action execution are mentioned in the above detailed description, this division is not mandatory. In fact, according to the embodiments of the present disclosure, the features and functions of two or more modules or units described above can be embodied in one module or unit. On the contrary, the features and functions of one module or unit described above can be further divided into multiple modules or units to be embodied.

[0157] In addition, although the steps of the method in the present disclosure are described in a specific order in the drawings, this does not require or imply that the steps must be performed in this specific order, or that all the steps shown must be performed to achieve the desired results. Additionally or alternatively, some steps may be omitted, multiple steps may be combined into one step, and / or one step may be decomposed into multiple steps, etc.

[0158] Through the description of the above implementation, it is easy for those skilled in the art to understand that the example implementation described here can be implemented by software, or by software combined with necessary hardware. Therefore, the technical solution according to the implementation of the present disclosure can be embodied in the form of a software product, which can be stored in a non-volatile storage medium (which can be a CD-ROM, a USB flash drive, a mobile hard disk, etc.) or on a network, including several instructions to enable a computing device (which can be a personal computer, a server, a mobile terminal, or a network device, etc.) to execute the method according to the implementation of the present disclosure.

[0159] Those skilled in the art will readily appreciate other embodiments of the present disclosure after considering the specification and practicing the invention disclosed herein. This application is intended to cover any modification, use or adaptation of the present disclosure, which follows the general principles of the present disclosure and includes common knowledge or customary techniques in the art that are not disclosed in the present disclosure. The specification and examples are intended to be exemplary only, and the true scope and spirit of the present disclosure are indicated by the appended claims.

Claims

1. A text data processing method, characterized in that: include: Acquiring an intention conversion feature associated with a target intention in a dialogue text to be identified, including: constructing a directed graph of an intention flow process in a dialogue according to the dialogue text; extracting the intention conversion feature associated with the target intention from the directed graph; Extracting text embedding features related to the target intent from the conversation text; The intent conversion feature and the text embedding feature are classified based on a classification model to obtain a first intent classification or a second intent classification.

2. The text data processing method according to claim 1, characterized in that: The step of constructing a directed graph of the intention flow process in the conversation according to the conversation text includes: Determine the intent label corresponding to each sentence in the conversation text; Convert the intent label into an intent node; A directed graph of the intention flow process is generated based on the intention nodes.

3. The text data processing method according to claim 2, characterized in that: The generating of the directed graph of the intention transfer process based on the intention node includes: Determining the flow direction according to the word order between adjacent intention nodes representing different intentions; Generating directed edges between adjacent intention nodes based on the flow direction; The directed graph is constructed based on the multiple intention nodes and the corresponding directed edges.

4. The text data processing method according to claim 2, characterized in that: The extracting the intention conversion feature associated with the target intention from the directed graph comprises: The intent conversion features associated with the target intent are extracted from the directed graph in a graph embedding manner.

5. The text data processing method according to claim 4, characterized in that: The extracting the intent conversion feature associated with the target intent from the directed graph by adopting a graph embedding method includes: The plurality of intention nodes are configured into a node sequence by using a random walk method; A neural network training is performed on the unsupervised tasks of other nodes in the node sequence except the node of the target intent to obtain the intent conversion feature in the form of a vector.

6. The text data processing method according to claim 5, characterized in that: The performing of neural network training on the unsupervised task of other nodes in the node sequence other than the node of the target intent to obtain the intent conversion feature in vector form includes: Performing the neural network training on the unsupervised tasks of the other nodes based on the skip-word model; Based on the result of the neural network training, a graph embedding feature representation of the intent node is obtained from the hidden layer of the neural network to serve as the intent conversion feature.

7. The text data processing method according to claim 1, characterized in that: The extracting of text embedding features related to the target intent in the conversation text comprises: A pre-trained model is used to extract the text embedding features related to the target intent from the conversation text.

8. The text data processing method according to claim 7, characterized in that: The pre-training model is a BERT pre-training model; The extracting the text embedding features related to the target intent from the conversation text using the pre-trained model comprises: Based on the target intent, the dialogue text is divided into a plurality of sub-texts; Performing a splicing operation on the sub-texts to obtain a spliced ​​text; The concatenated text is input into the BERT pre-trained model to output the text embedding features.

9. The text data processing method according to claim 8, characterized in that: The step of cutting the dialogue text into a plurality of sub-texts based on the target intent includes: Extracting a first sentence including the target intention, a second sentence before the first sentence, and a third sentence after the first sentence from the conversation text; The first sentence, the second sentence and the third sentence are taken as one subtext, and a segmentation operation is performed to obtain the multiple subtexts.

10. The text data processing method according to claim 8, characterized in that: The step of inputting the concatenated text into the BERT pre-trained model to output the text embedding feature comprises: Input the concatenated text into the BERT pre-trained model, the BERT pre-trained model includes multiple Transformer layers, use the embedding vector output by the previous Transformer layer as the input vector of the next Transformer layer, and use the hidden state output by the last Transformer layer as the text embedding feature; The text embedding feature includes at least one of a semantic feature, a tone feature and a voice feature.

11. The text data processing method according to any one of claims 1 to 10, characterized in that: The classifying the intent conversion feature and the text embedding feature based on the classification model to obtain a first intent classification or a second intent classification includes: Get the attribute information of the dialog object; fusing the attribute information, the intent conversion feature and the text embedding feature to generate a feature vector; The feature vector is input into the classification model to classify the feature vector based on a classification boundary of the classification model to output the first intent classification or the second intent classification.

12. The text data processing method according to claim 11, characterized in that: The attribute information includes risk attribute information of the dialogue object.

13. The text data processing method according to claim 11, characterized in that: The text data processing method further comprises: Generate training samples based on historical feature vectors; Training a logistic regression model based on a minimization processing rule for a loss function of the classification model and the training samples to generate the classification boundary; The classification model is generated based on the classification boundary.

14. A text data processing device, characterized in that: include: An acquisition module is used to acquire the intention conversion feature associated with the target intention in the dialogue text to be identified, including: constructing a directed graph of the intention flow process in the dialogue according to the dialogue text; extracting the intention conversion feature associated with the target intention from the directed graph; An extraction module, used for extracting text embedding features related to the target intent in the conversation text; A classification module is used to classify the intent conversion feature and the text embedding feature based on a classification model to obtain a first intent classification or a second intent classification.

15. An electronic device, characterized in that: include: processor; as well as A memory, configured to store executable instructions of the processor; The processor is configured to execute the text data processing method according to any one of claims 1 to 13 by executing the executable instructions.

16. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the text data processing method according to any one of claims 1 to 13 is implemented.

Images