Data processing method and device based on deep learning, equipment and storage medium

Abstract

The invention provides a deep learning-based data processing method and apparatus, an electronic device and a storage medium. The method comprises the steps of obtaining to-be-predicted text data, and user information and package information corresponding to the to-be-predicted text data; and performing classification prediction processing on the to-be-predicted text data, the user information and the package information based on a trained judgment model to obtain a classification prediction processing result, the judgment model comprising a first network model, a second network model, and a full connection and Softmax layer connected with the first network model and the second network model. The problem that analysis and processing of customer satisfaction only pay attention to complaint content of customers and neglect personal information and package types of the customers in related technologies is solved, the customer satisfaction prediction accuracy is improved, corresponding customer retention strategies can be formulated in advance, the customer loss rate is reduced, and the service quality is improved.

Description

Technical Field

[0001] The present invention relates to data processing methods and technologies, and in particular to a data processing method, device, electronic device and computer-readable storage medium based on deep learning. Background Art

[0002] In today's service industry, customer churn is a major problem faced by enterprises. After losing customers, they need to find new customers, which increases costs. Therefore, it is of vital importance for enterprises to accurately predict customer satisfaction in order to retain customers.

[0003] In the related art, the customer churn prediction method usually relies on statistical models to predict customer behavior, or uses user satisfaction analysis models to determine based on customer complaints. However, although statistical models can predict customer behavior, it is difficult to capture other information, and through user satisfaction analysis models, it is difficult to take into account the customer's own objective factors, which affects satisfaction, reduces the accuracy of predicting customer satisfaction, and leads to customer churn. Summary of the invention

[0004] The present invention provides a data processing method, device, electronic device and computer-readable storage medium based on deep learning, so as to at least solve the problem in the related art that it is difficult to take into account the objective factors of the customer himself when predicting customer satisfaction, resulting in reduced accuracy in predicting customer satisfaction and customer loss. The technical solution of the present invention is as follows:

[0005] According to a first aspect of an embodiment of the present invention, a data processing method based on deep learning is provided, comprising:

[0006] Acquire text data to be predicted, and user information and package information corresponding to the text data to be predicted;

[0007] Based on the trained judgment model, the text data to be predicted, the user information and the package information are classified and predicted to obtain a classification prediction result, wherein the judgment model includes: a first network model, a second network model, and a fully connected and softmax layer connected to the first network model and the second network model.

[0008] Optionally, the classification prediction processing is performed on the text data to be predicted, the user information and the package information based on the trained judgment model to obtain the classification prediction processing result, including:

[0009] Based on the first network model in the judgment model, the text data to be predicted and the user information are vector-fused to obtain a first fused vector; and based on the second network model in the judgment model, the text data to be predicted and the package information are vector-fused to obtain a second fused vector;

[0010] The first fusion vector and the second fusion vector are input into the full connection and Softmax layers in the judgment model for vector normalization processing to obtain a classification prediction processing result.

[0011] Optionally, the first network model includes: a first BERT layer, a first BiLSTM layer, and a first multi-head attention layer connected in series in sequence; the second network model includes: a second BERT layer, a second BiLSTM layer, and a second multi-head attention layer connected in series in sequence;

[0012] The step of performing vector fusion on the text data to be predicted and the user information based on the first network model in the judgment model to obtain a first fusion vector includes:

[0013] Input the text data to be predicted into the first BERT layer for feature vector learning, and input the output feature vector into the first BiLSTM layer for learning iteration, and fuse the obtained first iteration result with the vector of the user information in the first multi-head attention layer to obtain a first fused vector;

[0014] The step of performing vector fusion on the text data to be predicted and the package information based on the second network model in the judgment model to obtain a second fusion vector includes:

[0015] The text data to be predicted is input into the second BERT layer for feature vector learning, and the output feature vector is input into the second BiLSTM layer for learning iteration. The second iteration result obtained is fused with the vector of the package information in the second multi-head attention layer to obtain a second fused vector.

[0016] Optionally, inputting the text data to be predicted into the first BERT layer for feature vector learning, and inputting the output feature vector into the first BiLSTM layer for learning iteration, comprises:

[0017] Input the text data to be predicted into the first BERT layer for vector conversion processing; extract the word vectors output by the first BERT layer as the first group of word vectors for the text data to be predicted; input the first group of word vectors into the first BiLSTM layer for continuous learning iteration to obtain first forward hidden layer features and first reverse hidden layer features; concatenate the first forward hidden layer features and the second reverse hidden layer features to obtain the hidden layer features of the first BiLSTM layer;

[0018] The step of inputting the text data to be predicted into the second BERT layer for feature vector learning, and inputting the output feature vector into the second BiLSTM layer for learning iteration includes:

[0019] The text data to be predicted is input into the second BERT layer for vector conversion processing; the word vectors output by the second BERT layer are extracted as the second group of word vectors for the text data to be predicted, and the second group of word vectors are input into the second BiLSTM layer for continuous learning iteration to obtain the second forward hidden layer features and the second reverse hidden layer features; the second forward hidden layer features and the second reverse hidden layer features are concatenated to obtain the hidden layer features of the second BiLSTM layer.

[0020] Optionally, fusing the obtained first iteration result with the user information in the first multi-head attention layer to obtain a first fusion vector includes: performing vector splitting and mapping of the vector of hidden layer features output by the first BiLSTM layer and the user vector of the user information according to the number of heads, and inputting them into the first multi-head attention layer for fusion; learning different weights in each subspace, and performing vector splicing of the attention fusion results obtained in each subspace to obtain a first fusion vector output by the first multi-head attention layer;

[0021] The second iterative result is fused with the user information in the second multi-head attention layer to obtain a second fusion vector, including: splitting and mapping the vector of the hidden layer features output by the second BiLSTM layer and the package vector of the package information according to the number of heads, and inputting them into the second multi-head attention layer for fusion; learning different weights in each subspace, and vectorizing the attention fusion results obtained in each subspace to obtain the second fusion vector output by the second multi-head attention layer.

[0022] Optionally, the first fusion vector and the second fusion vector are input into the full connection and Softmax layer for vector normalization processing to obtain a classification prediction processing result, including:

[0023] Inputting the first fused vector and the second fused vector into the fully connected and Softmax layers for vector concatenation;

[0024] The concatenated fusion vector is normalized to obtain the classification prediction processing result.

[0025] Optionally, the method further includes: pre-training the judgment model in the following manner:

[0026] Acquire a data set, the data set comprising: a training set and a validation set of the training set;

[0027] Preprocessing the training set;

[0028] Get the user dictionary and package dictionary of the complaining user;

[0029] Convert the user dictionary into a user vector with a length of BiLSTM dimension and a width of the number of users, and convert the package dictionary into a package vector with a length of BiLSTM dimension and a width of the number of packages;

[0030] The preprocessed training set, the user vector and the package vector are input into the judgment model for training. When the judgment model is trained, the user vector is randomly initialized, and the user corresponding to the complaint data is added to the training in the row where the user vector is located; and the package vector is randomly initialized, and the package corresponding to the complaint data is added to the training in the row where the package vector is located, so as to obtain the classification prediction training result; based on the classification prediction training result, the verification set is compared, and the loss value is determined according to the comparison result. Based on the loss value, the parameters of the judgment model are updated through the back propagation mechanism, and after multiple iterations, the judgment model converges to obtain a trained judgment model.

[0031] According to a second aspect of an embodiment of the present invention, there is provided a data processing device based on deep learning, comprising:

[0032] An acquisition module, used to acquire text data to be predicted, and user information and package information corresponding to the text data to be predicted;

[0033] The prediction module is used to perform classification prediction processing on the text data to be predicted, the user information and the package information based on the trained judgment model to obtain the classification prediction processing result, wherein the judgment model includes: a first network model, a second network model, and a fully connected and Softmax layer connected to the first network model and the second network model.

[0034] Optionally, the prediction module includes:

[0035] a first fusion module, configured to perform vector fusion on the text data to be predicted and the user information based on the first network model in the judgment model to obtain a first fusion vector, and

[0036] A second fusion module is used to perform vector fusion on the text data to be predicted and the package information based on the second network model in the judgment model to obtain a second fusion vector;

[0037] The normalization processing module is used to input the first fusion vector and the second fusion vector into the full connection and Softmax layers in the judgment model to perform vector normalization processing to obtain a classification prediction processing result.

[0038] Optionally, the first network model includes: a first BERT layer, a first BiLSTM layer, and a first multi-head attention layer connected in series in sequence; the second network model includes: a second BERT layer, a second BiLSTM layer, and a second multi-head attention layer connected in series in sequence;

[0039] The first fusion module includes:

[0040] A first iteration module, used for inputting the text data to be predicted into the first BERT layer for feature vector learning, and inputting the output feature vector into the first BiLSTM layer for learning iteration;

[0041] A first vector fusion module, configured to fuse the first iteration result obtained by the first iterative learning module with the vector of the user information in the first multi-head attention layer to obtain a first fused vector;

[0042] The second fusion module includes:

[0043] A second iteration module, used for inputting the text data to be predicted into the second BERT layer for feature vector learning, and inputting the output feature vector into the second BiLSTM layer for learning iteration;

[0044] The second vector fusion module is used to fuse the second iteration result obtained by the second iterative learning module with the vector of the package information in the second multi-head attention layer to obtain a second fused vector.

[0045] Optionally, the first iteration module includes:

[0046] A first conversion module, used for inputting the text data to be predicted into the first BERT layer for vector conversion processing;

[0047] A first extraction module, used to extract the word vectors output by the first BERT layer as the first group of word vectors of the text data to be predicted;

[0048] A first iterative learning module, used for inputting the first group of word vectors into the first BiLSTM layer for continuous learning iteration to obtain a first forward hidden layer feature and a first reverse hidden layer feature;

[0049] A first concatenation module, used for concatenating the first forward hidden layer features and the second reverse hidden layer features to obtain the hidden layer features of the first BiLSTM layer;

[0050] The second iteration module includes:

[0051] A second conversion module, used for inputting the text data to be predicted into the second BERT layer for vector conversion processing;

[0052] A second extraction module, used to extract the word vectors output by the second BERT layer as a second set of word vectors for the text data to be predicted;

[0053] A second iterative learning module, used for inputting the second group of word vectors into the second BiLSTM layer for continuous learning iteration to obtain a second forward hidden layer feature and a second reverse hidden layer feature;

[0054] The second concatenation module is used to concatenate the second forward hidden layer features and the second reverse hidden layer features to obtain the hidden layer features of the second BiLSTM layer.

[0055] Optionally, the first vector fusion module includes:

[0056] A first splitting and mapping module, used for performing vector splitting and mapping of the vector of the hidden layer features output by the first BiLSTM layer and the user vector of the user information according to the number of heads;

[0057] A first multi-attention fusion module, used for inputting the mapping result of the first split mapping module into the first multi-head attention layer for fusion;

[0058] A first multi-attention concatenation module is used to learn different weights in each subspace and concatenate the attention fusion results obtained in each subspace to obtain a first fusion vector output by the first multi-head attention layer;

[0059] The first vector fusion module includes:

[0060] A first splitting and mapping module is used to perform vector splitting and mapping on the vector of the hidden layer features output by the second BiLSTM layer and the package vector of the package information according to the number of heads;

[0061] A second multi-attention fusion module, used to input the mapping result of the second split mapping module into the second multi-head attention layer for fusion;

[0062] The second multi-attention splicing module is used to learn different weights in each subspace, and vector-spin the attention fusion structures obtained from each subspace to obtain a second fusion vector output by the second multi-head attention layer.

[0063] Optionally, the normalization processing module includes:

[0064] A fusion vector splicing module, used for inputting the first fusion vector and the second fusion vector into the full connection and Softmax layer for vector splicing;

[0065] The normalization processing submodule is used to normalize the concatenated fusion vector to obtain the classification prediction processing result.

[0066] Optionally, the device further comprises: a training module, configured to pre-train the judgment model in the following manner:

[0067] A data set acquisition module, used to acquire a data set, the data set comprising: a training set, and a validation set of the training set;

[0068] A preprocessing module, used for preprocessing the training set;

[0069] Dictionary acquisition module, used to obtain the user dictionary and package dictionary of the complaining user;

[0070] A conversion module, used to convert the user dictionary into a user vector with a length of BiLSTM dimension and a width of the number of users, and to convert the package dictionary into a package vector with a length of BiLSTM dimension and a width of the number of packages;

[0071] A training module is used to input the preprocessed training set, the user vector and the package vector into the judgment model for training. When training the judgment model, the user vector is randomly initialized, and the user corresponding to the complaint data is added to the training in the row where the user vector is located; and the package vector is randomly initialized, and the package corresponding to the complaint data is added to the training in the row where the package vector is located to obtain a classification prediction training result; based on the classification prediction training result, the training result is compared with the validation set, and a loss value is determined according to the comparison result. Based on the loss value, the parameters of the judgment model are updated through a back propagation mechanism, and after multiple iterations, the judgment model converges to obtain a trained judgment model.

[0072] According to a third aspect of an embodiment of the present invention, there is provided an electronic device, including:

[0073] processor;

[0074] a memory for storing instructions executable by the processor;

[0075] Wherein, the processor is configured to execute the instructions to implement the data processing method based on deep learning as described above.

[0076] According to a fourth aspect of an embodiment of the present invention, a computer-readable storage medium is provided. When instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to perform the data processing method based on deep learning as described above.

[0077] According to a fifth aspect of an embodiment of the present invention, there is provided a computer program product, comprising a computer program or instructions, which, when executed by a processor of an electronic device, implements the deep learning-based data processing method as described above.

[0078] The technical solution provided by the embodiments of the present invention brings at least the following beneficial effects:

[0079] In an embodiment of the present invention, text data to be predicted, as well as user information and package information corresponding to the text data to be predicted are obtained; the text data to be predicted, the user information and the package information are classified and predicted based on a trained judgment model to obtain a classification prediction result, and the judgment model includes: a first network model, a second network model, and a fully connected and softmax layer connected to the first network model and the second network model. That is to say, in an embodiment of the present invention, the analysis and processing of customer satisfaction only focuses on the customer's complaint content and ignores the customer's personal information and package type issues, while in an embodiment of the present invention, based on the customer's complaint content, the user information and package information corresponding to the complaint content are integrated, and the user information and package features are effectively extracted from the complaint voice dialogue converted into text, thereby improving the accuracy of predicting customer satisfaction, formulating corresponding customer retention strategies in advance, thereby reducing customer churn and improving service quality.

[0080] It is to be understood that the foregoing general description and the following detailed description are exemplary and explanatory only and are not restrictive of the invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0081] The drawings herein are incorporated into the specification and constitute a part of the specification, showing embodiments consistent with the present invention, and are used together with the specification to explain the principles of the present invention, and do not constitute an improper limitation of the present invention. In order to more clearly illustrate the technical solutions in the embodiments of the present application or the prior art, the drawings required for use in the embodiments or the prior art description will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0082] Figure 1 It is a flowchart of a data processing method based on deep learning provided by an embodiment of the present invention.

[0083] Figure 2 It is a structural diagram of a judgment model provided by an embodiment of the present invention.

[0084] Figure 3 is a schematic diagram of a training process of a judgment model provided by an embodiment of the present invention;

[0085] Figure 4 It is a block diagram of a data processing device based on deep learning provided by an embodiment of the present invention.

[0086] Figure 5 It is a block diagram of a prediction module provided by an embodiment of the present invention.

[0087] Figure 6 It is a block diagram of a normalization processing module provided by an embodiment of the present invention.

[0088] Figure 7 It is another block diagram of a data processing device based on deep learning provided by an embodiment of the present invention.

[0089] Figure 8 It is a block diagram of an electronic device provided by an embodiment of the present invention.

[0090] Fig. 9 It is a block diagram of a device for deep learning-based data processing provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0091] In order to enable ordinary persons in the art to better understand the technical solutions of the present invention, the technical solutions in the embodiments of the present invention will be clearly and completely described below in conjunction with the accompanying drawings.

[0092] It should be noted that the terms "first", "second", etc. in the specification and claims of the present invention and the above-mentioned drawings are used to distinguish similar objects, and are not necessarily used to describe a specific order or sequence. It should be understood that the data used in this way can be interchanged where appropriate, so that the embodiments of the present invention described herein can be implemented in an order other than those illustrated or described herein. The embodiments described in the following exemplary embodiments do not represent all embodiments consistent with the present invention. Instead, they are merely examples of devices and methods consistent with some aspects of the present invention as detailed in the appended claims.

[0093] Technical terms:

[0094] BERT (Bidirectional Encoder Representations from Transformers): is a language representation model, BERT stands for Bidirectional Encoder Representations from Transformer. BERT aims to pre-train deep bidirectional representations by jointly adjusting the left and right contexts in all layers. Therefore, only one additional output layer is needed to fine-tune the pre-trained BERT representation to create state-of-the-art models for a wide range of tasks (such as answering questions and language inference tasks) without making a lot of task-specific modifications to the model structure. BERT is simple in concept, but powerful in experimental results, and is a breakthrough in the field of NLP.

[0095] Natural language processing (NLP) is an important direction in the fields of computer science and artificial intelligence. It studies various theories and methods that can achieve effective communication between people and computers using natural language. Natural language processing is a science that integrates linguistics, computer science, and mathematics. Therefore, research in this field will involve natural language, that is, the language used by people in daily life, so it is closely related to the study of linguistics, but there are important differences. Natural language processing is not a general study of natural language, but the development of computer systems that can effectively realize natural language communication, especially the software systems therein. Therefore, it is a part of computer science. Natural language processing is mainly used in machine translation, public opinion monitoring, automatic summarization, opinion extraction, text classification, question answering, text semantic comparison, speech recognition, Chinese OCR, etc.

[0096] Bidirectional Long Short-Term Memory (BiLSTM): It has been proposed and successfully applied to various tasks. It is based on the recurrent neural network (RNN) in deep learning and is widely used in sequence modeling tasks such as language modeling, machine translation, text classification, etc. Traditional unidirectional RNNs cannot handle context-related problems because they can only consider past information or future information and do not know current information. Bidirectional long short-term memory networks can consider forward and backward information, which can perfectly solve this problem.

[0097] Multi-head attention machanism is a technique used in computer science and machine learning. It originates from the study of human vision and is inspired by the study of human cognitive processes in the fields of psychology and neuroscience. In cognitive science, due to the bottleneck of information processing, humans selectively focus on part of all information while ignoring other visible information. This mechanism is often referred to as an attention mechanism. The main purpose of the attention mechanism is to select specific input categories and focus on a subset of the input, thereby improving the accuracy and efficiency of the model when processing sequence data. In deep learning, especially in the fields of natural language processing, image recognition, and speech recognition, the attention mechanism allows the model to dynamically focus on key parts of the data instead of treating all information equally. It extends the traditional attention mechanism and allows the model to focus on different parts of the input and multiple aspects of information at the same time.

[0098] Normalized exponential function (Softmax) is a generalization of the binary classification function sigmoid in multi-classification. Its purpose is to present the results of multi-classification in the form of probability. It maps the outputs of multiple neurons to the interval (0,1), which can be understood as probability, so as to perform multi-classification.

[0099] On the basis of understanding the above technical terms, please refer to the following embodiments.

[0100] Figure 1 is a flow chart of a data processing method based on deep learning provided by an embodiment of the present invention. Figure 1 As shown, the method comprises the following steps:

[0101] Step 101: Acquire text data to be predicted, and user information and package information corresponding to the text data to be predicted.

[0102] Step 102: Based on the trained judgment model, the text data to be predicted, the user information and the package information are classified and predicted to obtain a classification prediction result, wherein the judgment model includes: a first network model, a second network model, and a fully connected and softmax layer connected to the first network model and the second network model.

[0103] In the embodiment of the present invention, based on the content of customer complaints, the personal information and package type of the complaint content are integrated to solve the accuracy of customer satisfaction prediction so as to formulate corresponding customer retention strategies in advance, thereby reducing customer churn rate and improving service quality.

[0104] The deep learning-based data processing method described in the present invention can be applied to terminals, servers, etc., without limitation herein. The terminal implementation device can be electronic devices such as smart phones, laptops, tablet computers, desktop computers, personal digital assistants (PDAs) and wearable devices. The server can be an independent server or a server cluster, or a server that provides cloud services, cloud databases, cloud computing, cloud functions, cloud storage, network services, cloud communications, intermediate services, domain name services, security services, content distribution networks, or big data and artificial intelligence platforms, etc., without limitation herein.

[0105] Combine the following Figure 1 , the specific implementation steps of a data processing method based on deep learning provided by an embodiment of the present invention are described in detail.

[0106] In step 101, text data to be predicted, and user information and package information corresponding to the text data to be predicted are obtained.

[0107] In this step, there are many ways to obtain the text data to be predicted, which can be obtained locally, remotely, or in real time, and this embodiment does not limit this.

[0108] Among them, this step may include: obtaining a voice call between the complaining user and the customer service; converting the voice call into text; for example, converting into text data, etc. In this embodiment, the converted text data is used as the text data to be tested.

[0109] In step 102, the text data to be predicted, the user information and the package information are classified and predicted based on the trained judgment model to obtain a classification prediction result, wherein the judgment model includes: a first network model, a second network model, and a fully connected and softmax layer connected to the first network model and the second network model.

[0110] In this step, specifically, first, based on the first network model in the judgment model, the text data to be predicted and the user information are vector fused to obtain a first fusion vector, and based on the second network model in the judgment model, the text data to be predicted and the package information are vector fused to obtain a second fusion vector; secondly, the first fusion vector and the second fusion vector are input into the full connection and Softmax layers in the judgment model for vector normalization processing to obtain a classification prediction processing result.

[0111] In an embodiment of the present invention, text data to be predicted, as well as user information and package information corresponding to the text data to be predicted are obtained; the text data to be predicted, the user information and the package information are classified and predicted based on a trained judgment model to obtain a classification prediction result, and the judgment model includes: a first network model, a second network model, and a fully connected and softmax layer connected to the first network model and the second network model. That is to say, in an embodiment of the present invention, the analysis and processing of customer satisfaction only focuses on the customer's complaint content and ignores the customer's personal information and package type issues, while in an embodiment of the present invention, based on the customer's complaint content, the user information and package information corresponding to the complaint content are integrated, and the user information and package features are effectively extracted from the complaint voice dialogue converted into text, thereby improving the accuracy of predicting customer satisfaction, so as to formulate corresponding customer retention strategies in advance, thereby reducing customer churn and improving service quality.

[0112] Optionally, in another embodiment, based on the above embodiment, the first network model includes: a first BERT layer, a first BiLSTM layer and a first multi-head attention layer connected in series; the second network model includes: a second BERT layer, a second BiLSTM layer and a second multi-head attention layer connected in series.

[0113] The method of performing vector fusion on the text data to be predicted and the user information based on the first network model in the judgment model to obtain a first fusion vector includes: inputting the text data to be predicted into the first BERT layer for feature vector learning, and inputting the output feature vector into the first BiLSTM layer for learning iteration, and fusing the obtained first iteration result with the vector of the user information in the first multi-head attention layer to obtain a first fusion vector.

[0114] The method of performing vector fusion on the text data to be predicted and the package information based on the second network model in the judgment model to obtain a second fusion vector includes: inputting the text data to be predicted into the second BERT layer for feature vector learning, and inputting the output feature vector into the second BiLSTM layer for learning iteration, and fusing the obtained second iteration result with the vector of the package information in the second multi-head attention layer to obtain a second fusion vector.

[0115] In an embodiment of the present invention, a feature vector output by a BERT layer in the judgment model is input to a BiLSTM layer connected thereto, and is connected in series with a multi-head additive attention layer adding user information, and a feature vector output by another BERT layer in the judgment model is input to another BiLSTM layer connected thereto, and is connected in series with a multi-head additive attention layer adding package information, and the two series connection results are connected in parallel to the full connection and Softmax layers of the judgment model for vector normalization processing, thereby obtaining a classification prediction processing result. The judgment model provided in an embodiment of the present invention has a richer structure and can be integrated with more parameters to determine the accuracy of the prediction results of the model.

[0116] Optionally, the step of inputting the text data to be predicted into the first BERT layer for feature vector learning, and inputting the output feature vector into the first BiLSTM layer for learning iteration includes: inputting the text data to be predicted into the first BERT layer for vector conversion processing; extracting the word vectors output by the first BERT layer as the first group of word vectors for the text data to be predicted; inputting the first group of word vectors into the first BiLSTM layer for continuous learning iteration to obtain first forward hidden layer features and first reverse hidden layer features; concatenating the first forward hidden layer features and the second reverse hidden layer features to obtain hidden layer features of the first BiLSTM layer;

[0117] The step of inputting the text data to be predicted into the second BERT layer for feature vector learning includes: inputting the text data to be predicted into the second BERT layer for vector conversion processing; extracting the word vectors output by the second BERT layer as the second group of word vectors of the text data to be predicted, and inputting the second group of word vectors into the second BiLSTM layer for continuous learning iteration to obtain second forward hidden layer features and second reverse hidden layer features; and concatenating the second forward hidden layer features and the second reverse hidden layer features to obtain the hidden layer features of the second BiLSTM layer.

[0118] The embodiment of the present invention is based on deep learning technology and has powerful learning and processing capabilities. It can meet the satisfaction and network exit prediction scenarios of various situations. It has strong generalization and adaptability, and supports automatic learning of features from data without manual design. This makes it more effective in processing high-dimensional data and can discover hidden connections in the data.

[0119] Optionally, fusing the obtained first iteration result with the user information in the first multi-head attention layer to obtain a first fusion vector includes: performing vector splitting and mapping of the vector of hidden layer features output by the first BiLSTM layer and the user vector of the user information according to the number of heads, and inputting them into the first multi-head attention layer for fusion; learning different weights in each subspace, and performing vector splicing of the attention fusion results obtained in each subspace to obtain a first fusion vector output by the first multi-head attention layer;

[0120] The second iterative result is fused with the user information in the second multi-head attention layer to obtain a second fusion vector, including: splitting and mapping the vector of the hidden layer features output by the second BiLSTM layer and the package vector of the package information according to the number of heads, and inputting them into the second multi-head attention layer for fusion; learning different weights in each subspace, and vectorizing the attention fusion results obtained in each subspace to obtain the second fusion vector output by the second multi-head attention layer.

[0121] In the embodiment of the present invention, a judgment model with BERT, BiLSTM, a multi-head attention mechanism integrating users, a multi-head attention mechanism integrating package information, and a full connection and Softmax layer is proposed. When using the judgment model for classification and prediction processing, not only the data information of the complaint content itself is considered, but also the user information and package information of the complaint content are integrated, and the advantages of the pre-trained language model and the recurrent neural network are fully utilized. It can effectively extract features from the complaint dialogue converted from speech to text, and improve the accuracy of predicting customer satisfaction, so as to formulate corresponding customer retention strategies in advance, thereby reducing customer churn rate and improving service quality.

[0122] In an embodiment of the present invention, the text data to be predicted, the user information and the package information are input into a trained judgment model for classification prediction processing to obtain a classification prediction processing result of the text data to be predicted, and the judgment model is a network model based on deep learning, including: a first BERT layer, a second BERT layer, a first BiLSTM layer, a second BiLSTM layer, a first multi-head attention layer integrated with the user information, a second multi-head attention layer integrated with the package information, and a fully connected and Softmax layer, wherein the first BERT layer, the first BiLSTM layer and the first multi-head attention layer are connected in series in sequence, the second BERT layer, the second BiLSTM layer and the second multi-head attention layer are connected in series in sequence, and the first multi-head attention layer and the second multi-head attention layer are connected in parallel to the fully connected and Softmax layers.

[0123] Optionally, in another embodiment, based on the above embodiment, the first fusion vector and the second fusion vector are input into the fully connected and Softmax layers for vector normalization processing to obtain a classification prediction processing result, including: inputting the first fusion vector and the second fusion vector into the fully connected and Softmax layers for vector splicing; normalizing the spliced ​​fusion vector to obtain a classification prediction processing result.

[0124] In the embodiment of the present invention, in the process of judging the model parameter training, the word vector can also participate in the training and perform adaptive changes.

[0125] The embodiment of the present invention adopts multi-head additive attention (referred to as multi-head attention) that integrates user information and package information. Multi-head attention can allocate more computing subspaces and can perform parallel calculations. The advantage is that there are more trainable parameters without causing additional time increase.

[0126] See also Figure 2 , is a structural diagram of a judgment model provided by an embodiment of the present invention, the judgment module includes: a first network model, a second network model, and a fully connected and Softmax layer, wherein the fully connected and Softmax layers are connected to the first network model and the second network model respectively.

[0127] like Figure 2As shown, the first network model includes: a first BERT layer, a first BiLSTM layer and a first multi-head attention layer connected in series in sequence; wherein, the first multi-head attention layer can also be called a multi-head additive attention layer integrating user information (referred to as the first multi-head attention layer, the same below); the second network model includes: a second BERT layer, a second BiLSTM layer and a second multi-head attention layer connected in series in sequence; wherein, the second multi-head attention layer can also be called a multi-head attention layer integrating package information (referred to as the second multi-head attention layer, the same below); the first multi-head attention layer and the second multi-head attention layer are connected in parallel to the full connection and Softmax layers.

[0128] First, the text data to be predicted is input into the first BERT layer for feature vector learning, and the output feature vector is input into the first BiLSTM layer for learning iteration, and the first iteration result is fused with the vector of the user information (represented by a in the figure) in the first multi-head attention layer to obtain a first fused vector; and the text data to be predicted is input into the second BERT layer for feature vector learning, and the output feature vector is input into the second BiLSTM layer for learning iteration, and the second iteration result is fused with the vector of the package information (represented by u in the figure) in the second multi-head attention layer to obtain a second fused vector.

[0129] Secondly, the first fusion vector and the second fusion vector are input into the full connection and Softmax layer for vector splicing; the spliced ​​fusion vector is normalized to obtain a classification prediction processing result.

[0130] The present invention uses multi-head additive attention that fuses user information and multi-head additive attention that fuses package information. The multi-head additive attention is composed of multiple groups of attention combinations and can learn more contextual situations, thereby increasing the expressive power of the judgment model and allocating more computing subspaces. It can not only perform parallel calculations and train more parameters, but also does not cause additional time increase. The use of multi-head additive attention can focus on key information related to complaining users and packages, thereby improving the accuracy of the model in predicting future user behaviors. At the same time, the fusion information calculation method can share parameters for parallel calculation without taking up additional processing time.

[0131] Optionally, in another embodiment, based on the above implementation, the method may also include: pre-training the judgment model in the following manner: obtaining a data set, the data set including: a training set, and a validation set of the training set; preprocessing the training set; obtaining a user dictionary and a package dictionary of the complaining user; converting the user dictionary into a user vector with a length of BiLSTM dimension and a width of the number of users, and converting the package dictionary into a package vector with a length of BiLSTM dimension and a width of the number of packages; inputting the preprocessed training set, the user vector and the package vector into the judgment model for training, and when training the judgment model, randomly initializing the user vector, and adding the user corresponding to the complaint data to the row where the user vector is located for training; and randomly initializing the package vector, and adding the package corresponding to the complaint data to the row where the package vector is located for training, to obtain a classification prediction training result; comparing the classification prediction training result with the validation set, determining a loss value based on the comparison result, and updating the parameters of the judgment model based on the loss value through a back propagation mechanism, and after multiple iterations, until the judgment model converges, a trained judgment model is obtained.

[0132] Please also refer to Figure 2 and Figure 3 ,in, Figure 3 A flowchart of a training process of a judgment model provided by an embodiment of the present invention specifically includes:

[0133] Step 300: Start.

[0134] Step 301: Create a data set and divide the data set into a training set, a validation set, and a test set.

[0135] Among them, the data category label can take whether to log off the network after 1 month or user satisfaction as an example.

[0136] Step 302: Data preprocessing.

[0137] In this step, the data set is preprocessed, including removing outliers, missing values, and duplicate values ​​in the data;

[0138] It should be noted that in this step, the data set may be preprocessed before division, or the training set, validation set and test set may be preprocessed after division, etc., and this embodiment does not impose any limitation thereto.

[0139] Step 303: Create a complaint user dictionary and a package dictionary.

[0140] Among them, in this step, the user dictionary field includes the user code and user name, and the package dictionary includes the package code and package name. The method of using the user dictionary in the model is to first convert the user dictionary into a user vector with a length of bilstm dimension and a width of the number of users. When training the judgment model, the user vector is randomly initialized, and the user corresponding to the complaint data is added to the training in the row where the user vector is located; the method of using the package dictionary in the model is to first convert the package dictionary into a package vector with a length of bilstm dimension and a width of the number of packages. When training the judgment model, the package vector is randomly initialized, and the package corresponding to the complaint data is added to the training in the row where the package vector is located.

[0141] Step 304: Define the evaluation indicators, activation functions, loss functions, and regularization processing required for model training.

[0142] In this step, the model evaluation indicators are accuracy, recall and F1 score; accuracy refers to the proportion of the number of samples correctly predicted by the model to the total number of samples, and its calculation formula is as follows

[0143]

[0144] Among them, Accuracy——represents the accuracy of the model, that is, the proportion of correctly predicted samples to the total samples;

[0145] TP——indicates the number of samples that the model predicts to be positive samples and are actually positive samples;

[0146] TN——indicates the number of samples that the model predicts to be negative samples and are actually negative samples;

[0147] FP — indicates the number of samples that the model predicts to be positive samples but are actually negative samples;

[0148] FN——Indicates the number of samples that the model predicts to be negative samples but are actually positive samples.

[0149] Recall refers to the ratio of the number of samples correctly predicted by the model as positive examples to the total number of actual positive examples. The calculation formula is:

[0150]

[0151] In the following formula, the F1 score is the harmonic mean of the precision and recall, which combines the performance indicators of the precision and recall. The calculation formula is:

[0152]

[0153]

[0154] Among them, precision represents the proportion of samples predicted by the model as positive examples and actually positive examples.

[0155] This method determines that the model activation function is the Adam algorithm. In the Adam algorithm, each parameter has an adaptive learning rate, which calculates the average of the first-order moment estimate of the gradient and the exponential moving average of the second-order moment estimate, and uses them as the denominator of the learning rate. Therefore, as an optimizer, it can adaptively adjust the learning rate without manually adjusting the size of the learning rate. It can usually adapt to different tasks and data sets, and the effect is relatively good. At the same time, the Adam algorithm can also reduce the direction of the gradient update, making the training process smoother and preventing violent excessive oscillations during the parameter update process.

[0156] This method determines that the model loss function is the cross entropy function. The cross entropy can be used to measure the difference between the model's predicted results and the actual results. The smaller the difference, the better the model effect. Each time training on the training set, all parameter values ​​are learned and updated iteratively, thereby reducing the cross entropy loss.

[0157] Step 305: batch the data. The batch data size can be 512 or 1024, etc., which is not limited in this embodiment.

[0158] For example, the data is divided into batches of data, each batch of data is called batch data, and the batch data size is 512 or 1024.

[0159] Step 306: Determine whether the batch data is empty, if yes, execute step 309, if no, execute step 307.

[0160] Step 307: training parameters, where the parameters may include complaint content feature vectors, user feature vectors, package content vectors, etc.

[0161] In this step, training parameters means adjusting model parameters and iteratively training the model until the model converges.

[0162] Step 308: Save the best judgment model.

[0163] Step 309: End.

[0164] Optionally, in this embodiment, the design of the BERT layer (i.e., the first BERT layer or the BERT layer, the same below) is specifically as follows:

[0165] Text the call c Pass it into the BERT model and take the vector output by the last layer of BERT to convert word x l The calculation process of the transformed vector w is as follows:

[0166] w=W c x l

[0167] Where: x i represents the i-th word in the text, W e Represents weight.

[0168] Preferably, the design of the BiLSTM layer (i.e., the first BiLSTM layer or the second BiLSTM layer, the same below) is specifically as follows:

[0169] The output layer of the recurrent neural network is calculated as follows:

[0170] o i =g(Vs i )

[0171] Among them, t represents the output of the recurrent neural network at time t; g represents the activation function; V represents the weight matrix of the output layer;

[0172] The hidden layer calculation method of the recurrent neural network is as follows:

[0173] s t =f(Ux t +Ws t-1 )

[0174] Among them, s i represents the output of the hidden layer of the recurrent neural network at time t; f represents the activation function; x represents the input of the recurrent neural network; U represents the weight matrix of the input layer; W represents the weight matrix of the last hidden layer as input.

[0175] The output is obtained by iterating through the weight cycles over time and tracing back to the output of each previous hidden layer.

[0176] Optionally, in an embodiment of the present invention, the LSTM adds a gating unit based on the RNN, and the gating unit controls which information is forgotten and which information is remembered through the cell state of the previous step.

[0177] Optional: In the embodiment of the present invention, the BiLSTM is to connect two LSTMs in reverse order, so that the weight parameters in all neurons in the forward calculation are consistent, and the weight parameters in all neurons in the reverse calculation are also consistent, and weight sharing can be performed;

[0178] The word vectors [w1, w2, …, w n] is used as the input of BiLSTM. Through the continuous learning iteration of BiLSTM, the forward hidden layer and the reverse hidden layer can be obtained. They are concatenated together to obtain the hidden layer state of BiLSTM, which is expressed as [h1, h2, …, h n ]; The calculation formula of the hidden layer of BiLSTM is as follows:

[0179]

[0180] in, Represents the hidden layer state of the forward BiLSTM; Represents the hidden layer state of the reverse BiLSTM.

[0181] Optionally, in this embodiment, the multi-head attention that fuses user information is to use different weight parameters to calculate the fused user information attention in multiple subspaces respectively, and then splice the subspace attention, so as to obtain text features with richer information. The vector output by the BiLSTM hidden layer and the user vector are vector-split and mapped according to the number of heads as the input of the multi-head attention, and then different weights are learned and trained in each subspace. Finally, the attention obtained in each subspace is vector-spliced ​​to obtain the final multi-head attention. The calculation formula for multi-head attention is as follows:

[0182] MAtt=Concat(head1,...,head n )=MultiHeadAtt(head i )

[0183] Among them, head1,...,head n represents the sub-attention after each subspace is calculated; MAtt represents multi-head attention; head i Represents the low-dimensional vector of the BiLSTM hidden layer output layer after split mapping.

[0184] And the dimensions after splitting and mapping and the original dimensions must comply with the principles shown in the following formula:

[0185] d head =d LSTM / num h

[0186] Among them, d head Represents the vector dimension of each head; d LSTM Represents the vector dimension of the BiLSTM hidden layer; num h Indicates the number of heads for multi-head attention.

[0187] In each computational subspace, based on extracting the style features of the complaining user and combining them with the attention mechanism, the weight calculation process for the entire conversation text represented as the BiLSTM hidden layer state combined with the attention mechanism is as follows:

[0188]

[0189] Among them, d a Indicates the text representation of the entire dialogue; Represents the hidden layer state of the i-th word in the dialogue text; Represents the hidden layer state The attention weight represents the importance of the i-th word in the conversation text containing the features of the complaining user;

[0190] Map the complaint user vector a to a continuous vector d a represents the dimension of the complaint user vector, then the attention weight The definition is as follows:

[0191]

[0192]

[0193] in, express The attention weight of represents the weight vector, is its transpose; Represents the hidden layer The weight matrix of Represents the weight matrix of the complaint user vector a; Represents the bias term of the function;

[0194] in, a and As parameters of the training model, they are assigned random initialization variables, functions The role of is to calculate the importance of all words in the complaint text.

[0195] The multi-head attention that integrates package information is to use different weight parameters to calculate the attention of the integrated package information in multiple subspaces, and then splice the subspace attention, so as to obtain text features with richer information. Since each subspace is calculated in parallel, there is no additional time loss. The vector output by the BiLSTM hidden layer and the package vector are split and mapped according to the number of heads as the input of the multi-head attention, and then different weights are learned and trained in each subspace. Finally, the attention obtained in each subspace is vector spliced ​​to obtain the final multi-head attention. The calculation formula of multi-head attention is as follows:

[0196] MAtt=Concat(head1,...,head n )=MultiHeadAtt(head i )

[0197] Among them, head1,...,head n represents the sub-attention after each subspace is calculated; MAtt represents multi-head attention; head i Represents the low-dimensional vector of the BiLSTM hidden layer output layer after split mapping.

[0198] And the dimensions after splitting and mapping and the original dimensions must comply with the principles shown in the following formula:

[0199] d head =d LSTM / num h

[0200] Among them, d head Represents the vector dimension of each head; d LSTM Represents the vector dimension of the BiLSTM hidden layer; num h Indicates the number of heads for multi-head attention.

[0201] In each computational subspace, for the representation d u The weights of the attention mechanism for the BiLSTM hidden layer states are calculated as follows:

[0202]

[0203] Among them, d u The text representation of the package type features; Represents the hidden layer state of the i-th word in the call text; Represents the hidden layer state The attention weight represents the importance of the package feature contained in the i-th word in the text;

[0204] Map the package type (or package vector) u to a continuous vector d u represents the dimension of the package type vector, then the attention weight The definition is as follows:

[0205]

[0206]

[0207] Where: express The attention weight of represents the weight vector, is its transpose; Represents the hidden layer The weight matrix of Represents the weight matrix of package type u; Represents the bias term of the function;

[0208] in, u and As parameters of the training model, they are assigned random initialization variables, functions The function of is to calculate the importance of all words in the call text, and the call text vectors on both sides of the model share weights; in the multi-head attention layer, the hidden layer state of BiLSTM is combined with the multi-head attention mechanism containing user features and package features, and a weight is assigned to each hidden layer state. Finally, the call text dc containing user features (i.e., the first fusion vector) and the call text du containing package features (i.e., the second fusion vector) are obtained.

[0209] Optional: During the training process, the input of the Softmax layer is usually an unnormalized vector representing the score or probability of each category; then, the Softmax layer normalizes this vector so that each element falls within the interval [0, 1] and the sum of all elements is 1; Softmax is defined as follows:

[0210] Among them, z i represents the output value of the i-th node; C represents the number of output nodes, that is, the number of classification categories;

[0211] The call text containing user features d will be obtained a and call texts containing package features u Combine them for model prediction and training, concatenate the two vectors as the final classification document representation d, and then send d to the fully connected layer and softmax layer to map d into the final classification p. The process is as follows:

[0212]

[0213] p=softmax(W d +b)

[0214] Among them, p represents the final classification, that is, the probability value of each category; W represents the weight matrix of softmax; b represents the bias term of softmax.

[0215] Of course, in this step, the call text containing the user characteristics can also be first a (i.e. the first fusion vector) and the call text containing the package features d uThe second fused vector is concatenated, and the concatenated vector is input into the full connection and Softmax layer for processing, and the processed vector is mapped into the final classification.

[0216] Optional: In this embodiment, when performing deep learning training, some configurations other than model construction and model evaluation indicators can also be reasonably selected, such as using the following methods to solve the overfitting problem:

[0217] 1. Use Dropout to randomly discard neurons, thereby reducing the dependency between neurons to alleviate overfitting. The specific operation method is as follows:

[0218] (1) A Dropout layer is added between the fully connected layer and the softmax layer. During the training process, the outputs of some neurons are randomly set to 0, so that these neurons do not participate in back propagation and parameter updating.

[0219] (2) In the Dropout layer, a Dropout probability p is set, which indicates the probability that each neuron is set to 0.5 during training.

[0220] (3) During the forward propagation process, when the input data passes through the neural network, the Dropout layer randomly sets the output of some neurons to 0 according to the set Dropout probability p. Then, these neurons set to 0 do not participate in subsequent calculations until the next forward propagation.

[0221] (4) During the back-propagation process, the Dropout layer adjusts the gradient of the neuron according to the set Dropout probability p. For neurons that are set to 0, their gradients are also set to 0, so they do not participate in parameter updates. For neurons that are not set to 0, their gradients are multiplied by a factor of 1-p to compensate for the reduction in neuron output caused by Dropout.

[0222] (5) During the parameter update process, only the parameters of neurons that are not set to 0 will be updated. Specifically, based on the gradient calculated by back-propagation, an optimization algorithm (such as stochastic gradient descent) is used to update the parameters of the neural network.

[0223] 2. Use early stopping techniques to avoid overfitting by monitoring the performance of the validation set during training and stopping training when the performance no longer improves. The specific operation method is as follows:

[0224] (1) Monitor the performance of the validation set during training: After each iteration, calculate performance indicators such as accuracy and loss on the validation set.

[0225] (2) Determine the conditions for early stopping and stop training if the performance of the validation set does not improve after three consecutive iterations.

[0226] (3) Save the optimal model. During the training process, save the model parameters with the best performance on the validation set.

[0227] 3. Use L2 regularization technology to add penalty terms to limit the value range of model parameters to reduce the complexity of the model. The loss function calculation formula is as follows:

[0228]

[0229] in, Indicates the category of the i-th call text; Represents the model's predicted category for the i-th call text; k represents the number of categories; λ represents the coefficient size of the L2 regularization penalty term; 0 represents all trainable parameters of the model.

[0230] It should be noted that, for the method embodiments, for the sake of simplicity, they are all described as a series of action combinations, but those skilled in the art should know that this implementation disclosure is not limited by the order of the actions described, because according to the present invention, certain steps can be performed in other orders or simultaneously. Secondly, those skilled in the art should also know that the embodiments described in the specification are all preferred embodiments, and the actions involved are not necessarily required by the present invention.

[0231] Figure 4 4 is a block diagram of a data processing device based on deep learning provided by an embodiment of the present invention. The device includes: an acquisition module 401 and a prediction module 402, wherein:

[0232] The acquisition module 401 is used to acquire text data to be predicted, and user information and package information corresponding to the text data to be predicted.

[0233] The prediction module 402 is used to perform classification prediction processing on the text data to be predicted, the user information and the package information based on the trained judgment model to obtain the classification prediction processing result, wherein the judgment model includes: a first network model, a second network model, and a fully connected and Softmax layer connected to the first network model and the second network model.

[0234] Optionally, in another embodiment, based on the above embodiment, the prediction module 402 includes: a first fusion module 501, a second fusion module 502 and a normalization processing module 503, and its structural block diagram is as follows: Figure 5 As shown,

[0235] A first fusion module 501 is used to perform vector fusion on the text data to be predicted and the user information based on the first network model in the judgment model to obtain a first fusion vector; and

[0236] A second fusion module 502 is used to perform vector fusion on the text data to be predicted and the package information based on the second network model in the judgment model to obtain a second fusion vector;

[0237] The normalization processing module 503 is used to input the first fusion vector and the second fusion vector into the full connection and Softmax layers in the judgment model to perform vector normalization processing to obtain a classification prediction processing result.

[0238] Optionally, in another embodiment, based on the above embodiment, the first network model includes: a first BERT layer, a first BiLSTM layer and a first multi-head attention layer connected in series; the second network model includes: a second BERT layer, a second BiLSTM layer and a second multi-head attention layer connected in series.

[0239] In another embodiment, based on the above embodiment, the first fusion module includes: a first iteration module and a first vector fusion module, wherein:

[0240] A first iteration module, used for inputting the text data to be predicted into the first BERT layer for feature vector learning, and inputting the output feature vector into the first BiLSTM layer for learning iteration;

[0241] The first vector fusion module is used to fuse the first iteration result obtained by the first iterative learning module with the vector of the user information in the first multi-head attention layer to obtain a first fused vector.

[0242] Optionally, in another embodiment, based on the above embodiment, the second fusion module includes: a second iteration module and a second vector fusion module, wherein:

[0243] A second iteration module, used for inputting the text data to be predicted into the second BERT layer for feature vector learning, and inputting the output feature vector into the second BiLSTM layer for learning iteration;

[0244] The second vector fusion module is used to fuse the second iteration result obtained by the second iterative learning module with the vector of the package information in the second multi-head attention layer to obtain a second fused vector.

[0245] Optionally, in another embodiment, based on the above embodiment, the first iteration module includes:

[0246] A first conversion module, used for inputting the text data to be predicted into the first BERT layer for vector conversion processing;

[0247] A first extraction module, used to extract the word vectors output by the first BERT layer as the first group of word vectors of the text data to be predicted;

[0248] A first iterative learning module, used for inputting the first group of word vectors into the first BiLSTM layer for continuous learning iteration to obtain a first forward hidden layer feature and a first reverse hidden layer feature;

[0249] The first concatenation module is used to concatenate the first forward hidden layer features and the second reverse hidden layer features to obtain the hidden layer features of the first BiLSTM layer.

[0250] In another embodiment, based on the above embodiment, the second iteration module includes:

[0251] A second conversion module, used for inputting the text data to be predicted into the second BERT layer for vector conversion processing;

[0252] A second extraction module, used to extract the word vectors output by the second BERT layer as a second set of word vectors for the text data to be predicted;

[0253] A second iterative learning module, used for inputting the second group of word vectors into the second BiLSTM layer for continuous learning iteration to obtain a second forward hidden layer feature and a second reverse hidden layer feature;

[0254] The second concatenation module is used to concatenate the second forward hidden layer features and the second reverse hidden layer features to obtain the hidden layer features of the second BiLSTM layer.

[0255] Optionally, in another embodiment, based on the above embodiment, the first vector fusion module includes:

[0256] A first splitting and mapping module, used for performing vector splitting and mapping of the vector of the hidden layer features output by the first BiLSTM layer and the user vector of the user information according to the number of heads;

[0257] A first multi-attention fusion module, used for inputting the mapping result of the first split mapping module into the first multi-head attention layer for fusion;

[0258] A first multi-attention concatenation module is used to learn different weights in each subspace and concatenate the attention fusion results obtained in each subspace to obtain a first fusion vector output by the first multi-head attention layer;

[0259] The first vector fusion module includes:

[0260] A first splitting and mapping module is used to perform vector splitting and mapping on the vector of the hidden layer features output by the second BiLSTM layer and the package vector of the package information according to the number of heads;

[0261] A second multi-attention fusion module, used to input the mapping result of the second split mapping module into the second multi-head attention layer for fusion;

[0262] The second multi-attention splicing module is used to learn different weights in each subspace, and vector-spin the attention fusion structures obtained from each subspace to obtain a second fusion vector output by the second multi-head attention layer.

[0263] Optionally, in another embodiment, based on the above embodiment, the normalization processing module 503 includes: a fusion vector splicing module 601 and a normalization processing submodule 602, and its structural block diagram is as follows: Figure 6 As shown,

[0264] A fusion vector concatenation module 601 is used to input the first fusion vector and the second fusion vector into the full connection and Softmax layer for vector concatenation;

[0265] The normalization processing submodule 602 is used to perform normalization processing on the concatenated fusion vector to obtain a classification prediction processing result.

[0266] Optionally, in another embodiment, based on the above embodiment, the device further includes: a training module 701, whose structural block diagram is as follows: Figure 7 shown.

[0267] The training modules include:

[0268] A data set acquisition module, used to acquire a data set, the data set comprising: a training set, and a validation set of the training set;

[0269] A preprocessing module, used for preprocessing the training set;

[0270] Dictionary acquisition module, used to obtain the user dictionary and package dictionary of the complaining user;

[0271] A conversion module, used to convert the user dictionary into a user vector with a length of BiLSTM dimension and a width of the number of users, and to convert the package dictionary into a package vector with a length of BiLSTM dimension and a width of the number of packages;

[0272] A training module is used to input the preprocessed training set, the user vector and the package vector into the judgment model for training. When training the judgment model, the user vector is randomly initialized, and the user corresponding to the complaint data is added to the training in the row where the user vector is located; and the package vector is randomly initialized, and the package corresponding to the complaint data is added to the training in the row where the package vector is located to obtain a classification prediction training result; based on the classification prediction training result, the training result is compared with the validation set, and a loss value is determined according to the comparison result. Based on the loss value, the parameters of the judgment model are updated through a back propagation mechanism, and after multiple iterations, the judgment model converges to obtain a trained judgment model.

[0273] Optionally, an embodiment of the present invention further provides an electronic device, including:

[0274] processor;

[0275] a memory for storing instructions executable by the processor;

[0276] Wherein, the processor is configured to execute the instructions to implement the data processing method based on deep learning as described above.

[0277] Optionally, an embodiment of the present invention further provides a computer-readable storage medium, which, when instructions in the computer-readable storage medium are executed by a processor of an electronic device, enables the electronic device to execute the deep learning-based data processing method as described above.

[0278] Optionally, an embodiment of the present invention further provides a computer program product, including a computer program or instructions, which, when executed by a processor of an electronic device, implements the deep learning-based data processing method as described above.

[0279] Regarding the device in the above embodiment, the specific manner in which each module performs operations has been described in detail in the embodiment of the method, and will not be elaborated here.

[0280] The device embodiments described above are merely illustrative, wherein the units described as separate components may or may not be physically separated, and the components displayed as units may or may not be physical units, that is, they may be located in one place, or they may be distributed on multiple network units. Some or all of the modules may be selected according to actual needs to achieve the purpose of the scheme of this embodiment. Ordinary technicians in this field can understand and implement it without paying creative labor.

[0281] Figure 8 800 is a block diagram of an electronic device 800 provided in an embodiment of the present invention. For example, the electronic device 800 may be a mobile terminal or a server, and the electronic device is a mobile terminal as an example for explanation in the embodiment of the present invention. For example, the electronic device 800 may be a mobile phone, a computer, a digital broadcast terminal, a message transceiver device, a game console, a tablet device, a medical device, a fitness device, a personal digital assistant, etc.

[0282] Reference Figure 8 , the electronic device 800 may include one or more of the following components: a processing component 802 , a memory 804 , a power component 806 , a multimedia component 808 , an audio component 810 , an input / output (I / O) interface 812 , a sensor component 814 , and a communication component 816 .

[0283] The processing component 802 generally controls the overall operation of the electronic device 800, such as operations associated with display, phone calls, data communications, camera operations, and recording operations. The processing component 802 may include one or more processors 820 to execute instructions to complete all or part of the steps of the above-mentioned method. In addition, the processing component 802 may include one or more modules to facilitate the interaction between the processing component 802 and other components. For example, the processing component 802 may include a multimedia module to facilitate the interaction between the multimedia component 808 and the processing component 802.

[0284] The memory 804 is configured to store various types of data to support operations on the device 800. Examples of such data include instructions for any application or method operating on the electronic device 800, contact data, phone book data, messages, pictures, videos, etc. The memory 804 can be implemented by any type of volatile or non-volatile storage device or a combination thereof, such as static random access memory (SRAM), electrically erasable programmable read-only memory (EEPROM), erasable programmable read-only memory (EPROM), programmable read-only memory (PROM), read-only memory (ROM), magnetic memory, flash memory, magnetic disk or optical disk.

[0285] The power supply component 806 provides power to the various components of the electronic device 800. The power supply component 806 may include a power management system, one or more power supplies, and other components associated with generating, managing, and distributing power to the electronic device 800.

[0286] The multimedia component 808 includes a screen that provides an output interface between the electronic device 800 and the user. In some embodiments, the screen may include a liquid crystal display (LCD) and a touch panel (TP). If the screen includes a touch panel, the screen may be implemented as a touch screen to receive input signals from the user. The touch panel includes one or more touch sensors to sense touch, slide, and gestures on the touch panel. The touch sensor may not only sense the boundaries of the touch or slide action, but also detect the duration and pressure associated with the touch or slide operation. In some embodiments, the multimedia component 808 includes a front camera and / or a rear camera. When the device 800 is in an operating mode, such as a shooting mode or a video mode, the front camera and / or the rear camera may receive external multimedia data. Each front camera and the rear camera may be a fixed optical lens system or have a focal length and optical zoom capability.

[0287] The audio component 810 is configured to output and / or input audio signals. For example, the audio component 810 includes a microphone (MIC), and when the electronic device 800 is in an operating mode, such as a call mode, a recording mode, and a voice recognition mode, the microphone is configured to receive an external audio signal. The received audio signal can be further stored in the memory 804 or sent via the communication component 816. In some embodiments, the audio component 810 also includes a speaker for outputting audio signals.

[0288] I / O interface 812 provides an interface between processing component 802 and peripheral interface modules, such as keyboards, click wheels, buttons, etc. These buttons may include but are not limited to: a home button, a volume button, a start button, and a lock button.

[0289] The sensor assembly 814 includes one or more sensors for providing various aspects of status assessment for the electronic device 800. For example, the sensor assembly 814 can detect the open / closed state of the device 800, the relative positioning of components, such as the display and keypad of the electronic device 800, and the sensor assembly 814 can also detect the position change of the electronic device 800 or a component of the electronic device 800, the presence or absence of contact between the user and the electronic device 800, the orientation or acceleration / deceleration of the electronic device 800, and the temperature change of the electronic device 800. The sensor assembly 814 may include a proximity sensor configured to detect the presence of nearby objects without any physical contact. The sensor assembly 814 may also include an optical sensor, such as a CMOS or CCD image sensor, for use in imaging applications. In some embodiments, the sensor assembly 814 may also include an accelerometer, a gyroscope sensor, a magnetic sensor, a pressure sensor, or a temperature sensor.

[0290] The communication component 816 is configured to facilitate wired or wireless communication between the electronic device 800 and other devices. The electronic device 800 can access a wireless network based on a communication standard, such as WiFi, a carrier network (such as 2G, 3G, 4G or 5G), or a combination thereof. In an exemplary embodiment, the communication component 816 receives a broadcast signal or broadcast-related information from an external broadcast management system via a broadcast channel. In an exemplary embodiment, the communication component 816 also includes a near field communication (NFC) module to facilitate short-range communication. For example, the NFC module can be implemented based on radio frequency identification (RFID) technology, infrared data association (IrDA) technology, ultra-wideband (UWB) technology, Bluetooth (BT) technology and other technologies.

[0291] In an embodiment, the electronic device 800 may be implemented by one or more application-specific integrated circuits (ASICs), digital signal processors (DSPs), digital signal processing devices (DSPDs), programmable logic devices (PLDs), field programmable gate arrays (FPGAs), controllers, microcontrollers, microprocessors, or other electronic components to execute the deep learning-based data processing method shown above.

[0292] In an embodiment, a computer-readable storage medium is also provided, and when the instructions in the computer-readable storage medium are executed by the processor of the electronic device, the electronic device 800 can perform the data processing method based on deep learning shown above. For example, the computer-readable storage medium can be a ROM, a random access memory (RAM), a CD-ROM, a magnetic tape, a floppy disk, an optical data storage device, etc.

[0293] In an embodiment, a computer program product is also provided, including a computer program or instructions. When the computer program or instructions are executed by the processor 820 of the electronic device 800, the electronic device 800 executes the above-mentioned deep learning-based data processing method.

[0294] Fig. 9 is a block diagram of a device 900 for deep learning-based data processing provided by an embodiment of the present invention. For example, the device 900 may be provided as a server. Fig. 9 , the apparatus 900 includes a processing component 922, which further includes one or more processors, and a memory resource represented by a memory 932 for storing instructions, such as an application, that can be executed by the processing component 922. The application stored in the memory 932 may include one or more modules, each corresponding to a set of instructions. In addition, the processing component 922 is configured to execute the instructions to perform the above method.

[0295] The device 900 may also include a power supply component 926 configured to perform power management of the device 900, a wired or wireless network interface 950 configured to connect the device 900 to a network, and an input / output (I / O) interface 958. The device 900 may operate based on an operating system stored in the memory 932, such as Windows Server™, Mac OS X™, Unix™, Linux™, FreeBSD™, or the like.

[0296] Those skilled in the art will readily appreciate other embodiments of the present invention after considering the specification and practicing the invention disclosed herein. This application is intended to cover any variations, uses or adaptations of the present invention that follow the general principles of the present invention and include common knowledge or customary techniques in the art that are not disclosed by the present invention. The specification and examples are to be considered exemplary only, and the true scope and spirit of the present invention are indicated by the following claims.

[0297] It should be understood that the present invention is not limited to the exact construction that has been described above and shown in the drawings and that various modifications and changes may be made without departing from the scope thereof. The scope of the present invention is limited only by the appended claims.

Claims

1. A data processing method based on deep learning, characterized in that: include: Acquire text data to be predicted, and user information and package information corresponding to the text data to be predicted; Based on the trained judgment model, the text data to be predicted, the user information and the package information are classified and predicted to obtain a classification prediction result, wherein the judgment model includes: a first network model, a second network model, and a fully connected and softmax layer connected to the first network model and the second network model.

2. The data processing method based on deep learning according to claim 1, characterized in that: The method of performing classification prediction processing on the text data to be predicted, the user information and the package information based on the trained judgment model to obtain the classification prediction processing result includes: Based on the first network model in the judgment model, the text data to be predicted and the user information are vector-fused to obtain a first fused vector; and based on the second network model in the judgment model, the text data to be predicted and the package information are vector-fused to obtain a second fused vector; The first fusion vector and the second fusion vector are input into the full connection and Softmax layers in the judgment model for vector normalization processing to obtain a classification prediction processing result.

3. The data processing method based on deep learning according to claim 2, characterized in that: The first network model includes: a first BERT layer, a first BiLSTM layer, and a first multi-head attention layer connected in series; the second network model includes: a second BERT layer, a second BiLSTM layer, and a second multi-head attention layer connected in series; The step of performing vector fusion on the text data to be predicted and the user information based on the first network model in the judgment model to obtain a first fusion vector includes: Input the text data to be predicted into the first BERT layer for feature vector learning, and input the output feature vector into the first BiLSTM layer for learning iteration, and fuse the obtained first iteration result with the vector of the user information in the first multi-head attention layer to obtain a first fused vector; The step of performing vector fusion on the text data to be predicted and the package information based on the second network model in the judgment model to obtain a second fusion vector includes: The text data to be predicted is input into the second BERT layer for feature vector learning, and the output feature vector is input into the second BiLSTM layer for learning iteration. The second iteration result obtained is fused with the vector of the package information in the second multi-head attention layer to obtain a second fused vector.

4. The data processing method based on deep learning according to claim 3 is characterized in that: The step of inputting the text data to be predicted into the first BERT layer for feature vector learning, and inputting the output feature vector into the first BiLSTM layer for learning iteration includes: Input the text data to be predicted into the first BERT layer for vector conversion processing; extract the word vectors output by the first BERT layer as the first group of word vectors for the text data to be predicted; input the first group of word vectors into the first BiLSTM layer for continuous learning iteration to obtain first forward hidden layer features and first reverse hidden layer features; concatenate the first forward hidden layer features and the second reverse hidden layer features to obtain the hidden layer features of the first BiLSTM layer; The step of inputting the text data to be predicted into the second BERT layer for feature vector learning, and inputting the output feature vector into the second BiLSTM layer for learning iteration includes: The text data to be predicted is input into the second BERT layer for vector conversion processing; the word vectors output by the second BERT layer are extracted as the second group of word vectors for the text data to be predicted, and the second group of word vectors are input into the second BiLSTM layer for continuous learning iteration to obtain the second forward hidden layer features and the second reverse hidden layer features; the second forward hidden layer features and the second reverse hidden layer features are concatenated to obtain the hidden layer features of the second BiLSTM layer.

5. The data processing method based on deep learning according to claim 4, characterized in that: The first iteration result is fused with the user information in the first multi-head attention layer to obtain the first fusion vector, including: performing vector splitting and mapping of the vector of the hidden layer features output by the first BiLSTM layer and the user vector of the user information according to the number of heads, and inputting them into the first multi-head attention layer for fusion; learning different weights in each subspace, and vector splicing the attention fusion results obtained in each subspace to obtain the first fusion vector output by the first multi-head attention layer; The second iterative result is fused with the user information in the second multi-head attention layer to obtain a second fusion vector, including: splitting and mapping the vector of the hidden layer features output by the second BiLSTM layer and the package vector of the package information according to the number of heads, and inputting them into the second multi-head attention layer for fusion; learning different weights in each subspace, and vectorizing the attention fusion results obtained in each subspace to obtain the second fusion vector output by the second multi-head attention layer.

6. The data processing method based on deep learning according to any one of claims 1 to 5, characterized in that: The first fused vector and the second fused vector are input into the full connection and Softmax layer for vector normalization processing to obtain a classification prediction processing result, including: Inputting the first fused vector and the second fused vector into the fully connected and Softmax layers for vector concatenation; The concatenated fusion vector is normalized to obtain the classification prediction result.

7. The data processing method based on deep learning according to any one of claims 1 to 5, characterized in that: The method further comprises: pre-training the judgment model in the following manner: Acquire a data set, the data set comprising: a training set and a validation set of the training set; Preprocessing the training set; Get the user dictionary and package dictionary of the complaining user; Convert the user dictionary into a user vector with a length of BiLSTM dimension and a width of the number of users, and convert the package dictionary into a package vector with a length of BiLSTM dimension and a width of the number of packages; The preprocessed training set, the user vector and the package vector are input into the judgment model for training. When the judgment model is trained, the user vector is randomly initialized, and the user corresponding to the complaint data is added to the training in the row where the user vector is located; and the package vector is randomly initialized, and the package corresponding to the complaint data is added to the training in the row where the package vector is located, so as to obtain the classification prediction training result; based on the classification prediction training result, the verification set is compared, and the loss value is determined according to the comparison result. Based on the loss value, the parameters of the judgment model are updated through the back propagation mechanism, and after multiple iterations, the judgment model converges to obtain a trained judgment model.

8. A data processing device based on deep learning, characterized in that: include: An acquisition module, used to acquire text data to be predicted, and user information and package information corresponding to the text data to be predicted; The prediction module is used to perform classification prediction processing on the text data to be predicted, the user information and the package information based on the trained judgment model to obtain the classification prediction processing result, wherein the judgment model includes: a first network model, a second network model, and a fully connected and Softmax layer connected to the first network model and the second network model.

9. An electronic device, characterized in that: include: processor; a memory for storing instructions executable by the processor; The processor is configured to execute the instructions to implement the deep learning-based data processing method as described in any one of claims 1 to 7.

10. A computer-readable storage medium, characterized in that: When the instructions in the computer-readable storage medium are executed by a processor of an electronic device, the electronic device is enabled to execute the deep learning-based data processing method as described in any one of claims 1 to 7.

Images