Text classification method and device, computer device and storage medium

Abstract

The present disclosure relates to the technical field of computers, and discloses a text classification method and device, computer equipment and a storage medium. The method comprises: obtaining a target text; using a target text classification model to obtain a classification result of the target text according to the target text, wherein the target text classification model comprises a word embedding model, a target convolutional neural network, a set of hidden Markov models and a decision tree model.

Description

Technical Field

[0001] This disclosure relates to the field of computer technology, specifically to a text classification method, apparatus, computer device, and storage medium. Background Technology

[0002] Text classification is an important task in natural language processing, applicable to various scenarios and industries, such as news classification, sentiment analysis, spam filtering, advertising targeting, product positioning, content recommendation, and research literature classification. It has significant research value in practical applications.

[0003] In related technologies, text classification utilizes complex neural networks such as Transformer models and BERT. While this can achieve high text classification accuracy, the large number of parameters in these complex neural networks results in a high computational cost. Therefore, reducing the computational cost of text classification is a problem that needs to be solved. Summary of the Invention

[0004] In view of this, embodiments of the present disclosure provide a text classification method, apparatus, computer device, and storage medium.

[0005] In a first aspect, embodiments of this disclosure provide a text classification method, the method comprising:

[0006] Obtain the target text;

[0007] Using a target text classification model, the classification results of the target text are obtained, including:

[0008] Using a word embedding model, we obtain the word embedding representation of the target text based on the target text.

[0009] Using a target convolutional neural network, multi-scale features of the target text are obtained based on the word embeddings corresponding to the target text. This process includes: obtaining the target output channel of the target convolutional layer based on its input; where the target convolutional layer is any convolutional layer in the target convolutional neural network; and obtaining the target output channel based on its input includes: performing a convolution operation on the input of the target convolutional layer using each of a plurality of convolutional kernels corresponding to the position of the target output channel, thereby obtaining a sub-output channel of the target output channel corresponding to the convolutional kernel. Each convolutional kernel has a different kernel size, and the target output channel is any output channel of the target convolutional layer.

[0010] For each Hidden Markov Model in a set of Hidden Markov Models, the prediction result corresponding to the target text is obtained by using the Hidden Markov Model output according to the multi-scale features of the target text.

[0011] Using a decision tree model, the classification result of the target text is obtained based on the prediction result corresponding to the target text output by each Hidden Markov Model.

[0012] In one possible implementation, a word embedding model is used to obtain the word embedding representation of the target text, including:

[0013] For each of the multiple word embedding models, the word embedding matrix corresponding to the target text is obtained by using the word embedding model output by the word embedding model based on the target text.

[0014] Based on the word embedding matrix corresponding to the target text output by each word embedding model, the word embedding representation of the target text is obtained.

[0015] In one possible implementation, based on the word embedding matrix corresponding to the target text output by each word embedding model, the word embedding representation corresponding to the target text is obtained, including:

[0016] Based on the word embedding matrix corresponding to the target text output by each word embedding model and the adaptive parameters corresponding to each word embedding model, the word embedding representation corresponding to the target text is obtained.

[0017] In one possible implementation, using a target convolutional neural network to obtain multi-scale features of the target text based on the word embeddings corresponding to the target text also includes:

[0018] Using a pooling layer connected to the target convolutional layer, K-Max pooling is performed on each sub-output channel of the target output channel to obtain the pooling result of each sub-output channel of the target output channel;

[0019] Based on the pooling result of each sub-output channel of the target output channel and the adaptive parameters corresponding to each sub-output channel of the target output channel, the input channel in the input of the next unit of the convolutional unit to which the target convolutional layer belongs is obtained.

[0020] Secondly, embodiments of this disclosure provide a text classification device, which includes:

[0021] The acquisition unit is used to acquire the target text;

[0022] The classification unit is used to obtain the classification result of the target text based on the target text using a target text classification model. Obtaining the classification result of the target text using the target text classification model includes: obtaining the word embedding representation of the target text based on the target text using a word embedding model; and obtaining the multi-scale features of the target text based on the word embedding representation of the target text using a target convolutional neural network. Specifically, obtaining the multi-scale features of the target text based on the word embedding representation of the target text using the target convolutional neural network includes: obtaining the target output channel of the target convolutional layer based on the input of the target convolutional layer in the target convolutional neural network, wherein the target convolutional layer is any convolutional layer in the target convolutional neural network, and obtaining the target output channel based on the input of the target convolutional layer in the target convolutional neural network. The target output channel of the convolutional layer includes: for each of a plurality of convolutional kernels corresponding to the position of the target output channel of the target convolutional layer, performing a convolution operation on the input of the target convolutional layer using the convolutional kernel to obtain a sub-output channel of the target output channel corresponding to the convolutional kernel, wherein each convolutional kernel has a different convolutional kernel size, and the target output channel is any one of the output channels of the target convolutional layer; for each Hidden Markov Model in a set of Hidden Markov Models, using the Hidden Markov Model, based on the multi-scale features of the target text, obtaining the prediction result corresponding to the target text output by the Hidden Markov Model; and using a decision tree model, obtaining the classification result of the target text based on the prediction result corresponding to the target text output by each Hidden Markov Model.

[0023] In one possible implementation, the classification unit is further configured to, for each of the multiple word embedding models, use the word embedding model to obtain a word embedding matrix corresponding to the target text output by the word embedding model based on the target text; and obtain a word embedding representation corresponding to the target text based on the word embedding matrix output by each word embedding model corresponding to the target text.

[0024] In one possible implementation, the classification unit is further configured to obtain the word embedding representation corresponding to the target text based on the word embedding matrix corresponding to the target text output by each word embedding model and the adaptive parameters corresponding to each word embedding model.

[0025] In one possible implementation, the classification unit is further configured to use a pooling layer connected to the target convolutional layer to perform K-Max pooling on each sub-output channel of the target output channel to obtain the pooling result of each sub-output channel of the target output channel; and based on the pooling result of each sub-output channel of the target output channel and the adaptive parameters corresponding to each sub-output channel of the target output channel, to obtain the input channel in the input of the next unit of the convolutional unit to which the target convolutional layer belongs.

[0026] Thirdly, embodiments of this disclosure provide a computer device, including: a memory and a processor, which are communicatively connected to each other. The memory stores computer instructions, and the processor executes the computer instructions to perform the method described in the first aspect or any corresponding embodiment.

[0027] Fourthly, embodiments of this disclosure provide a computer-readable storage medium storing computer instructions for causing a computer to perform the methods described in the first aspect or any corresponding embodiment.

[0028] Fifthly, the present invention provides a computer program product, including computer instructions for causing a computer to perform the method described in the first aspect or any corresponding embodiment thereof.

[0029] The text classification method provided in this disclosure, on the one hand, compared with Transformer and BERT models, utilizes a lightweight target convolutional neural network (DCNN) and Hidden Markov Model (HMM), which requires significantly fewer parameters, thus reducing the computational burden of text classification. Simultaneously, the DCNN can extract rich multi-scale structural features from text data, while the HMM can capture contextual information and perform preliminary classification by determining the evolutionary patterns of the hidden states behind the features. The HMM leverages its efficient learning and inference capabilities. By fully utilizing the powerful feature extraction capabilities of neural network models and the efficient learning and inference capabilities of the HMM, a high prediction accuracy is ensured.

[0030] On the other hand, Hidden Markov Models (HMMs) mine global contextual information, obtaining preliminary prediction results by determining the evolutionary patterns of hidden states behind features. For different categories of text data, corresponding HMMs are trained, and all trained HMMs form an HMM group. Finally, the HMM group is deeply integrated with a Stamina ensemble learning mechanism. Specifically, one set of HMMs acts as a meta-learner, and a decision tree model acts as the top-level learner. The decision tree model makes the final category determination based on the preliminary prediction results obtained from the HMM group. This ensemble learning mechanism naturally utilizes the characteristics of each HMM in the HMM group, integrating all models in the HMM group for collective decision-making, effectively avoiding overfitting while improving prediction accuracy. In particular, the parameters and state transition probabilities in HMMs have certain semantics, which can be used to explain the model's decisions, thus improving the model's interpretability to some extent. HMMs are used to capture high-level contextual information, and preliminary classification is performed by determining the evolutionary patterns of hidden states behind features. The parameters and state transition probabilities in an Hidden Markov Model (HMM) have certain semantics, which can be used to explain the model's decisions, thus improving the model's interpretability to some extent.

[0031] On the other hand, it naturally utilizes the differences between each HMM in a set of HMMs, and integrates multiple models together for group decision-making through the Stamina ensemble learning mechanism, which can improve prediction accuracy and effectively solve the problem of overfitting. Attached Figure Description

[0032] To more clearly illustrate the technical solutions in the specific embodiments of this disclosure or the prior art, the drawings used in the description of the specific embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of this disclosure. For those skilled in the art, other drawings can be obtained from these drawings without creative effort.

[0033] Figure 1 This is a flowchart illustrating the text classification method provided in this embodiment of the disclosure;

[0034] Figure 2 This is a schematic diagram illustrating an example of the structure of a target convolutional neural network;

[0035] Figure 3 This is a flowchart illustrating an example of obtaining prediction results from the HMM output in a set of HMMs;

[0036] Figure 4 This is a flowchart illustrating an example of obtaining the classification result of a target text using a target text classification model;

[0037] Figure 5This is a schematic diagram of the structure of a computer device provided in an embodiment of this disclosure. Detailed Implementation

[0038] To make the objectives, technical solutions, and advantages of the embodiments of this disclosure clearer, the technical solutions of the embodiments of this disclosure will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this disclosure, and not all embodiments. Based on the embodiments of this disclosure, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this disclosure.

[0039] refer to Figure 1 The diagram illustrates an example flowchart of a text classification method provided in an embodiment of this disclosure.

[0040] In step S101, the target text is obtained.

[0041] In step S102, the target text classification model is used to obtain the classification result of the target text based on the target text.

[0042] Target text classification models include: word embedding models, target convolutional neural networks, a set of Hidden Markov Models (HMMs), and decision tree models. In a target text classification model, each HMM in a set can correspond to a different text category. Each HMM in a set of target text classification models has corresponding parameters. As an example, HMM_1 in a set of HMMs has parameter λ1, HMM_2 in a set of HMMs has parameter λ2, and HMM_M in a set of HMMs has parameter M.

[0043] Step S102 includes: steps S1021-S1024.

[0044] In step S1021, the word embedding model is used to obtain the word embedding representation corresponding to the target text based on the target text.

[0045] In one possible implementation, the word embedding model is one of the following: Word2Vec, GloVe, or FastText. The target text is input into the word embedding model to obtain the word embedding representation of the target text.

[0046] In another possible implementation, using word embedding models to obtain the word embedding representation of the target text includes: for each of the multiple word embedding models, using that word embedding model to obtain the word embedding matrix corresponding to the target text output by that word embedding model; and obtaining the word embedding representation of the target text based on the word embedding matrix corresponding to the target text output by each word embedding model.

[0047] The word embedding matrix output by each word embedding model corresponding to the target text can be calculated accordingly. For example, the corresponding matrix elements in the word embedding matrix output by each word embedding model corresponding to the target text can be added position by position to obtain the word embedding representation corresponding to the target text.

[0048] As an example, multiple word embedding models can include: Word2Vec, GloVe, and FastText. The word embedding model can perform word vectorization on the target text, resulting in a word embedding matrix, where each row represents a word in the text data. Padding and truncation operations are then performed on the word embedding matrix corresponding to the target text output by the word embedding model to obtain a matrix of the required input dimensions for a convolutional neural network.

[0049] In one possible implementation, obtaining the word embedding representation of the target text based on the word embedding matrix corresponding to the target text output by each word embedding model includes: obtaining the word embedding representation of the target text based on the word embedding matrix corresponding to the target text output by each word embedding model and the adaptive parameters corresponding to each word embedding model.

[0050] The adaptive parameters corresponding to each word embedding model can be used as the weights of each word embedding model, and the weighted sum of the word embedding matrices output by each word embedding model corresponding to the target text can be determined as the word embedding representation of the target text.

[0051] In this embodiment of the disclosure, by setting adaptive parameters corresponding to each word embedding model, the generalization ability of the algorithm can be guaranteed, and the model can be trained to move closer to a more suitable word embedding matrix.

[0052] In step S1022, the target convolutional neural network is used to obtain the multi-scale features of the target text based on the word embedding representation corresponding to the target text.

[0053] It should be noted that the target convolutional neural network provided in this embodiment may be called mTextCNN.

[0054] The word embeddings corresponding to the target text are used as input to the target convolutional neural network. In step S1022, the word embeddings corresponding to the target text are input into the target convolutional neural network, and the target convolutional neural network outputs the multi-scale features of the target text.

[0055] Step S1022 includes: Step S10221.

[0056] In step S10221, using the target convolutional neural network, the multi-scale features of the target text are obtained based on the word embedding representation corresponding to the target text, including: obtaining the target output channel of the target convolutional layer in the target convolutional neural network based on the input of the target convolutional layer in the target convolutional neural network.

[0057] Specifically, obtaining the target output channel of the target convolutional layer based on the input of the target convolutional layer in the target convolutional neural network includes: for each of the multiple convolutional kernels corresponding to the position of the target output channel of the target convolutional layer, performing a convolution operation on the input of the target convolutional layer using the convolutional kernel to obtain a sub-output channel of the target output channel of the target convolutional layer corresponding to the convolutional kernel, wherein each of the multiple convolutional kernels corresponding to the position of the target output channel of the target convolutional layer has a different convolutional kernel size; and obtaining the target output channel of the target convolutional layer based on each sub-output channel of the target output channel of the target convolutional layer.

[0058] The output of the target convolutional layer consists of the output channels of the target convolutional layer, and the output channels of the target convolutional layer include the features output by the target convolutional layer.

[0059] The target output channel is any one of the output channels of the target convolutional layer. The position indicator of the target output channel of the target convolutional layer indicates which output channel it is.

[0060] In this embodiment, the target convolutional neural network includes one or more convolutional units. The target convolutional neural network includes fully connected layers. To prevent overfitting, a random failure module is added to the fully connected network, and ReLU is used as the activation function. A convolutional unit in the target convolutional neural network includes a convolutional layer and a pooling layer connected to that convolutional layer. When the target convolutional neural network includes multiple convolutional units, the number of convolutional units in the target convolutional neural network is denoted as n, and the output of the i-th convolutional unit is used as the input of the (i+1)-th convolutional unit, where i is one of 1…n-1. The input of the convolutional layer in the first convolutional unit of the target convolutional neural network is the input of the target convolutional neural network, and the output of the last convolutional unit is the input of the fully connected layer in the target convolutional neural network.

[0061] In this embodiment of the disclosure, for each of the plurality of convolutional kernels corresponding to the position of the output channel h of the convolutional layer j, the input of the convolutional layer j is convolved using the convolutional kernel to obtain a sub-output channel of the output channel h corresponding to the convolutional kernel, wherein each of the plurality of convolutional kernels corresponding to the position of the output channel h of the convolutional layer j has a different convolutional kernel size.

[0062] Therefore, each sub-output channel of channel h constitutes output channel h.

[0063] As an example, the multiple convolutional kernels corresponding to the position h of the output channel of convolutional layer j consist of a 3×3 convolutional kernel corresponding to the position m of the output channel of convolutional layer j, a 5×5 convolutional kernel corresponding to the position m of the output channel of convolutional layer j, and a 7×7 convolutional kernel corresponding to the position h of the output channel of convolutional layer j. The input of convolutional layer j is convolved using the 3×3 convolutional kernel corresponding to the position h of the output channel of convolutional layer j, resulting in a sub-output channel h corresponding to the 3×3 convolutional kernel. Similarly, the input of convolutional layer j is convolved using the 5×5 convolutional kernel corresponding to the position h of the output channel of convolutional layer j, resulting in a sub-output channel h corresponding to the 5×5 convolutional kernel. Finally, the input of convolutional layer j is convolved using the 7×7 convolutional kernel corresponding to the position h of the output channel of convolutional layer j, resulting in a sub-output channel h corresponding to the 7×7 convolutional kernel.

[0064] In one possible implementation, the classification result of the target text using the target text classification model further includes: using a pooling layer connected to the target convolutional layer to perform K-Max pooling on each sub-output channel of the target output channel of the target convolutional layer, and obtaining the pooling result of each sub-output channel of the target output channel of the target convolutional layer; and obtaining the input channel in the input of the next unit of the convolutional unit to which the target convolutional layer belongs based on the pooling result of each sub-output channel of the target convolutional layer and the adaptive parameters corresponding to each sub-output channel of the target convolutional layer.

[0065] The adaptive parameters corresponding to the sub-output channels of the target output channel of the target convolutional layer can be used as the weights of the sub-output channels of the target output channel of the target convolutional layer. The weighted sum of the pooling results of each sub-output channel of the target output channel of the target convolutional layer is determined as the input channel in the input of the next unit of the convolutional unit to which the target convolutional layer belongs.

[0066] The pooling layer connected to the target convolutional layer can be the convolutional kernel in the convolutional unit to which the target convolutional layer belongs.

[0067] If the convolutional unit to which the target convolutional layer belongs is the last convolutional unit, then the next unit after the target convolutional layer is the fully connected layer of the target convolutional neural network. If the convolutional unit to which the target convolutional layer belongs is not the last convolutional unit, then the next unit after the target convolutional layer is the next convolutional unit connected to the target convolutional layer.

[0068] In this embodiment, the feature representation capability and robustness of the model are further improved by using adaptive parameters corresponding to each sub-output channel of the target output channel. The target convolutional neural network can extract rich multi-scale features with less computational resources. K-Max pooling differs from traditional max pooling by retaining the top M feature values ​​in the feature vector, which further enhances the model's robustness. After pooling, an adaptive parameter is assigned to the feature vectors obtained from convolutional layers of different scales, and then a concatenation operation is performed to obtain a combined feature vector. In particular, the adaptive parameter can improve the algorithm's feature representation capability.

[0069] refer to Figure 2 The diagram illustrates an example of the structure of a target convolutional neural network.

[0070] Figure 2 The multi-channel word embedding matrix is ​​shown, which represents the word embeddings of the target text.

[0071] Figure 2 The diagram illustrates a convolutional layer of a target convolutional neural network, and several convolutional kernels corresponding to the position of an output channel of that convolutional layer, including a 3×k kernel, a 5×k kernel, and a 7×k kernel. The input of the convolutional layer is convolved using the 3×k kernel corresponding to the output channel, resulting in a sub-output channel corresponding to that kernel. Similarly, the input of the convolutional layer is convolved using the 5×k kernel, and the input is convolved using the 7×k kernel, resulting in a sub-output channel corresponding to that kernel.

[0072] The convolutional unit to which this belongs is the last convolutional unit, and the unit following the convolutional unit to which this convolutional layer belongs is the fully connected layer of the target convolutional neural network.

[0073] The adaptive parameters corresponding to the sub-output channels of the target output channel of the target convolutional layer are used as the weights of the sub-output channels of the target output channel of the target convolutional layer, and the weighted sum of the pooling results of each sub-output channel of the target output channel of the target convolutional layer is calculated.

[0074] Figure 2 The diagram shows the weighted sum of the pooling results for each sub-output channel of the target output channel of the target convolutional layer, i.e., the feature vector combined with adaptive parameters.

[0075] The feature vectors, which are combined with adaptive parameters, are used as input channels in the fully connected layers of the target convolutional neural network.

[0076] In step S1023, for each HMM in a set of HMMs in the target text classification model, the prediction result corresponding to the target text is obtained by using the HMM output according to the multi-scale features of the target text.

[0077] For each HMM in a set of HMMs in the target text classification model, the multi-scale features of the target text are input into the HMM to obtain the prediction result corresponding to the target text output by the HMM.

[0078] In step S1024, the decision tree model in the target text classification model is used to obtain the classification result of the target text based on the prediction result corresponding to the target text output by each Hidden Markov model.

[0079] In the target text classification model, the decision tree model receives the prediction results corresponding to the target text output by each Hidden Markov Model, and outputs the classification result of the target text.

[0080] In this embodiment of the disclosure, before the text classification method provided in this embodiment is executed for the first time, an mTextCNN model is trained using a text training set. During the training of the target convolutional neural network, training text is input to the mTextCNN model, and the output of the mTextCNN model is used as the input to the Softmax layer, which outputs the probability of each text category. Using the cross-entropy function, the loss corresponding to the training text is calculated based on the probability of each text category output by the Softmax layer and the label of the training text. Using the backpropagation algorithm, the parameters of the mTextCNN network are updated based on the losses corresponding to multiple training texts.

[0081] In this embodiment of the disclosure, after the target convolutional neural network is trained, a set of HMMs is trained using the training text set.

[0082] For ease of explanation, the following will illustrate how HMM works by describing its training and testing processes. In the previous step, the feature vector output by the mTextCNN model was used as the observation sequence input into the HMM.

[0083] In this context, let o = [o1, o2, ..., o...] M (1)

[0084] In this model, the Marmov chain is assumed to have M observation states and N hidden states. During the training phase, the parameters of the HMM corresponding to each text class are initialized to λ = (π, A, B).

[0085] π = [π n [] represents the initial probability value for each hidden state;

[0086] π n =P(q) t =s n ), 1≤n≤N

[0087] Where, q t This represents a hidden state variable. Furthermore, A = [a nh ] N×N Represents the state transition matrix;

[0088] a nh =P(q) t+1 =s h |q t =s n ), 1≤n, h≤N

[0089] B = [b] nm ] N×M Represents the observation probability matrix, b nm =P(p t =o m |q t =s n ), 1≤n≤N, 1≤m≤M, where p t This represents the observed variable.

[0090] The main task of training a Hidden Markov Model (HMM) is to use the feature vectors output by mTextCNN as observation sequences to update the parameter λ. The training set includes training texts of N text types. During HMM training, the training texts in the training text set are input into mTextCNN. The training text set of the HMM includes training texts of N text categories. The multi-scale features, i.e., the multi-scale feature vectors, of the training texts output by mTextCNN are used as observation vectors input into the HMM. In the HMM, the parameter λ corresponding to the highest probability p(o|λ) is obtained according to the Baum-Welch algorithm. It can be considered that the probability p(o|λ) reaches its maximum value when the difference between the probability values ​​obtained in two consecutive iterations is less than a certain predetermined threshold. For each text category in the training text set, the optimal parameters of the HMM are calculated using the same method described above, resulting in a matching HMM for that text category. In other words, for each text category, an optimal HMM will be trained. Alternatively, the training text set involves each text type, and training is performed using the training texts of that text type to obtain the HMM corresponding to that text type. In this context, for a given text type, if the training text set includes training texts of that text type, then that text type is considered part of the training text set. If the training dataset contains N different types of text data, then N Hidden Markov Models (HMMs) can be obtained, and these N HMMs form a set of HMMs. When a new training text is input into a set of HMMs, the set of HMMs outputs N predicted probabilities, each corresponding to one of the N text types. Ultimately, the HMMs corresponding to the training texts of all categories in the training text set constitute a set of HMMs, used to distinguish text data across all text categories.

[0091] After training a set of Hidden Markov Models (HMMs), a decision tree model is trained using the Stamina ensemble learning method. A label inference module is then implemented based on the Stamina ensemble learning mechanism. Ensemble learning is a technique that uses multiple learners or classifiers to perform the same task, combining the results of all learners or classifiers through a certain strategy to make a final decision. Currently, common types of ensemble learning include Bagging (Bootstrap Aggregating), Boosting, Bayesian Model Combination, and Stamina. Stamina ensemble learning is chosen for two reasons: first, it effectively suppresses overfitting; second, the various types of HMMs within a set are natural meta-learners, perfectly fitting the framework of Stamina ensemble learning. Assuming a set of HMMs consists of M different sets of HMMs, the number of meta-learners in Stamina is M. For each input sample, a set of HMMs will obtain M different probability values, which, along with the sample's label, will be provided as training data to the top-level learner.

[0092] For any sample in the training set, a Hidden Markov Model (HMM) will output N predicted probabilities. The training data for the decision tree model consists of the N predicted values ​​from the HMM and the true labels of the samples. This data is used to train the decision tree model. The decision tree model uses 5-fold cross-validation for both training and testing.

[0093] After training mTextCNN, a set of Hidden Markov Models (HMMs), and a decision tree model, testing was conducted. During testing, the data in the test set was input into the mTextCNN model, which extracted multi-scale features from the text data. These multi-scale features, i.e., multi-scale feature vectors, were then input as observation sequences into each of the HMMs in the set. The set of HMMs used a forward-backward algorithm to obtain multiple predicted values. Each class of HMMs in the set combined the observation sequences and used a forward-backward algorithm to obtain the maximum posterior probability corresponding to each text category, thus obtaining a preliminary classification result. Finally, the preliminary prediction results from the set of HMMs were input into the decision tree model for the final category determination.

[0094] refer to Figure 3 The flowchart shows an example of obtaining prediction results from the HMM outputs in a set of HMMs.

[0095] Text data is input into mTextCNN. mTextCNN outputs multi-scale feature vectors of the text data. The multi-scale feature vectors output by mTextCNN are then input into a set of Hidden Markov Models (HMMs). In the set of HMMs, HMM_1 outputs a prediction result with probability value 1. In the set of HMMs, HMM_2 outputs a prediction result with probability value 2. In the set of HMMs, HMM_M outputs a prediction result with probability value M. HMM_1 has parameter λ1, HMM_2 has parameter λ2, and HMM_M has parameter M.

[0096] refer to Figure 4 The flowchart shows an example of obtaining the classification result of target text using a target text classification model.

[0097] The target text is input into the target text classification model mTextCNN. mTextCNN outputs a multi-scale feature vector of the target text. The multi-scale feature vector output by mTextCNN is then input into each of a set of Hidden Markov Models (HMMs) in the target text classification model. The set of HMMs in the target text classification model consists of HMM_0 for category 0, HMM_1 for category 1, ..., HMM_1_M for category M. Categories 0, 1, ..., M are all text categories.

[0098] The decision tree model in the target text classification model receives the prediction results output by each HMM in a set of HMMs in the target text classification model, and predicts the classification result of the target text based on the prediction results output by each HMM in a set of HMMs in the target text classification model.

[0099] This disclosure provides a text classification device. This device is used to implement the above embodiments and preferred embodiments, and details already described will not be repeated. As used below, the term "unit" can refer to a combination of software and / or hardware that performs a predetermined function. Although the device described in the following embodiments is preferably implemented in software, hardware implementation, or a combination of software and hardware, is also possible and contemplated.

[0100] The text classification device includes:

[0101] The acquisition unit is used to acquire the target text;

[0102] The classification unit is used to obtain the classification result of the target text based on the target text using a target text classification model. Obtaining the classification result of the target text using the target text classification model includes: obtaining the word embedding representation of the target text based on the target text using a word embedding model; and obtaining the multi-scale features of the target text based on the word embedding representation of the target text using a target convolutional neural network. Specifically, obtaining the multi-scale features of the target text based on the word embedding representation of the target text using the target convolutional neural network includes: obtaining the target output channel of the target convolutional layer based on the input of the target convolutional layer in the target convolutional neural network, wherein the target convolutional layer is any convolutional layer in the target convolutional neural network, and obtaining the target output channel based on the input of the target convolutional layer in the target convolutional neural network. The target output channel of the convolutional layer includes: for each of a plurality of convolutional kernels corresponding to the position of the target output channel of the target convolutional layer, performing a convolution operation on the input of the target convolutional layer using the convolutional kernel to obtain a sub-output channel of the target output channel corresponding to the convolutional kernel, wherein each convolutional kernel has a different convolutional kernel size, and the target output channel is any one of the output channels of the target convolutional layer; for each Hidden Markov Model in a set of Hidden Markov Models, using the Hidden Markov Model, based on the multi-scale features of the target text, obtaining the prediction result corresponding to the target text output by the Hidden Markov Model; and using a decision tree model, obtaining the classification result of the target text based on the prediction result corresponding to the target text output by each Hidden Markov Model.

[0103] In one possible implementation, the classification unit is further configured to, for each of the multiple word embedding models, use the word embedding model to obtain a word embedding matrix corresponding to the target text output by the word embedding model based on the target text; and obtain a word embedding representation corresponding to the target text based on the word embedding matrix output by each word embedding model corresponding to the target text.

[0104] In one possible implementation, the classification unit is further configured to obtain the word embedding representation corresponding to the target text based on the word embedding matrix corresponding to the target text output by each word embedding model and the adaptive parameters corresponding to each word embedding model.

[0105] In one possible implementation, the classification unit is further configured to use a pooling layer connected to the target convolutional layer to perform K-Max pooling on each sub-output channel of the target output channel to obtain the pooling result of each sub-output channel of the target output channel; and based on the pooling result of each sub-output channel of the target output channel and the adaptive parameters corresponding to each sub-output channel of the target output channel, to obtain the input channel in the input of the next unit of the convolutional unit to which the target convolutional layer belongs.

[0106] In this embodiment, the device is presented in the form of a functional unit. Here, a unit refers to an ASIC circuit, a processor and memory that execute one or more software or fixed programs, and / or other devices that can provide the above-mentioned functions.

[0107] Further functional descriptions of the above-mentioned units are the same as those in the corresponding embodiments described above, and will not be repeated here.

[0108] refer to Figure 5 This illustration shows a schematic diagram of a computer device provided in an embodiment of the present disclosure. The computer device includes one or more processors 10, a memory 20, and interfaces for connecting the components, including high-speed interfaces and low-speed interfaces. The components are interconnected via different buses and can be mounted on a common motherboard or otherwise installed as needed. The processors can process instructions executed within the computer device, including instructions stored in or on memory to display graphical information of a GUI on an external input / output device (such as a display device coupled to the interface). In some alternative embodiments, multiple processors and / or multiple buses can be used with multiple memories and multiple memory modules, if desired. Similarly, multiple computer devices can be connected, each providing some of the necessary operations (e.g., as a server array, a group of blade servers, or a multiprocessor system).

[0109] Processor 10 may be a central processing unit, a network processor, or a combination thereof. Processor 10 may further include a hardware chip. The hardware chip may be an application-specific integrated circuit (ASIC), a programmable logic device (PLD), or a combination thereof. The programmable logic device may be a complex programmable logic device (CAMP), a field-programmable gate array (FPGA), a general-purpose array logic (GPA), or any combination thereof.

[0110] The memory 20 stores instructions executable by at least one processor 10 to cause the at least one processor 10 to perform the method shown in the above embodiments.

[0111] The memory 20 may include a program storage area and a data storage area. The program storage area may store the operating system and applications required for at least one function; the data storage area may store data created based on the use of the computer device. Furthermore, the memory 20 may include high-speed random access memory and may also include non-transitory memory, such as at least one disk storage device, flash memory device, or other non-transitory solid-state storage device. In some alternative embodiments, the memory 20 may optionally include memory remotely located relative to the processor 10, and these remote memories may be connected to the computer device via a network. Examples of such networks include, but are not limited to, the Internet, intranets, local area networks, mobile communication networks, and combinations thereof.

[0112] The memory 20 may include volatile memory, such as random access memory; the memory may also include non-volatile memory, such as flash memory, hard disk or solid-state drive; the memory 20 may also include a combination of the above types of memory.

[0113] The computer device also includes an input device 30 and an output device 40. The processor 10, memory 20, input device 30, and output device 40 can be connected via a bus or other means.

[0114] Input device 30 can receive input numerical or character information, and generate key signal inputs related to user settings and function control of the computer device, such as a touchscreen, keypad, mouse, trackpad, touchpad, joystick, one or more mouse buttons, trackball, joystick, etc. Output device 40 may include display devices, auxiliary lighting devices (e.g., LEDs), and haptic feedback devices (e.g., vibration motors). The aforementioned display devices include, but are not limited to, liquid crystal displays, light-emitting diodes, displays, and plasma displays. In some alternative embodiments, the display device may be a touchscreen.

[0115] This disclosure also provides a computer-readable storage medium in which the methods described in this disclosure can be implemented in hardware or firmware, or implemented as recordable on a storage medium, or implemented as computer code originally stored on a remote storage medium or a non-transitory machine-readable storage medium and subsequently stored on a local storage medium after being downloaded over a network. Thus, the methods described herein can be processed by software stored on a storage medium using a general-purpose computer, a dedicated processor, or programmable or dedicated hardware. The storage medium may be a magnetic disk, optical disk, read-only memory, random access memory, flash memory, hard disk, or solid-state drive, etc.; further, the storage medium may also include combinations of the above types of memory. It is understood that computers, processors, microprocessor controllers, or programmable hardware include storage components capable of storing or receiving software or computer code that, when accessed and executed by the computer, processor, or hardware, implements the methods shown in the above embodiments.

[0116] A portion of the embodiments disclosed herein can be applied to computer program products, such as computer program instructions, which, when executed by a computer, can invoke or provide the methods and / or technical solutions according to the present invention through the operation of the computer. Those skilled in the art will understand that the forms in which computer program instructions exist in a computer-readable medium include, but are not limited to, source files, executable files, installation package files, etc. Correspondingly, the ways in which computer program instructions are executed by a computer include, but are not limited to: the computer directly executing the instructions, or the computer compiling the instructions and then executing the corresponding compiled program, or the computer reading and executing the instructions, or the computer reading and installing the instructions and then executing the corresponding installed program. Here, the computer-readable medium can be any available computer-readable storage medium or communication medium accessible to a computer.

[0117] Although embodiments of the present disclosure have been described in conjunction with the accompanying drawings, those skilled in the art can make various modifications and variations without departing from the spirit and scope of the present disclosure, and such modifications and variations all fall within the scope defined by the appended claims.

Claims

1. A method of text classification, characterized by, The method comprises: acquiring a target text; using a target text classification model, obtaining a classification result of the target text according to the target text, comprising: using a word embedding model, obtaining a word embedding representation corresponding to the target text according to the target text; using a target convolutional neural network, obtaining a multi-scale feature of the target text according to the word embedding representation corresponding to the target text, wherein using the target convolutional neural network, obtaining the multi-scale feature of the target text according to the word embedding representation corresponding to the target text comprises: obtaining a target output channel of a target convolutional layer in the target convolutional neural network according to an input of the target convolutional layer, wherein the target convolutional layer is any convolutional layer in the target convolutional neural network, and obtaining the target output channel of the target convolutional layer according to the input of the target convolutional layer comprises: for each convolutional kernel of a plurality of convolutional kernels corresponding to a position of the target output channel of the target convolutional layer, performing convolutional operation on the input of the target convolutional layer using the convolutional kernel to obtain a sub-output channel of the target output channel corresponding to the convolutional kernel, wherein each convolutional kernel has a different convolutional kernel size, and the target output channel is any output channel of the target convolutional layer; for each hidden Markov model in a group of hidden Markov models, using the hidden Markov model, obtaining a prediction result output by the hidden Markov model corresponding to the target text according to the multi-scale feature of the target text; using a decision tree model, obtaining the classification result of the target text according to the prediction result output by each hidden Markov model corresponding to the target text.

2. The method of claim 1, wherein, Using a word embedding model, obtaining a word embedding representation corresponding to the target text according to the target text comprises: for each word embedding model in a plurality of word embedding models, using the word embedding model, obtaining a word embedding matrix output by the word embedding model corresponding to the target text according to the target text; obtaining the word embedding representation corresponding to the target text according to the word embedding matrix output by each word embedding model corresponding to the target text.

3. The method of claim 2, wherein, Obtaining the word embedding representation corresponding to the target text according to the word embedding matrix output by each word embedding model corresponding to the target text comprises: obtaining the word embedding representation corresponding to the target text according to the word embedding matrix output by each word embedding model corresponding to the target text and the adaptive parameters corresponding to each word embedding model.

4. The method of claim 1, wherein, Using a target text classification model, obtaining a classification result of the target text according to the target text further comprises: using a pooling layer connected to the target convolutional layer, performing K-Max pooling on each sub-output channel of the target output channel respectively to obtain a pooling result of each sub-output channel of the target output channel; obtaining an input channel in an input of a next unit of a convolutional unit to which the target convolutional layer belongs according to the pooling result of each sub-output channel of the target output channel and the adaptive parameters corresponding to each sub-output channel of the target output channel.

5. A text classification apparatus characterized by comprising: The device comprises: an acquisition unit configured to acquire a target text; The classification unit is configured to obtain a classification result of the target text according to the target text by using a target text classification model, and the obtaining of the classification result of the target text by using the target text classification model according to the target text includes: obtaining a word embedding representation corresponding to the target text according to the target text by using a word embedding model; and obtaining a multi-scale feature of the target text according to the word embedding representation corresponding to the target text by using a target convolutional neural network, wherein the obtaining of the multi-scale feature of the target text according to the word embedding representation corresponding to the target text by using the target convolutional neural network includes: obtaining a target output channel of a target convolutional layer in the target convolutional neural network according to an input of the target convolutional layer, wherein the target convolutional layer is any convolutional layer in the target convolutional neural network, and the obtaining of the target output channel of the target convolutional layer according to the input of the target convolutional layer in the target convolutional neural network includes: for each convolutional kernel of a plurality of convolutional kernels corresponding to a position of the target output channel of the target convolutional layer, performing a convolution operation on the input of the target convolutional layer by using the convolutional kernel to obtain a sub-output channel of the target output channel corresponding to the convolutional kernel, wherein each convolutional kernel has a different convolutional kernel size, and the target output channel is any output channel of the target convolutional layer; for each hidden Markov model in a group of hidden Markov models, obtaining a prediction result output by the hidden Markov model corresponding to the target text according to the multi-scale feature of the target text by using the hidden Markov model; and obtaining the classification result of the target text according to the prediction result output by each hidden Markov model corresponding to the target text by using a decision tree model.

6. The apparatus of claim 5, wherein, The classification unit is further configured to, for each word embedding model in a plurality of word embedding models, obtain a word embedding matrix output by the word embedding model corresponding to the target text according to the target text by using the word embedding model; and obtain the word embedding representation corresponding to the target text according to the word embedding matrix output by each word embedding model corresponding to the target text.

7. The apparatus of claim 6, wherein, The classification unit is further configured to obtain the word embedding representation corresponding to the target text according to the word embedding matrix output by each word embedding model corresponding to the target text and an adaptive parameter corresponding to each word embedding model.

8. A computer device, comprising: The memory and the processor are communicatively connected, the memory stores computer instructions, and the processor executes the computer instructions to perform the method in any one of claims 1 to 4. The computer readable storage medium stores computer instructions for causing a computer to perform the method in any one of claims 1 to 4.

9. A computer-readable storage medium, characterized in that, The computer instructions are used to cause a computer to perform the method in any one of claims 1 to 4.

10. A computer program product, characterised in that, ​

Images