A text classification method, device and storage medium

Abstract

The present invention provides a text classification method, device and storage medium, belonging to the field of text classification, and the method comprises: S1: dividing a plurality of original text data to obtain a training set and a test set; S2: performing data preprocessing on the training set and the test set respectively to obtain a plurality of word training matrices and a plurality of word test matrices; S3: performing vectorization processing on each word training matrix respectively to obtain a word vector matrix; S4: constructing a training model, training the training model according to a plurality of word vector matrices to obtain a model to be tested; S5: testing the model to be tested according to a plurality of word test matrices to obtain a classification model; S6: importing the text data to be classified, and classifying the text data to be classified by the classification model to obtain a classification result. The present invention can highlight the key information in the text, enhance the text context semantic information, and more accurately represent the text semantics, thereby improving the accuracy of text classification.

Description

Technical Field

[0001] The present invention mainly relates to the technical field of text classification, and in particular to a text classification method, device and storage medium. Background Art

[0002] In today's era of big data with massive amounts of information, how to obtain target text from massive text is a very important and challenging problem. Text classification is a hot issue in natural language processing and is widely used in sentiment analysis, topic classification, question-answer matching, etc. However, Chinese text has the problems of semantic ambiguity, sparse features, and high dimensionality. Therefore, traditional text classification methods often rely on manually designed features, resulting in poor classification results.

[0003] In recent years, a large number of text classification models based on deep learning have emerged, and their accuracy and precision in various classification tasks have exceeded those of traditional machine learning methods. However, most of the text classification methods based on deep learning focus on a single model structure, which lacks the ability to capture and utilize both global and local semantic features at the same time, thus reducing the accuracy of text classification. Summary of the invention

[0004] The technical problem to be solved by the present invention is to provide a text classification method, device and storage medium in view of the deficiencies in the prior art.

[0005] The technical solution of the present invention to solve the above technical problem is as follows: A text classification method comprises the following steps:

[0006] S1: importing a plurality of original text data, and dividing the plurality of original text data into a training set and a test set;

[0007] S2: performing data preprocessing on the training set and the test set respectively to obtain a plurality of word training matrices corresponding to the training set and a plurality of word test matrices corresponding to the test set;

[0008] S3: performing vectorization processing on each of the word training matrices respectively to obtain a word vector matrix corresponding to each of the word training matrices;

[0009] S4: constructing a training model, and training the training model according to the plurality of word vector matrices to obtain a model to be tested;

[0010] S5: Testing the model to be tested according to the multiple word test matrices to obtain a classification model;

[0011] S6: importing the text data to be classified, and classifying the text data to be classified using the classification model to obtain a classification result.

[0012] Another technical solution of the present invention to solve the above technical problem is as follows: A text classification device, comprising:

[0013] A data partitioning module, used for importing a plurality of original text data and partitioning the plurality of the original text data to obtain a training set and a test set;

[0014] A data preprocessing module is used to perform data preprocessing on the training set and the test set respectively to obtain a plurality of word training matrices corresponding to the training set and a plurality of word test matrices corresponding to the test set;

[0015] A vectorization processing module, used to perform vectorization processing on each of the word training matrices to obtain a word vector matrix corresponding to each of the word training matrices;

[0016] A model training module is used to construct a training model, and train the training model according to the plurality of word vector matrices to obtain a model to be tested;

[0017] A model testing module, used for testing the model to be tested according to a plurality of word test matrices to obtain a classification model;

[0018] The classification result obtaining module is used to import the text data to be classified, classify the text data to be classified through the classification model, and obtain the classification result.

[0019] Another technical solution of the present invention to solve the above technical problem is as follows: a text classification device includes a memory, a processor, and a computer program stored in the memory and executable on the processor. When the processor executes the computer program, the text classification method described above is implemented.

[0020] Another technical solution of the present invention to solve the above technical problem is as follows: a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the text classification method as described above is implemented.

[0021] The beneficial effects of the present invention are as follows: a training set and a test set are obtained by dividing a plurality of original text data, data of the training set and the test set are preprocessed respectively to obtain a plurality of word training matrices and a word test matrix, each word training matrix is ​​vectorized to obtain a word vector matrix, a training model is trained by a plurality of word vector matrices to obtain a model to be tested, a classification model is obtained by testing the model to be tested by a plurality of word test matrices, and a classification result is obtained by classifying text data to be classified by the classification model, which can highlight key information in the text, enhance text context semantic information, and more accurately represent text semantics, thereby improving the accuracy of text classification. BRIEF DESCRIPTION OF THE DRAWINGS

[0022] Figure 1 A flowchart of a text classification method provided by an embodiment of the present invention;

[0023] Figure 2 A module block diagram of a text classification device provided by an embodiment of the present invention. DETAILED DESCRIPTION

[0024] The principles and features of the present invention are described below in conjunction with the accompanying drawings. The examples given are only used to explain the present invention and are not used to limit the scope of the present invention.

[0025] Figure 1 A flowchart of a text classification method provided by an embodiment of the present invention.

[0026] like Figure 1 As shown, a text classification method includes the following steps:

[0027] S1: importing a plurality of original text data, and dividing the plurality of original text data into a training set and a test set;

[0028] S2: performing data preprocessing on the training set and the test set respectively to obtain a plurality of word training matrices corresponding to the training set and a plurality of word test matrices corresponding to the test set;

[0029] S3: performing vectorization processing on each of the word training matrices respectively to obtain a word vector matrix corresponding to each of the word training matrices;

[0030] S4: constructing a training model, and training the training model according to the plurality of word vector matrices to obtain a model to be tested;

[0031] S5: Testing the model to be tested according to the multiple word test matrices to obtain a classification model;

[0032] S6: importing the text data to be classified, and classifying the text data to be classified using the classification model to obtain a classification result.

[0033] It should be understood that when establishing a classified text corpus, 80% of the text (ie, the original text data) is used as a training set, and 20% of the text (ie, the original text data) is used as a test set.

[0034] In the above embodiment, a training set and a test set are obtained by dividing multiple original text data, and the data of the training set and the test set are preprocessed to obtain multiple word training matrices and word test matrices, respectively. Each word training matrix is ​​vectorized to obtain a word vector matrix, and a training model is trained by multiple word vector matrices to obtain a model to be tested, and a classification model is obtained by testing the model to be tested by multiple word test matrices. The classification model is used to classify the text data to be classified to obtain a classification result, which can highlight the key information in the text, enhance the text context semantic information, and more accurately represent the text semantics, thereby improving the accuracy of text classification.

[0035] Optionally, as an embodiment of the present invention, the training set includes a plurality of original text training data, the test set includes a plurality of original text test data, and the process of step S2 includes:

[0036] Use Python tools to perform word segmentation processing on each of the original text training data and each of the original text test data, to obtain a plurality of to-be-screened word training data corresponding to each of the original text training data and a plurality of to-be-screened word test data corresponding to each of the original text test data;

[0037] The python tool is used to screen stop words for each of the word training data to be screened and each of the word test data to be screened, to obtain a plurality of word training data corresponding to each of the original text training data and a plurality of word test data corresponding to each of the original text test data, and a word training matrix corresponding to each of the original text training data is obtained according to the plurality of word training data, and a word test matrix corresponding to each of the original text test data is obtained according to the plurality of word test data.

[0038] It should be understood that the Jieba natural language processing word segmentation library in the python tool is used to perform word segmentation processing and stop word screening.

[0039] It should be understood that Jieba Chinese word segmentation component can quickly scan out all the words that can be formed into words in a sentence, and at the same time try to segment the sentence most accurately, especially for long words, which can be segmented again, and the accuracy of word segmentation is relatively high.

[0040] It should be understood that data preprocessing is performed on the input text (ie, the original text training data or the original text test data).

[0041] It should be understood that a Chinese word segmenter (such as the Jieba natural language processing word segmentation library in Python) is used to segment the text (i.e., the original text training data or the original text test data), that is, a sequence of Chinese characters (sentence) is segmented into individual words (i.e., the word training data to be screened or the word test data to be screened) and saved.

[0042] Specifically, after normal word segmentation using a Chinese word segmenter (such as the Jieba natural language processing word segmentation library in Python), a stop word dictionary is established, that is, a set of words to be removed. Then, the stop words are removed from the word segmentation results (i.e., the word training data to be screened or the word test data to be screened), and the remaining words (i.e., the word training data or the word test data) are saved. The stop words mainly include some meaningless adverbs, adjectives, conjunctions, such as "de", "ne", etc.

[0043] In the above embodiments, the data preprocessing of the training set and the test set respectively obtains multiple word training matrices and multiple word test matrices, providing accurate data for subsequent data processing, being able to represent the text semantics more accurately, and thus improving the accuracy of text classification.

[0044] Optionally, as an embodiment of the present invention, the process of S3 includes:

[0045] The vectorization processing is respectively performed on each of the word training matrices through a preset language model to obtain a word vector matrix corresponding to each of the word training matrices.

[0046] It should be understood that the preset language model can be BERT. BERT is the encoder of the bidirectional Transformer. BERT is a pre-training model proposed by Google AI Research Institute in October 2018. The full name of BERT is Bidirectional Encoder Representation from Transformers. BERT showed amazing results in the top-level machine reading comprehension test SQuAD1.1: surpassing humans comprehensively in all two measurement indicators, and achieving SOTA performance in 11 different NLP tests, including pushing the GLUE benchmark to 80.4% (absolute improvement of 7.6%), and the accuracy of MultiNLI reaching 86.7% (absolute improvement of 5.6%), becoming a milestone model achievement in the history of NLP development. BERT is the first unsupervised, deep bidirectional system used in pre-training natural language tasks, and it uses the attention mechanism to improve the model training speed.

[0047] It should be understood that the preprocessed text (i.e., the word training matrix) is represented in vector form.

[0048] It should be understood that a text (i.e., the word training matrix) is input into the pre-trained language model (i.e., the preset language model), and an n×d-dimensional word vector matrix of the text is output, where n represents the actual length of the text and d represents the word vector dimension.

[0049] In the above embodiment, each word training matrix is ​​vectorized by a preset language model to obtain a word vector matrix, which provides data support for subsequent model training and can more accurately represent text semantics, thereby improving the accuracy of text classification.

[0050] Optionally, as an embodiment of the present invention, the training model includes a convolutional neural network and a bidirectional recurrent neural network, and the process of step S4 includes:

[0051] Performing feature extraction and analysis on each of the word vector matrices through the convolutional neural network to obtain local feature vectors corresponding to each of the word vector matrices;

[0052] Calculating the target hidden layer state vector for each of the word vector matrices through the bidirectional recurrent neural network to obtain a plurality of target hidden layer state vectors corresponding to each of the word vector matrices;

[0053] The context semantic vectors are calculated for the multiple target hidden layer state vectors corresponding to each of the word vector matrices by the first formula to obtain the context semantic vectors corresponding to each of the word vector matrices. The first formula is:

[0054]

[0055] in,

[0056] Among them, u t =tanh(W w h t +b w ),

[0057] Among them, F global is the context semantic vector, a t is the normalized weight of the word at time t, h t is the target hidden layer state vector at time t, u w is the word-level context vector, u t is the word vector at the tth moment, for u t The transpose of W w is a trainable parameter, b w is the bias term;

[0058] Calculating the sentence semantic features of each of the context semantic vectors respectively to obtain the sentence semantic feature vectors corresponding to each of the word vector matrices;

[0059] The local feature vectors and the sentence semantic feature vectors corresponding to the word vector matrices are concatenated by the second formula to obtain the global semantic feature vectors corresponding to the word vector matrices. The second formula is:

[0060]

[0061] Among them, F global is the sentence semantic feature vector, F local is the local feature vector, and F' is the global semantic feature vector;

[0062] Each of the global semantic feature vectors is predicted by the third formula to obtain the category prediction probability corresponding to each of the word vector matrices. The third formula is:

[0063] p=softmax(W g F'+b g ),

[0064] Among them, p is the category prediction probability, W g is the trainable weight, F' is the global semantic feature vector, b g is the bias term;

[0065] Import multiple real labels corresponding to the original text training data, perform loss value analysis on all category prediction probabilities and all real labels, and obtain a model to be tested.

[0066] It should be understood that the introduction of the attention mechanism can well measure the contribution of each word to the text classification task, select more critical information for the current task goal from the entire sentence, and can greatly improve the model effect.

[0067] Specifically, the bidirectional recurrent neural network h t As the sentence semantic feature representation of the text (i.e., the target hidden layer state vector), and as the input of the attention mechanism layer. The introduction of the attention mechanism can well measure the contribution of each word to the text classification task, select the more critical information for the current task goal from the entire sentence, and can well improve the model effect. The attention mechanism assigns a weight to each word representation vector, and finally obtains the sentence semantic feature representation F of the text by weighted summation. global (i.e., the context semantic vector). global The calculation process is as follows: (1)-(3), where (1) is u t =tanh(W w ht +b w ),

[0068] The formula (2) is

[0069] The formula (3) is

[0070] Among them, W w is a trainable parameter, b w is the bias term, is u t The transpose of u w is a randomly initialized, learnable word-level context vector, a t is the normalized weight of the word at time t.

[0071] It should be understood that the sentence semantic feature vector F of the text output by the attention mechanism layer (changed to the Chinese meaning interpretation function) global (i.e., the sentence semantic feature vector) and the keyword semantic feature vector F output by the convolutional neural network layer local (i.e. the local feature vectors) are concatenated to obtain a new global semantic feature vector F', and the calculation process is as follows:

[0072]

[0073] It should be understood that the global semantic feature vector F' obtained after concatenation is applied to the softmax classifier according to the following formula to obtain the category prediction probability p. The following formula is:

[0074] p=softmax(W g F'+b g ),

[0075] Among them, W g is the trainable weight, b g is the bias term.

[0076] Specifically, the obtained word vectors (i.e., the word vector matrix) are respectively input into the convolutional neural network and the bidirectional recurrent neural network to generate key character feature vectors (i.e., the local feature vectors) and sentence feature vectors (i.e., the context semantic vectors) integrating context semantic information, respectively; the sentence feature vectors (i.e., the context semantic vectors) are input into the attention mechanism layer to assign different weights to each word; the key character feature vectors (i.e., the local feature vectors) and the sentence feature vectors (i.e., the sentence semantic feature vectors) are spliced ​​into a global feature vector (i.e., the global semantic feature vector), and the predicted probability (i.e., the category predicted probability) is obtained by using the softmax function for the global feature vector (i.e., the global semantic feature vector) obtained after splicing, and then the cross entropy loss function is used to calculate the loss of the true value (i.e., the true label) and the predicted value (i.e., the category predicted probability), and the loss is minimized to complete the training of the text classification model (i.e., the model to be tested).

[0077] In the above embodiment, a training model is constructed, and the training model is trained according to multiple word vector matrices to obtain a model to be tested, which can highlight the key information in the text, enhance the contextual semantic information of the text, and integrate multi-feature semantic representation, which can more accurately represent the text semantics, thereby improving the accuracy of text classification.

[0078] Optionally, as an embodiment of the present invention,

[0079] The convolutional neural network includes a first convolution kernel, a second convolution kernel, a third convolution kernel and a fourth convolution kernel.

[0080] The process of performing feature extraction and analysis on each of the word vector matrices through the convolutional neural network to obtain a local feature vector corresponding to each of the word vector matrices includes:

[0081] Performing a first feature extraction on each of the word vector matrices using the first convolution kernel to obtain a first feature graph corresponding to each of the word vector matrices;

[0082] Performing a second feature extraction on each of the word vector matrices through the second convolution kernel to obtain a second feature graph corresponding to each of the word vector matrices;

[0083] Performing a third feature extraction on each of the word vector matrices through the third convolution kernel to obtain a third feature map corresponding to each of the word vector matrices;

[0084] Performing a fourth feature extraction on each of the word vector matrices through the fourth convolution kernel to obtain a fourth feature map corresponding to each of the word vector matrices;

[0085] Performing feature screening on each of the first feature maps respectively to obtain screened first feature maps corresponding to each of the word vector matrices;

[0086] Performing feature screening on each of the second feature maps respectively to obtain screened second feature maps corresponding to each of the word vector matrices;

[0087] Performing feature screening on each of the third feature maps respectively to obtain a screened third feature map corresponding to each of the word vector matrices;

[0088] Perform feature screening on each of the fourth feature maps respectively to obtain a screened fourth feature map corresponding to each of the word vector matrices;

[0089] Feature fusion is performed on each of the filtered first feature maps, the filtered second feature maps corresponding to each of the word vector matrices, the filtered third feature maps corresponding to each of the word vector matrices, and the filtered fourth feature maps corresponding to each of the word vector matrices to obtain local feature vectors corresponding to each of the word vector matrices.

[0090] It should be understood that the first convolution kernel may be a one-dimensional convolution kernel with a window size of 1, the second convolution kernel may be a one-dimensional convolution kernel with a window size of 2, the third convolution kernel may be a one-dimensional convolution kernel with a window size of 3, and the fourth convolution kernel may be a one-dimensional convolution kernel with a window size of 4.

[0091] It should be understood that the core idea of ​​the convolutional neural network is to capture local features. The local features of a text are composed of several words. The advantage of the convolutional neural network is that it can automatically combine and filter features to obtain semantic information at different levels of abstraction.

[0092] Specifically, a matrix representation of the text (i.e., the word vector matrix) is obtained from the output of the pre-trained language model (i.e., the preset language model), and is used as the input of the convolutional neural network layer. The word vector matrix of the text is convolved using one-dimensional convolution kernels (i.e., the first convolution kernel, the second convolution kernel, the third convolution kernel, and the fourth convolution kernel) with window sizes of 1, 2, 3, and 4, respectively, to obtain feature maps of different granularity sizes (i.e., the first feature map, the second feature map, the third feature map, and the fourth feature map), and then each feature map (i.e., the first feature map, the second feature map, the third feature map, and the fourth feature map) is subjected to feature screening using a maximum pooling operation, and finally the features screened out of different granularities (i.e., the first feature map after screening, the second feature map after screening, the third feature map after screening, and the fourth feature map after screening) are fused into a one-dimensional vector F by splicing. local(i.e., the local feature vector), and use the vector as the final local feature representation (i.e., the local feature vector). If h is used to represent the window size, the feature map (i.e., the first feature map, the second feature map, the third feature map, and the fourth feature map) can be expressed as:

[0093] C=[c1,c2,...,c (n+1)-h+1 ],

[0094] Each element c in the feature map (i.e., the first feature map, the second feature map, the third feature map, and the fourth feature map) i They are all calculated by the following formula:

[0095] c i =f(W c *W i:i+h-1 +b c ).

[0096] Among them, f represents the activation function, W c ∈R h×d represents the convolution kernel, * represents the element-wise product calculation, b c represents the bias term, h∈{1,2,3,4} represents the window size, i:i+h-1 represents from the i-th row to the i+h-1-th row of the word vector matrix of the text, and the value range of i is [1,n-h+2].

[0097] In the above embodiment, the local feature vectors are obtained by extracting and analyzing the features of each word vector matrix through a convolutional neural network, and the features can be automatically combined and screened to obtain semantic information at different abstract levels.

[0098] Optionally, as an embodiment of the present invention, the process of calculating the target hidden layer state vector for each of the word vector matrices respectively through the bidirectional recurrent neural network to obtain multiple target hidden layer state vectors corresponding to each of the word vector matrices includes:

[0099] Import an initial forward hidden layer state vector and an initial backward hidden layer state vector, and calculate the target hidden layer state vector for the initial forward hidden layer state vector, the initial backward hidden layer state vector, and each of the word vector matrices using the fourth formula to obtain a plurality of target hidden layer state vectors corresponding to each of the word vector matrices. The fourth formula is:

[0100]

[0101] Among them, r t =o t *tanh(c t ), l t=o' t *tanh(c' t ),

[0102] Among them, t =σ(W o ·[r t-1 , e t ]+b o ), c t =f t *c t-1 +i t *tanh(W c ·[r t-1 ,e t ]+b c ), o' t =σ(W o ·[l t-1 ,e t ]+b o ), c' t =f' t *c' t-1 +i' t *tanh(W c ·[l t-1 ,e t ]+b c ),

[0103] Among them, f t =σ(W f ·[r t-1 ,e t ]+b f ),i t =σ(W i ·[r t-1 ,e t ]+b i ), f' t =σ(W f ·[l t-1 ,e t ]+b f ), i' t =σ(W i ·[l t-1 ,e t ]+b i ),

[0104] Among them, h t is the target hidden layer state vector at time t, r t is the forward hidden layer state vector at time t, l t is the backward hidden layer state vector at time t, o t is the forward output gate at time t, c tis the forward state vector at time t, c t-1 is the forward state vector at time t-1, tanh is the hyperbolic tangent function, * is the element-wise product operation, o' t is the backward output gate at time t, c' t is the backward state vector at time t, c' t-1 is the backward state vector at time t-1, σ is the sigmoid activation function, W o , W c , W f and W i are weight matrices, b o 、b c 、b f and b i are all bias terms, e t is the word vector matrix at time t, r t-1 is the forward hidden layer state vector at time t-1, r0 is the initial forward hidden layer state vector, l t-1 is the backward hidden layer state vector at time t-1, l0 is the initial backward hidden layer state vector, f t is the forward forget gate at time t, i t is the forward input gate at time t, f' t The backward forget gate at time t, i' t is the backward input gate at time t, and is the dot product.

[0105] It should be understood that the bidirectional recurrent neural network is good at capturing longer semantic information and can better express contextual semantic information. At the same time, the bidirectional recurrent neural network solves the problem of gradient disappearance that occurs when ordinary neural networks process longer texts.

[0106] It should be understood that the matrix representation of the text (i.e., the word vector matrix) is obtained from the output of the pre-trained language model (i.e., the preset language model) and used as the input of the bidirectional recurrent neural network layer.

[0107] Specifically, the bidirectional recurrent neural network uses the forward and backward recurrent neural networks to obtain the forward hidden layer state vector r at the tth moment t (i.e., the forward hidden layer state vector at time t) and the backward hidden layer state l t (i.e., the backward hidden layer state vector at the tth moment), and r is converted into t and l t Concatenate them together as the final hidden layer state vector h t (i.e., the target hidden layer state vector), h tIt can be used as the context vector representation of the corresponding word at time t (i.e., the target hidden layer state vector). The formula (1) is:

[0108]

[0109] The hidden layer state vector r of the forward recurrent neural network t The calculation process of (i.e., the forward hidden layer state vector at the t-th moment) is as follows: Formula (4) to Formula (8).

[0110] The formula (4) is i t =σ(W i ·[r t-1 ,e t ]+b i ),

[0111] The formula (5) is f t =σ(W f ·[r t-1 ,e t ]+b f ),

[0112] The formula (6) is o t =σ(W o ·[r t-1 ,e t ]+b o ),

[0113] The formula (7) is c t =f t *c t-1 +i t *tanh(W c ·[r t-1 ,e t ]+b c ),

[0114] The formula (8) is t =o t *tanh(c t ).

[0115] Among them, i t is the input gate, f t It's the forget gate, t is the output gate, r t-1 represents the forward hidden layer state vector at time t-1, e t is the output vector of the t-th position of the embedding layer (i.e., the pre-trained language model) (i.e., the word vector matrix at the t-th moment), σ is the sigmoid activation function, w is the weight matrix, b is the bias term, tanh is the hyperbolic tangent function, * is the element-wise product operation, c t is the state vector, ct Together with the output gate, it determines the final output.

[0116] The hidden layer state vector l of the backward recurrent neural network t The calculation process of (i.e., the backward hidden layer state vector at the t-th moment) is as follows: Formula (9) to Formula (13).

[0117] The formula (9) is i' t =σ(W i ·[l t-1 ,e t ]+b i ),

[0118] The formula (10) is f' t =σ(W f ·[l t-1 ,e t ]+b f ),

[0119] The formula (11) is o' t =σ(W o ·[l t-1 ,e t ]+b o ),

[0120] The formula (12) is c' t =f' t *c' t-1 +i' t *tanh(W c ·[l t-1 ,e t ]+b c ),

[0121] The formula (13) is t =o' t *tanh(c' t ).

[0122] Among them, i' t is the input gate, f' t It's the Forgetting Gate, o' t is the output gate, l t-1 represents the backward hidden layer state vector at time t-1, e t is the output vector of the t-th position of the embedding layer (i.e., the pre-trained language model) (i.e., the word vector matrix at the t-th moment), σ is the sigmoid activation function, w is the weight matrix, b is the bias term, tanh is the hyperbolic tangent function, * is the element-wise product operation, c' t is the state vector, c' t Together with the output gate, it determines the final output.

[0123] In the above embodiment, a plurality of target hidden layer state vectors are obtained by respectively calculating the target hidden layer state vector of each word vector matrix through a bidirectional recurrent neural network, which captures longer semantic information, can better express contextual semantic information, and solves the problem of gradient disappearance that occurs in ordinary neural networks when processing longer texts.

[0124] Optionally, as an embodiment of the present invention, the process of performing loss value analysis on all category prediction probabilities and all true labels to obtain the model to be tested includes:

[0125] The loss value is calculated by the fifth formula for all the category prediction probabilities and all the true labels to obtain the loss value. The fifth formula is:

[0126]

[0127] Among them, H(p,q) is the loss value, is the class prediction probability, is the true label, N is the number of original text training data, and C is the number of categories;

[0128] The parameters of the training model are updated according to the loss value, and after the parameters are updated, the process returns to step S1 until a preset number of iterations is reached, and the training model after the parameters are updated is used as the model to be tested.

[0129] It should be understood that the cross entropy loss function is used as the objective function of model training according to the following formula. The following formula is:

[0130]

[0131] Among them, N is the number of training samples, C is the number of categories, and q is the true label of the sample.

[0132] The predicted value (i.e., the category prediction probability) and the corresponding sample true value (i.e., the true label) are compared and calculated to obtain the loss value, and all the learnable parameters of steps S1 to S4 are iteratively updated through the back propagation mechanism and the gradient descent method to minimize the loss value, and finally complete the training of the entire model.

[0133] In the above embodiment, the loss value analysis of all category prediction probabilities and all true labels obtains the model to be tested, which can highlight the key information in the text, while enhancing the text context semantic information, and more accurately represent the text semantics, thereby improving the accuracy of text classification.

[0134] Figure 2 A module block diagram of a text classification device provided by an embodiment of the present invention.

[0135] Optionally, as another embodiment of the present invention, Figure 2 As shown, a text classification device includes:

[0136] A data partitioning module, used for importing a plurality of original text data and partitioning the plurality of the original text data to obtain a training set and a test set;

[0137] A data preprocessing module is used to perform data preprocessing on the training set and the test set respectively to obtain a plurality of word training matrices corresponding to the training set and a plurality of word test matrices corresponding to the test set;

[0138] A vectorization processing module, used to perform vectorization processing on each of the word training matrices to obtain a word vector matrix corresponding to each of the word training matrices;

[0139] A model training module is used to construct a training model, and train the training model according to the plurality of word vector matrices to obtain a model to be tested;

[0140] A model testing module, used for testing the model to be tested according to a plurality of word test matrices to obtain a classification model;

[0141] The classification result obtaining module is used to import the text data to be classified, classify the text data to be classified through the classification model, and obtain the classification result.

[0142] Optionally, another embodiment of the present invention provides a text classification device, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein when the processor executes the computer program, the text classification method described above is implemented. The device may be a computer or other device.

[0143] Optionally, another embodiment of the present invention provides a computer-readable storage medium, wherein the computer-readable storage medium stores a computer program, and when the computer program is executed by a processor, the text classification method as described above is implemented.

[0144] It should be noted that, in this article, relational terms such as first and second, etc. are only used to distinguish one entity or operation from another entity or operation, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Moreover, the terms "include", "comprise" or any other variants thereof are intended to cover non-exclusive inclusion, so that a process, method, article or device including a series of elements includes not only those elements, but also other elements not explicitly listed, or also includes elements inherent to such process, method, article or device.

[0145] Those skilled in the art can clearly understand that, for the convenience and brevity of description, the specific working processes of the above-described devices and units can refer to the corresponding processes in the aforementioned method embodiments and will not be repeated here.

[0146] In the several embodiments provided in this application, it should be understood that the disclosed devices and methods can be implemented in other ways. For example, the device embodiments described above are only schematic, for example, the division of units is only a logical function division, and there may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed.

[0147] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the embodiments of the present invention.

[0148] In addition, each functional unit in each embodiment of the present invention may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.

[0149] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention is essentially or the part that contributes to the prior art, or all or part of the technical solution can be embodied in the form of a software product. The computer software product is stored in a storage medium, including several instructions for a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the methods of each embodiment of the present invention. The aforementioned storage medium includes: U disk, mobile hard disk, read-only memory (ROM, Read-Only Memory), random access memory (RAM, Random Access Memory), disk or optical disk and other media that can store program codes.

[0150] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principle of the present invention should be included in the protection scope of the present invention.

Claims

1. A text classification method, characterized in that: The steps include: S1: importing a plurality of original text data, and dividing the plurality of original text data into a training set and a test set; S2: performing data preprocessing on the training set and the test set respectively to obtain a plurality of word training matrices corresponding to the training set and a plurality of word test matrices corresponding to the test set; S3: performing vectorization processing on each of the word training matrices respectively to obtain a word vector matrix corresponding to each of the word training matrices; S4: constructing a training model, and training the training model according to the plurality of word vector matrices to obtain a model to be tested; S5: Testing the model to be tested according to the multiple word test matrices to obtain a classification model; S6: importing the text data to be classified, and classifying the text data to be classified by the classification model to obtain a classification result; The training model includes a convolutional neural network and a bidirectional recurrent neural network, and the process of step S4 includes: Performing feature extraction and analysis on each of the word vector matrices through the convolutional neural network to obtain local feature vectors corresponding to each of the word vector matrices; Calculating the target hidden layer state vector for each of the word vector matrices through the bidirectional recurrent neural network to obtain a plurality of target hidden layer state vectors corresponding to each of the word vector matrices; The context semantic vectors are calculated for the multiple target hidden layer state vectors corresponding to each of the word vector matrices by the first formula to obtain the context semantic vectors corresponding to each of the word vector matrices. The first formula is: in, Among them, u t =tanh(W w h t +b w ), Among them, F global is the context semantic vector, a t is the normalized weight of the word at time t, h t is the target hidden layer state vector at time t, u w is the word-level context vector, u t is the word vector at the tth moment, for u t The transpose of W w is a trainable parameter, b w is the bias term; Calculating the sentence semantic features of each of the context semantic vectors respectively to obtain the sentence semantic feature vectors corresponding to each of the word vector matrices; The local feature vectors and the sentence semantic feature vectors corresponding to the word vector matrices are concatenated by the second formula to obtain the global semantic feature vectors corresponding to the word vector matrices. The second formula is: Among them, F global is the sentence semantic feature vector, F local is the local feature vector, and F' is the global semantic feature vector; Each of the global semantic feature vectors is predicted by the third formula to obtain the category prediction probability corresponding to each of the word vector matrices. The third formula is: p=softmax(W g F'+b g ), Among them, p is the category prediction probability, W g is the trainable weight, F' is the global semantic feature vector, b g is the bias term; Import multiple real labels corresponding to the original text training data, perform loss value analysis on all category prediction probabilities and all real labels, and obtain a model to be tested.

2. The text classification method according to claim 1, characterized in that: The training set includes a plurality of original text training data, the test set includes a plurality of original text test data, and the process of step S2 includes: Use Python tools to perform word segmentation processing on each of the original text training data and each of the original text test data, to obtain a plurality of to-be-screened word training data corresponding to each of the original text training data and a plurality of to-be-screened word test data corresponding to each of the original text test data; The python tool is used to screen stop words for each of the word training data to be screened and each of the word test data to be screened, to obtain a plurality of word training data corresponding to each of the original text training data and a plurality of word test data corresponding to each of the original text test data, and a word training matrix corresponding to each of the original text training data is obtained according to the plurality of word training data, and a word test matrix corresponding to each of the original text test data is obtained according to the plurality of word test data.

3. The text classification method according to claim 1, characterized in that: The S3 process includes: Each of the word training matrices is vectorized by a preset language model to obtain a word vector matrix corresponding to each of the word training matrices.

4. The text classification method according to claim 1, characterized in that: The convolutional neural network includes a first convolution kernel, a second convolution kernel, a third convolution kernel and a fourth convolution kernel. The process of performing feature extraction and analysis on each of the word vector matrices through the convolutional neural network to obtain a local feature vector corresponding to each of the word vector matrices includes: Performing a first feature extraction on each of the word vector matrices using the first convolution kernel to obtain a first feature graph corresponding to each of the word vector matrices; Performing a second feature extraction on each of the word vector matrices through the second convolution kernel to obtain a second feature map corresponding to each of the word vector matrices; Performing a third feature extraction on each of the word vector matrices through the third convolution kernel to obtain a third feature map corresponding to each of the word vector matrices; Performing a fourth feature extraction on each of the word vector matrices through the fourth convolution kernel to obtain a fourth feature map corresponding to each of the word vector matrices; Performing feature screening on each of the first feature maps respectively to obtain screened first feature maps corresponding to each of the word vector matrices; Performing feature screening on each of the second feature maps respectively to obtain screened second feature maps corresponding to each of the word vector matrices; Performing feature screening on each of the third feature maps respectively to obtain a screened third feature map corresponding to each of the word vector matrices; Perform feature screening on each of the fourth feature maps respectively to obtain a screened fourth feature map corresponding to each of the word vector matrices; Feature fusion is performed on each of the filtered first feature maps, the filtered second feature maps corresponding to each of the word vector matrices, the filtered third feature maps corresponding to each of the word vector matrices, and the filtered fourth feature maps corresponding to each of the word vector matrices to obtain local feature vectors corresponding to each of the word vector matrices.

5. The text classification method according to claim 1, characterized in that: The process of calculating the target hidden layer state vector for each of the word vector matrices respectively through the bidirectional recurrent neural network to obtain a plurality of target hidden layer state vectors corresponding to each of the word vector matrices includes: Import an initial forward hidden layer state vector and an initial backward hidden layer state vector, and calculate the target hidden layer state vector for the initial forward hidden layer state vector, the initial backward hidden layer state vector, and each of the word vector matrices using the fourth formula to obtain a plurality of target hidden layer state vectors corresponding to each of the word vector matrices. The fourth formula is: Among them, r t =o t *tanh(c t ), l t =o' t *tanh(c' t ), where, o t = σ(W o ·[r t-1 , e t + b o ), c t = f t * c t-1 + i t * tanh(W c ·[r t-1 , e t + b c ), o' t = σ(W o ·[l t-1 , e t + b o ), c' t = f' t * c' t-1 + i' t * tanh(W c ·[l t-1 , e t + b c ) among them, f t =σ(W f ·[r t-1 ,e t ]+b f ),i t =σ(W i ·[r t-1 ,e t ]+b i ),f' t =σ(W f ·[l t-1 ,e t ]+b f ),i' t =σ(W i ·[l t-1 ,e t ]+b i ), Among them, h t is the target hidden layer state vector at time t, r t is the forward hidden layer state vector at time t, l t is the backward hidden layer state vector at time t, o t is the forward output gate at time t, c t is the forward state vector at time t, c t-1 is the forward state vector at time t-1, tanh is the hyperbolic tangent function, * is the element-wise product operation, o' t is the backward output gate at time t, c' t is the backward state vector at time t, c' t-1 is the backward state vector at time t-1, σ is the sigmoid activation function, W o , W c , W f and W i are weight matrices, b o , b c , b f and b i are all bias terms, e t is the word vector matrix at time t, r t-1 is the forward hidden layer state vector at time t-1, r0 is the initial forward hidden layer state vector, l t-1 is the backward hidden layer state vector at time t-1, l0 is the initial backward hidden layer state vector, f t is the forward forget gate at time t, i t is the forward input gate at time t, f' t The backward forget gate at time t, i' t is the backward input gate at time t, and is the dot product.

6. The text classification method according to claim 1, characterized in that: The process of performing loss value analysis on all category prediction probabilities and all true labels to obtain the model to be tested includes: The loss value is calculated by the fifth formula for all the category prediction probabilities and all the true labels to obtain the loss value. The fifth formula is: Among them, H(p,q) is the loss value, is the class prediction probability, is the true label, N is the number of original text training data, and C is the number of categories; The parameters of the training model are updated according to the loss value, and after the parameters are updated, the process returns to step S1 until a preset number of iterations is reached, and the training model after the parameters are updated is used as the model to be tested.

7. A text classification device, characterized in that: include: A data partitioning module, used for importing a plurality of original text data and partitioning the plurality of the original text data to obtain a training set and a test set; A data preprocessing module is used to perform data preprocessing on the training set and the test set respectively to obtain a plurality of word training matrices corresponding to the training set and a plurality of word test matrices corresponding to the test set; A vectorization processing module, used to perform vectorization processing on each of the word training matrices to obtain a word vector matrix corresponding to each of the word training matrices; A model training module is used to construct a training model, and train the training model according to the plurality of word vector matrices to obtain a model to be tested; A model testing module, used for testing the model to be tested according to a plurality of word test matrices to obtain a classification model; A classification result obtaining module is used to import the text data to be classified, classify the text data to be classified by the classification model, and obtain the classification result; The training model includes a convolutional neural network and a bidirectional recurrent neural network, and the model training module is specifically used for: Performing feature extraction and analysis on each of the word vector matrices through the convolutional neural network to obtain local feature vectors corresponding to each of the word vector matrices; Calculating the target hidden layer state vector for each of the word vector matrices through the bidirectional recurrent neural network to obtain a plurality of target hidden layer state vectors corresponding to each of the word vector matrices; The context semantic vectors are calculated for the multiple target hidden layer state vectors corresponding to each of the word vector matrices by the first formula to obtain the context semantic vectors corresponding to each of the word vector matrices. The first formula is: in, Among them, u t =tanh(W w h t +b w ), Among them, F global is the context semantic vector, a t is the normalized weight of the word at time t, h t is the target hidden layer state vector at time t, u w is the word-level context vector, u t is the word vector at the tth moment, for u t The transpose of W w is a trainable parameter, b w is the bias term; Calculating the sentence semantic features of each of the context semantic vectors respectively to obtain the sentence semantic feature vectors corresponding to each of the word vector matrices; The local feature vectors and the sentence semantic feature vectors corresponding to the word vector matrices are concatenated by the second formula to obtain the global semantic feature vectors corresponding to the word vector matrices. The second formula is: Among them, F global is the sentence semantic feature vector, F local is the local feature vector, and F' is the global semantic feature vector; Each of the global semantic feature vectors is predicted by the third formula to obtain the category prediction probability corresponding to each of the word vector matrices. The third formula is: p=softmax(W g F'+b g ), Among them, p is the category prediction probability, W g is the trainable weight, F' is the global semantic feature vector, b g is the bias term; Import multiple real labels corresponding to the original text training data, perform loss value analysis on all category prediction probabilities and all real labels, and obtain a model to be tested.

8. A text classification system, comprising a memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that: When the processor executes the computer program, the text classification method according to any one of claims 1 to 6 is implemented.

9. A computer-readable storage medium storing a computer program, characterized in that: When the computer program is executed by a processor, the text classification method according to any one of claims 1 to 6 is implemented.

Images