Text classification method and device, electronic equipment, vehicle and storage medium
By introducing dynamic convolution layer into text classification method, the convolution kernel weights are adaptively adjusted to extract different features, the problem of insufficient representation ability caused by the convolution neural network using the same convolution kernel for all text word sequences is solved, and the text is improved Classification accuracy.
Patent Information
- Application Number
- CN202310602506.9
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2023-05-25
- Publication Date
- 2024-11-26
AI Technical Summary
The existing text classification method based on convolutional neural networks uses the same convolutional kernel to pair all input text word sequences during feature extraction, resulting in limited representation ability and affecting the accuracy of classification results.
The dynamic convolution layer is adopted to adaptively adjust the weights of multiple preset convolution kernels according to the text word sequence, and different features are extracted through different convolution kernels to generate a more comprehensive and accurate text feature matrix.
The accuracy of text classification results is improved, and the text feature matrix output by the dynamic convolution layer expresses the characteristics of text word sequence more comprehensively, which enhances the feature extraction ability of different texts.
Smart Images

Figure CN119025671A_ABST
Abstract
Description
Technical Field
[0001] The embodiments of the present application relate to the field of natural language processing technology, and in particular, to a text classification method, device, electronic device, vehicle and storage medium. Background Art
[0002] With the continuous development of deep learning, many text classification methods based on deep learning have emerged. Among them, the text classification method based on Convolutional Neural Networks (CNN) has been widely used in text classification because it can effectively extract local information features in text.
[0003] The key to text classification is to accurately extract the central idea of a document or sentence, and the way to extract the central idea is to extract the keywords of the text as features. In CNN, the features of the input text word sequence are extracted through the convolution layer, and different convolution kernel parameters will extract different features.
[0004] However, in CNN, the convolution kernel parameters used for all input text word sequences are the same, and feature extraction is not targeted, resulting in limited representation capabilities, which in turn affects the accuracy of the final text classification results. Summary of the invention
[0005] The embodiments of the present application provide a text classification method, device, electronic device, vehicle and storage medium, which can extract multiple features in a text word sequence, so that the text feature matrix obtained by convolution processing can represent the text word sequence more comprehensively and accurately.
[0006] In a first aspect, an embodiment of the present application provides a text classification method, comprising: obtaining a target sequence, the target sequence being used to indicate a text to be classified; inputting the target sequence into a trained text classification model for classification to obtain a classification result, the trained text classification model comprising a dynamic convolution layer, the dynamic convolution layer being used to: determine the weight of each of a plurality of preset convolution kernels based on a text word sequence, and based on the plurality of preset convolution kernels and the weight of each preset convolution kernel, convolve the text word sequence to obtain a text feature matrix; the text word sequence is obtained based on the target sequence, the text feature matrix is used to determine the classification result, and the plurality of preset convolution kernels have the same size and different convolution kernel parameters.
[0007] In some embodiments of the present application, before inputting the target sequence into a trained text classification model for classification, the method also includes: performing word segmentation processing on the target sequence to obtain at least one ordered word; filtering out words in a stop word list from at least one ordered word to obtain a target word sequence, wherein the stop word list includes multiple stop words, and each stop word does not affect the text classification result; inputting the target sequence into a trained text classification model for classification to obtain a classification result, including: inputting the target word sequence into a trained text classification model for classification to obtain a classification result.
[0008] In some embodiments of the present application, after filtering out words in at least one ordered word that are in a stop word list to obtain a target word sequence, the method further includes: inputting the target word sequence into a pre-trained word vector model to obtain a target word sequence vector; inputting the target sequence into a trained text classification model for classification to obtain a classification result, including: inputting the target word sequence vector into a trained text classification model for classification to obtain a classification result.
[0009] In some embodiments of the present application, before inputting the target word sequence into a pre-trained word vector model to obtain the target word sequence vector, the method also includes: training a global vector GloVe model based on a corpus to obtain a trained GloVe model; inputting the target word sequence into a pre-trained word vector model to obtain a target word sequence vector, including: inputting the target word sequence into the trained GloVe model to obtain a target word sequence vector.
[0010] In some embodiments of the present application, the trained text classification model also includes: a recurrent neural network layer, a pooling layer, a fully connected layer, and a classification layer; the target word sequence vector is input into the trained text classification model for classification to obtain a classification result, including: inputting the target word sequence vector into the recurrent neural network layer, outputting the text word sequence corresponding to the target word sequence vector, and the text word sequence integrates the context information of the target word sequence vector; inputting the text word sequence into the dynamic convolution layer for feature extraction and feature mapping, and outputting a text feature matrix; inputting the text feature matrix into the pooling layer for dimensionality reduction and compression, and outputting a target feature matrix; inputting the target feature matrix into the fully connected layer for feature fusion, and outputting a target vector; inputting the target vector into the classification layer for classification to obtain a classification result.
[0011] In some embodiments of the present application, the recurrent neural network layer includes any of the following: a long short-term memory layer, a bidirectional long short-term memory layer.
[0012] In some embodiments of the present application, the target sequence is input into a trained text classification model for classification, and before obtaining the classification result, the method also includes: constructing a text classification model; constructing a training set, the training set including multiple sample texts and actual classification results corresponding to each sample text; preprocessing each sample text in the multiple sample texts to obtain a training word sequence corresponding to each sample text, so as to obtain a training word sequence set; training the text classification model through the training word sequence set to obtain a trained text classification model; wherein, multiple sample texts are classified through the trained text classification model, and among the multiple predicted classification results obtained, the proportion of predicted classification results that are the same as the corresponding actual classification results is greater than or equal to the accuracy threshold.
[0013] In a second aspect, an embodiment of the present application provides a text classification device, comprising: an acquisition module, used to acquire a target sequence, the target sequence is used to indicate a text to be classified; an input module, used to input the target sequence into a trained text classification model for classification to obtain a classification result, the trained text classification model includes a dynamic convolution layer, the dynamic convolution layer is used to: determine the weight of each preset convolution kernel in a plurality of preset convolution kernels based on a text word sequence, and based on the plurality of preset convolution kernels and the weight of each preset convolution kernel, perform convolution processing on the text word sequence to obtain a text feature matrix; the text word sequence is obtained based on the target sequence, the text feature matrix is used to determine the classification result, and the plurality of preset convolution kernels have the same size and different convolution kernel parameters.
[0014] In some embodiments of the present application, the text classification device also includes: a word segmentation processing module and a filtering module; the word segmentation processing module is used to perform word segmentation processing on the target sequence before inputting the target sequence into a trained text classification model for classification to obtain at least one ordered word; the filtering module is used to filter out words in the at least one ordered word that belong to a stop word list to obtain a target word sequence, the stop word list includes multiple stop words, and each stop word does not affect the text classification result; the input module is specifically used to input the target word sequence into the trained text classification model for classification to obtain a classification result.
[0015] In some embodiments of the present application, the input module is also used to filter out at least one word in the ordered word list that belongs to the stop word list to obtain a target word sequence, and then input the target word sequence into a pre-trained word vector model to obtain a target word sequence vector; the input module is specifically used to input the target word sequence vector into a trained text classification model for classification to obtain a classification result.
[0016] In some embodiments of the present application, the text classification device also includes: a training module; the training module is used to train the global vector GloVe model based on the corpus to obtain a trained GloVe model before inputting the target word sequence into a pre-trained word vector model to obtain the target word sequence vector; the input module is specifically used to input the target word sequence into the trained GloVe model to obtain the target word sequence vector.
[0017] In some embodiments of the present application, the trained text classification model also includes: a recurrent neural network layer, a pooling layer, a fully connected layer, and a classification layer; the input module is specifically used to input the target word sequence vector into the recurrent neural network layer, and output the text word sequence corresponding to the target word sequence vector, and the text word sequence integrates the context information of the target word sequence vector; input the text word sequence into the dynamic convolution layer for feature extraction and feature mapping, and output the text feature matrix; input the text feature matrix into the pooling layer for dimensionality reduction and compression, and output the target feature matrix; input the target feature matrix into the fully connected layer for feature fusion, and output the target vector; input the target vector into the classification layer for classification to obtain the classification result.
[0018] In some embodiments of the present application, the recurrent neural network layer includes any of the following: a long short-term memory layer, a bidirectional long short-term memory layer.
[0019] In some embodiments of the present application, the text classification device also includes: a construction module and a preprocessing module; the construction module is used to construct a text classification model before inputting the target sequence into the trained text classification model for classification to obtain the classification result; the construction module is also used to construct a training set, the training set includes multiple sample texts and actual classification results corresponding to each sample text; the preprocessing module is used to preprocess each sample text in the multiple sample texts to obtain a training word sequence corresponding to each sample text to obtain a training word sequence set; the training module is also used to train the text classification model through the training word sequence set to obtain a trained text classification model; wherein, when multiple sample texts are classified through the trained text classification model, among the multiple predicted classification results obtained, the proportion of predicted classification results that are the same as the corresponding actual classification results is greater than or equal to the accuracy threshold.
[0020] In a third aspect, an embodiment of the present application provides an electronic device, comprising: a processor, wherein the processor is used to execute a computer program stored in a memory, wherein the computer program implements the steps of any one of the methods provided in the first aspect when executed by the processor.
[0021] In a fourth aspect, an embodiment of the present application provides a vehicle, comprising: the text classification device of the second aspect or the electronic device of the third aspect.
[0022] In a fifth aspect, an embodiment of the present application provides a computer-readable storage medium having a computer program stored thereon, and when the computer program is executed by a processor, the steps of any one of the methods provided in the first aspect are implemented.
[0023] In the technical solution provided by the embodiment of the present application, a target sequence is obtained; the target sequence is input into a trained text classification model for classification to obtain a classification result, and the trained text classification model includes a dynamic convolution layer, and the dynamic convolution layer is used to: determine the weight of each preset convolution kernel in multiple preset convolution kernels based on the text word sequence, and convolve the text word sequence based on multiple preset convolution kernels and the weight of each preset convolution kernel to obtain a text feature matrix; the text word sequence is obtained based on the target sequence, and the text feature matrix is used to determine the classification result, and the multiple preset convolution kernels have the same size and different convolution kernel parameters. In this solution, based on the output of the upper layer (text word sequence), the weight of each convolution kernel in multiple preset convolution kernels is determined, and different preset convolution kernels can extract different features in the input, and the weight of each convolution kernel further defines the importance of each different feature, that is, for different text word sequences, dynamic convolution can extract more features, and different inputs, the extracted multiple features contribute differently to the output of the dynamic convolution layer. Therefore, the text feature matrix output by the dynamic convolution layer can more comprehensively and accurately express the features of the text word sequence, thereby improving the accuracy of the text classification result. BRIEF DESCRIPTION OF THE DRAWINGS
[0024] The accompanying drawings, which are incorporated in and constitute a part of this specification, illustrate embodiments consistent with the present application and, together with the description, serve to explain the principles of the present application.
[0025] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, for ordinary technicians in this field, other drawings can be obtained based on these drawings without paying any creative labor.
[0026] Figure 1 A flowchart of a text classification method provided in an embodiment of the present application;
[0027] Figure 2 A schematic diagram of a network structure of a dynamic convolutional layer provided in an embodiment of the present application;
[0028] Figure 3 A schematic diagram of a network structure of another dynamic convolutional layer provided in an embodiment of the present application;
[0029] Figure 4 A flowchart of another text classification method provided in an embodiment of the present application;
[0030] Figure 5 A flowchart of another text classification method provided in an embodiment of the present application;
[0031] Figure 6 A flowchart of another text classification method provided in an embodiment of the present application;
[0032] Figure 7 A flowchart of another text classification method provided in an embodiment of the present application;
[0033] Figure 8 A flowchart of another text classification method provided in an embodiment of the present application;
[0034] Fig. 9 A schematic diagram of the structure of a text classification device provided in an embodiment of the present application;
[0035] Fig.10 A schematic diagram of the hardware structure of an electronic device is provided for an embodiment of the present application. DETAILED DESCRIPTION
[0036] In order to more clearly understand the above-mentioned purposes, features and advantages of the present application, the scheme of the present application will be further described below. It should be noted that the embodiments of the present application and the features in the embodiments can be combined with each other without conflict.
[0037] In the following description, many specific details are set forth to facilitate a full understanding of the present application, but the present application may also be implemented in other ways different from those described herein; obviously, the embodiments in the specification are only part of the embodiments of the present application, rather than all of the embodiments.
[0038] First, some nouns or terms involved in the claims and description of the present invention are explained below.
[0039] Static convolution refers to the same convolution kernel, which will be applied to all data sets. Dynamic convolution introduces variable-length convolution kernels to adapt to the length changes of the input text word sequence. In traditional convolutional neural networks, whether using narrow convolution or wide convolution, the size of the convolution kernel is fixed, so the input sequence must be padded or truncated to a fixed length, which may cause information loss or increase useless calculations. In order for CNN to fully express the input text word sequence, it is necessary to increase the depth (number of convolution layers) and width (number of channels) of CNN, but increasing the depth and width of CNN will make the model more complex, resulting in a significant increase in the difficulty of model training.
[0040] The dynamic convolution layer is used to use different convolution kernels for different inputs, and then perform attention weighting on these different convolution kernels to obtain the final convolution kernel, and use the convolution kernel to perform convolution operations on the input.
[0041] Figure 1 A flowchart of a text classification method provided in an embodiment of the present application is shown in FIG. Figure 1 As shown, the text classification method provided in the embodiment of the present application includes the following steps 101 and 102.
[0042] 101. Obtain the target sequence.
[0043] The target sequence is used to indicate the text to be classified. The target sequence may be the text to be classified, or may be obtained by processing the text to be classified.
[0044] 102. Input the target sequence into the trained text classification model for classification to obtain the classification result.
[0045] Among them, the trained text classification model includes a dynamic convolution layer, which is used to: determine the weight of each convolution kernel in multiple preset convolution kernels based on the text word sequence, and based on multiple preset convolution kernels and the weight of each convolution kernel, perform convolution processing on the text word sequence to obtain a text feature matrix, the text word sequence is obtained based on the target sequence, and the text feature matrix is used to determine the classification result.
[0046] The text word sequence is obtained based on the target sequence. Specifically, the target sequence is processed by other layers before the dynamic convolution layer of the trained text classification model, and the output of the previous layer of the dynamic convolution layer is the text word sequence.
[0047] Exemplarily, the dynamic convolution layer is used to adaptively adjust the convolution kernel parameters based on the size of the text word sequence. Specifically, multiple convolution kernels with different convolution kernel parameters can be pre-set. The dynamic convolution layer determines the weight of each convolution kernel according to the text word sequence. Each convolution kernel is convolved with the target sequence to obtain multiple initial text feature matrices. Then, based on the weight of each convolution kernel, the multiple initial text feature matrices are weighted and summed to obtain the final text feature matrix. Figure 2 , which is a schematic diagram of a dynamic convolution layer in a trained text classification model, is shown in the figure with three convolution kernels, including convolution kernel 1, convolution kernel 2 and convolution kernel 3. Specifically, the output of the dynamic convolution layer is obtained by the following formula (1).
[0048] Output(x)=σ(α1·(W1*x)+α2·(W2*x)+...+α n ·(W n *x)) (1)
[0049] Among them, x is the output of the previous layer, σ is the activation function, α1, α2, ... α n is the routing function, which can be learned by gradient descent method, W1, W2, ... W n are n different convolution kernels, and Output(x) is the output of the dynamic convolution layer.
[0050] Exemplarily, the dynamic convolution layer is used to adaptively adjust the convolution kernel parameters based on the size of the text word sequence. Specifically, multiple convolution kernels with different convolution kernel parameters can be pre-set. The dynamic convolution layer determines the weight of each convolution kernel according to the text word sequence, and then obtains the target convolution kernel by weighted summation of the multiple convolution kernels according to the weights. The target convolution kernel is used to perform convolution processing on the text word sequence to obtain a text feature matrix. Figure 3 , which is a schematic diagram of another dynamic convolution layer in a trained text classification model, is shown in the figure with three convolution kernels, including convolution kernel 1, convolution kernel 2 and convolution kernel 3. Specifically, the output of the dynamic convolution layer is obtained by the following formula (2).
[0051] Output(x)=σ((α1·W1+α2·W2+...+α n ·W n )*x) (2)
[0052] Among them, x is the output of the previous layer, σ is the activation function, α1, α2, ... α n The weight coefficients W1, W2, ... W are obtained by the routing function according to the input and can be learned by the gradient descent method. n are n different convolution kernels, and Output(x) is the output of the dynamic convolution layer.
[0053] Specifically, the routing function is expressed as the following formula (3).
[0054] r(x)=Sigmoid(GAP(x)R) (3)
[0055] Among them, α i =r i (x), for input x, first perform average pooling (GlobalAveragePooling, GAP), then multiply by a matrix R (the purpose of matrix R is to map the dimension to n W to achieve subsequent linear combination), and finally use sigmoid to reduce the weight on each dimension to the interval [0,1]. Therefore, different input x will get different weight vectors.
[0056] It should be noted that formula (1) and formula (2) are two different ways of implementing dynamic convolution. The two expressions are equivalent, but formula (2) is more efficient in actual inference because it only requires one convolution for a single input, which can greatly reduce the amount of calculation compared to formula (1). Therefore, in practical applications, formula (2) is usually used. Formula (1) and formula (2) are only exemplary descriptions and are not intended to limit the present application. Other solutions that use similar strategies to the dynamic convolution layer of the embodiment of the present application are also within the scope of protection of the present application.
[0057] In the embodiment of the present application, a target sequence is obtained; the target sequence is input into a trained text classification model for classification to obtain a classification result, and the trained text classification model includes a dynamic convolution layer, and the dynamic convolution layer is used to: determine the weight of each preset convolution kernel in multiple preset convolution kernels based on the text word sequence, and based on multiple preset convolution kernels and the weight of each preset convolution kernel, convolution processing is performed on the text word sequence to obtain a text feature matrix; the text word sequence is obtained based on the target sequence, and the text feature matrix is used to determine the classification result, and the multiple preset convolution kernels have the same size and different convolution kernel parameters. In this scheme, based on the output of the upper layer (text word sequence), the weight of each convolution kernel in multiple preset convolution kernels is determined, and different preset convolution kernels can extract different features in the input, and the weight of each convolution kernel further defines the importance of each different feature, that is, for different text word sequences, dynamic convolution can extract more features, and different inputs, the extracted multiple features contribute differently to the output of the dynamic convolution layer. Therefore, the text feature matrix output by the dynamic convolution layer can more comprehensively and accurately express the features of the text word sequence, thereby improving the accuracy of the text classification result.
[0058] In some embodiments of the present application, Figure 1 ,like Figure 4 As shown, before the above step 102, the text classification method provided in the embodiment of the present application further includes the following steps 103 and 104, and the above step 102 can be specifically implemented by the following step 102a.
[0059] 103. Perform word segmentation on the target sequence to obtain at least one ordered word.
[0060] 104. Filter out at least one word in the ordered words that is in the stop word list to obtain a target word sequence.
[0061] The stop word list includes multiple stop words, and each stop word does not affect the text classification result.
[0062] 102a. Input the target word sequence into the trained text classification model for classification to obtain a classification result.
[0063] It can be understood that the target sequence is segmented to obtain at least one ordered word. Specifically, the segmentation process can be implemented by a segmentation tool. The specific segmentation tool used is not limited in this application, such as: Jieba segmentation, SnowNLP, a tool library written in Python, LTP, HanNLP, etc. The target sequence is divided into individual words by the segmentation tool, and the target sequence after the segmentation process is arranged in order to obtain at least one ordered word.
[0064] It can be understood that based on the stop word list, each word in at least one ordered word is traversed to determine whether each word is a word in the stop word list, and the words in the stop word list are removed to finally obtain the target word sequence.
[0065] It can be understood that the target word sequence is obtained by filtering out the words in the stop word list from at least one ordered word. Specifically, the stop word list includes words without actual meaning, such as: modal particles, adverbs, prepositions, conjunctions, etc., and words that appear in large quantities in text classification, such as: personal pronouns, etc. The stop word list may also include: punctuation marks, etc. according to the text classified according to actual needs. Removing the stop words included in the target sequence is conducive to improving the density of keywords, thereby facilitating the determination of the classification to which the target sequence belongs.
[0066] In an embodiment of the present application, before inputting the target sequence into a trained text classification model for classification, the method further includes: performing word segmentation processing on the target sequence to obtain at least one ordered word; filtering out words in a stop word list from at least one ordered word to obtain a target word sequence, wherein the stop word list includes multiple stop words, and each stop word does not affect the text classification result; inputting the target sequence into a trained text classification model for classification to obtain a classification result, including: inputting the target word sequence into a trained text classification model for classification to obtain a classification result. In this way, filtering out words in the stop word list from the target sequence can reduce the interference of stop words on the classification result, improve the density of keywords, and thus make the classification result of the target sequence more accurate.
[0067] In some embodiments of the present application, Figure 4 ,like Figure 5 As shown, after the above step 104, the text classification method provided in the embodiment of the present application further includes the following step 105, and the above step 102 can be specifically implemented by the following step 102b.
[0068] 105. Input the target word sequence into the pre-trained word vector model to obtain the target word sequence vector.
[0069] 102b. Input the target word sequence vector into the trained text classification model for classification to obtain the classification result.
[0070] Optionally, the trained text classification model includes an embedding layer, and the target word sequence is input into the embedding layer to obtain a target word sequence vector. That is, the target word sequence is input into the trained text classification model for classification to obtain a classification result. The target word sequence is represented by the embedding layer to obtain a target word sequence vector.
[0071] It can be understood that the target word sequence is input into the pre-trained word vector model to obtain the target word sequence vector. Specifically, the word vector model can be any existing word vector model, such as the classic word vector models: word2vec, glove, ELMo, BERT, etc.
[0072] In an embodiment of the present application, after filtering out at least one word in the ordered word list that is in the stop word list and obtaining the target word sequence, the method further includes: inputting the target word sequence into a pre-trained word vector model to obtain a target word sequence vector; inputting the target sequence into a trained text classification model for classification to obtain a classification result, including: inputting the target word sequence vector into a trained text classification model for classification to obtain a classification result. By characterizing the target word sequence through various classic word vector models, the target word sequence vector can contain richer information, thereby making the classification result obtained based on the target word sequence vector more accurate.
[0073] In some embodiments of the present application, Figure 5 ,like Figure 6 As shown, before the above step 105, the text classification method provided in the embodiment of the present application further includes the following step 106. The above step 105 can be specifically implemented by the following step 105a.
[0074] 106. Based on the corpus, train the global vector GloVe model to obtain a trained GloVe model.
[0075] 105a. Input the target word sequence into the trained GloVe model to obtain the target word sequence vector.
[0076] It can be understood that a corpus refers to a database that stores language materials. The corpus stores language materials that have actually appeared in the actual use of the language. The corpus can be custom created based on the scene in which the text actually needs to be classified; it can also be an existing relatively mature corpus, such as: Wikipedia Thesaurus, National Language Commission Modern Chinese Corpus, People's Daily Annotated Corpus, etc.
[0077] It can be understood that the Global Vectors for Word Representation (GloVe) model is a word representation tool based on global word frequency statistics, which can express a word as a vector composed of real numbers. These vectors capture some semantic characteristics between words, such as similarity and analogy. We can calculate the semantic similarity between two words by performing operations on vectors, such as Euclidean distance or cosine similarity.
[0078] Specifically, the GloVe model training process includes: constructing a vocabulary, mapping the words included in the corpus to unique integer codes, and filtering out words below the word frequency threshold to reduce the amount of training; constructing a co-occurrence matrix: for each word, counting its co-occurrence frequency with surrounding words within a certain window size (such as 10) to obtain a co-occurrence matrix; calculating the loss function: using the co-occurrence matrix to calculate the loss function, the loss function measures the similarity and difference between word vectors. The specific form of the loss function is a weighted square error loss function, in which the weight is calculated by a function to weaken the influence of high-frequency words; minimizing the loss function through the stochastic gradient descent method (SGD) to obtain the word vector representation of each word; final word vector: for each word, add its context vector and word vector to obtain the final word vector representation. The context vector refers to the weighted average of the word vectors of all words that co-occur with the word.
[0079] In an embodiment of the present application, before inputting the target word sequence into a pre-trained word vector model to obtain the target word sequence vector, the method further includes: training a global vector GloVe model based on a corpus to obtain a trained GloVe model; inputting the target word sequence into a pre-trained word vector model to obtain a target word sequence vector, including: inputting the target word sequence into a trained GloVe model to obtain a target word sequence vector. Due to the information of word co-occurrence in the GloVe model, global information is utilized through the co-occurrence matrix. At the same time, the GloVe model is faster to train than other word vector modules, can be expanded to large-scale corpora, is also suitable for small-scale corpora, and the final effect is usually better.
[0080] In some embodiments of the present application, Figure 4 ,like Figure 7 As shown, the trained text classification model also includes: a recurrent neural network layer, a pooling layer, a fully connected layer, and a classification layer; the above step 102 can be specifically implemented through the following steps 102c to 102g.
[0081] 102c. Input the target word sequence vector into the recurrent neural network layer, and output the text word sequence corresponding to the target word sequence vector.
[0082] Among them, the text word sequence integrates the context information of the target word sequence vector.
[0083] 102d. Input the text word sequence into the dynamic convolution layer for feature extraction and feature mapping, and output the text feature matrix.
[0084] 102e. Input the text feature matrix into the pooling layer for dimensionality reduction compression, and output the target feature matrix.
[0085] 102f. Input the target feature matrix into the fully connected layer for feature fusion and output the target vector.
[0086] 102g. Input the target vector into the classification layer for classification to obtain the classification result.
[0087] Optionally, the recurrent neural network layer includes any of the following: a long short-term memory layer, a bidirectional long short-term memory layer.
[0088] It can be understood that the recurrent neural network layer is very effective for data with sequence characteristics, and can mine the temporal information and semantic information in the data. Therefore, the text word sequence obtained by the recurrent neural network layer incorporates the context information of the target word sequence vector. However, since the hidden state of the basic recurrent neural network at each moment is not only determined by the input at that moment, but also depends on the value of the hidden layer at the previous moment, if a text is very long, at the end of the text, the recurrent neural network will not remember the detailed content of the beginning of the text, resulting in gradient disappearance or gradient explosion.
[0089] It can be understood that the long short-term memory network solves the problem of gradient message or gradient explosion of the basic recurrent neural network. The long short-term memory network effectively alleviates this problem through its "gating device". Specifically, in order to effectively process sequence data, the basic recurrent neural network will save the value of the hidden layer at each moment and use it again at the next moment, so that each moment contains the information of the previous moment and stores the information of each moment in the memory cell. On this basis, the long short-term memory network provides three gating units, input gate, output gate and forget gate. Among them, when the input gate is opened, the input information will be input into the memory cell, otherwise, the input information will not be input into the memory cell; when the output gate is opened, the information in the memory cell will be output, otherwise, the information in the memory cell will not be output; when the forget gate is opened, the information in the memory cell will be cleared, otherwise, the information in the memory cell will be kept.
[0090] It is understandable that through training, the LSTM network can learn what information should be remembered and what information should be forgotten. However, the LSTM network cannot encode information from the back to the front, that is, the LSTM network can learn the "previous" context information very well, but cannot capture the "next" context information. The bidirectional LSTM network can solve this problem. The bidirectional LSTM network is composed of two LSTM networks, one of which processes the input sequence forward; the other processes the sequence in reverse. After the processing is completed, the outputs of the two LSTM networks are spliced together to obtain the final output result of the bidirectional LSTM network.
[0091] It can be understood that the essence of the pooling layer is downsampling, which can achieve dimensionality reduction, remove redundant information, compress features, simplify network complexity, reduce the amount of calculation, reduce memory consumption, etc., that is, reduce the number of parameters; while expanding the perceptual field, it can achieve translation invariance, rotation invariance and scale invariance.
[0092] It can be understood that the fully connected layer combines all the features extracted before the fully connected layer to obtain a global feature represented by a one-dimensional vector.
[0093] In the embodiment of the present application, the trained text classification model also includes: a recurrent neural network layer, a pooling layer, a fully connected layer, and a classification layer; the target word sequence vector is input into the trained text classification model for classification to obtain a classification result, including: the target word sequence vector is input into the recurrent neural network layer, and the text word sequence corresponding to the target word sequence vector is output, and the text word sequence integrates the context information of the target word sequence vector; the text word sequence is input into the dynamic convolution layer for feature extraction and feature mapping, and the text feature matrix is output; the text feature matrix is input into the pooling layer for dimensionality reduction compression, and the target feature matrix is output; the target feature matrix is input into the fully connected layer for feature fusion, and the target vector is output; the target vector is input into the classification layer for classification to obtain a classification result. In this way, the trained text classification model can effectively extract the context information of the target sequence, and combines the temporal information of the text, so that the text representation is more accurate and rich, thereby making the text classification result more accurate.
[0094] In some embodiments of the present application, Figure 1 ,like Figure 8 As shown, before the above step 102, the text classification method provided in the embodiment of the present application also includes the following steps 107 to 110.
[0095] 107. Build a text classification model.
[0096] 108. Build a training set.
[0097] The training set includes multiple sample texts and the actual classification results corresponding to each sample text.
[0098] 109. Preprocess each sample text among the multiple sample texts to obtain a training word sequence corresponding to each sample text, so as to obtain a training word sequence set.
[0099] 110. Train the text classification model by using a training word sequence set to obtain a trained text classification model.
[0100] Among them, multiple sample texts are classified by the trained text classification model, and among the multiple predicted classification results obtained, the proportion of predicted classification results that are the same as the corresponding actual classification results is greater than or equal to the accuracy threshold.
[0101] It can be understood that the text classification model is trained using the constructed training set to ensure that the classification accuracy of the trained text classification model is greater than or equal to the accuracy threshold.
[0102] In an embodiment of the present application, the target sequence is input into a trained text classification model for classification, and before obtaining the classification result, the method further includes: constructing a text classification model; constructing a training set, the training set including multiple sample texts and the actual classification result corresponding to each sample text; preprocessing each sample text in the multiple sample texts to obtain a training word sequence corresponding to each sample text, so as to obtain a training word sequence set; training the text classification model through the training word sequence set to obtain a trained text classification model; wherein, the multiple sample texts are classified through the trained text classification model, and among the multiple predicted classification results obtained, the proportion of predicted classification results that are the same as the corresponding actual classification results is greater than or equal to the accuracy threshold. The classification accuracy of the trained text classification model is greater than or equal to the accuracy threshold, thereby ensuring that when the target sequence is classified, the classification result obtained is more reliable.
[0103] like Fig. 9 As shown, a text classification device provided by an embodiment of the present application includes: an acquisition module 901, used to acquire a target sequence, the target sequence is used to indicate the text to be classified; an input module 902, used to input the target sequence into a trained text classification model for classification to obtain a classification result, the trained text classification model includes a dynamic convolution layer, the dynamic convolution layer is used to: determine the weight of each preset convolution kernel in a plurality of preset convolution kernels based on a text word sequence, and based on the plurality of preset convolution kernels and the weight of each preset convolution kernel, perform convolution processing on the text word sequence to obtain a text feature matrix; the text word sequence is obtained based on the target sequence, the text feature matrix is used to determine the classification result, and the plurality of preset convolution kernels have the same size and different convolution kernel parameters.
[0104] In some embodiments of the present application, the text classification device also includes: a word segmentation processing module and a filtering module; the word segmentation processing module is used to perform word segmentation processing on the target sequence before inputting the target sequence into a trained text classification model for classification to obtain at least one ordered word; the filtering module is used to filter out words in the at least one ordered word that belong to a stop word list to obtain a target word sequence, the stop word list includes multiple stop words, and each stop word does not affect the text classification result; the input module 902 is specifically used to input the target word sequence into a trained text classification model for classification to obtain a classification result.
[0105] In some embodiments of the present application, the input module 902 is also used to filter out words in the stop word list from at least one ordered word to obtain a target word sequence, and then input the target word sequence into a pre-trained word vector model to obtain a target word sequence vector; the input module 902 is specifically used to input the target word sequence vector into a trained text classification model for classification to obtain a classification result.
[0106] In some embodiments of the present application, the text classification device also includes: a training module; the training module is used to train the global vector GloVe model based on the corpus to obtain a trained GloVe model before inputting the target word sequence into a pre-trained word vector model to obtain the target word sequence vector; the input module 902 is specifically used to input the target word sequence into the trained GloVe model to obtain the target word sequence vector.
[0107] In some embodiments of the present application, the trained text classification model also includes: a recurrent neural network layer, a pooling layer, a fully connected layer, and a classification layer; the input module 902 is specifically used to input the target word sequence vector into the recurrent neural network layer, and output the text word sequence corresponding to the target word sequence vector, and the text word sequence integrates the context information of the target word sequence vector; input the text word sequence into the dynamic convolution layer for feature extraction and feature mapping, and output the text feature matrix; input the text feature matrix into the pooling layer for dimensionality reduction and compression, and output the target feature matrix; input the target feature matrix into the fully connected layer for feature fusion, and output the target vector; input the target vector into the classification layer for classification to obtain the classification result.
[0108] In some embodiments of the present application, the recurrent neural network layer includes any of the following: a long short-term memory layer, a bidirectional long short-term memory layer.
[0109] In some embodiments of the present application, the text classification device also includes: a construction module and a preprocessing module; the construction module is used to construct a text classification model before inputting the target sequence into the trained text classification model for classification to obtain the classification result; the construction module is also used to construct a training set, the training set includes multiple sample texts and actual classification results corresponding to each sample text; the preprocessing module is used to preprocess each sample text in the multiple sample texts to obtain a training word sequence corresponding to each sample text to obtain a training word sequence set; the training module is also used to train the text classification model through the training word sequence set to obtain a trained text classification model; wherein, when multiple sample texts are classified through the trained text classification model, among the multiple predicted classification results obtained, the proportion of predicted classification results that are the same as the corresponding actual classification results is greater than or equal to the accuracy threshold.
[0110] The text classification device provided in the embodiment of the present application can be used to execute the steps of the above method embodiment. Its implementation principle and technical effects are similar and will not be repeated here.
[0111] It should be noted that the above-mentioned text classification device can be the electronic device in the above-mentioned method embodiment of the present application, or it can be a functional module and / or functional entity in the electronic device that can implement the functions of the device embodiment, and the embodiment of the present application is not limited.
[0112] The beneficial effects of various implementation methods in this embodiment can be specifically referred to the beneficial effects of the corresponding implementation methods in the above-mentioned text classification method embodiment. To avoid repetition, they will not be described again here.
[0113] The present application also provides an electronic device, such as Fig.10 As shown, the electronic device may include: a processor 1001, a memory 1002, and a program or instruction stored in the memory 1002 and executable on the processor 1001. When the program or instruction is executed by the processor 1001, each process of the text classification method provided in the above method embodiment can be implemented, and the same technical effect can be achieved. To avoid repetition, it will not be described here.
[0114] An embodiment of the present application provides a vehicle, which includes the above-mentioned data processing device or electronic device and can achieve the same technical effect. To avoid repetition, it will not be described here.
[0115] An embodiment of the present application provides a computer-readable storage medium, on which a program or instruction is stored. When the program or instruction is executed by a processor, the various processes of the text classification method provided in the above method embodiment are implemented, and the same technical effect can be achieved. To avoid repetition, it will not be repeated here.
[0116] The embodiment of the present application also provides a computer program product, wherein the computer program product includes a computer program or instructions. When the computer program product runs on a processor, the processor executes the computer program or instructions to implement the various processes of the text classification method provided in the above method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0117] An embodiment of the present application further provides a chip, which includes a processor and a communication interface, wherein the communication interface is coupled to the processor, and the processor is used to run programs or instructions to implement the various processes of the above-mentioned text classification method embodiment, and can achieve the same technical effect. To avoid repetition, it will not be repeated here.
[0118] It should be understood that the chip mentioned in the embodiments of the present application can also be called a system-level chip, a system chip, a chip system or a system-on-chip chip, etc.
[0119] In the several embodiments provided in the present application, it should be understood that the disclosed systems, devices, servers and methods can be implemented in other ways. For example, the device embodiments described above are only schematic. For example, the division of the units is only a logical function division. There may be other division methods in actual implementation, such as multiple units or components can be combined or integrated into another system, or some features can be ignored or not executed. Another point is that the mutual coupling or direct coupling or communication connection shown or discussed can be an indirect coupling or communication connection through some interfaces, devices or units, which can be electrical, mechanical or other forms.
[0120] The units described as separate components may or may not be physically separated, and the components shown as units may or may not be physical units, that is, they may be located in one place or distributed on multiple network units. Some or all of the units may be selected according to actual needs to achieve the purpose of the solution of this embodiment.
[0121] In addition, each functional unit in each embodiment of the present application may be integrated into one processing unit, or each unit may exist physically separately, or two or more units may be integrated into one unit. The above-mentioned integrated unit may be implemented in the form of hardware or in the form of software functional units.
[0122] If the integrated unit is implemented in the form of a software functional unit and sold or used as an independent product, it can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present application is essentially or the part that contributes to the prior art or all or part of the technical solution can be embodied in the form of a software product, and the computer software product is stored in a storage medium, including a number of instructions to enable a computer device (which can be a personal computer, a server, or a network device, etc.) to perform all or part of the steps of the method described in each embodiment of the present application. The aforementioned storage medium includes: various media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk.
[0123] As described above, the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit them. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.
Claims
1. A text classification method, characterized in that: include: Obtaining a target sequence, where the target sequence is used to indicate a text to be classified; Inputting the target sequence into a trained text classification model for classification to obtain a classification result, wherein the trained text classification model includes a dynamic convolution layer, and the dynamic convolution layer is used to: determine a weight of each of a plurality of preset convolution kernels based on a text word sequence, and perform convolution processing on the text word sequence based on the plurality of preset convolution kernels and the weight of each preset convolution kernel to obtain a text feature matrix; The text word sequence is obtained based on the target sequence, the text feature matrix is used to determine the classification result, and the multiple preset convolution kernels have the same size and different convolution kernel parameters.
2. The method according to claim 1, characterized in that Before inputting the target sequence into the trained text classification model for classification, the method further includes: Perform word segmentation on the target sequence to obtain at least one ordered word; Filtering out words in a stop word list from the at least one ordered word to obtain a target word sequence, wherein the stop word list includes a plurality of stop words, each of which does not affect a text classification result; Inputting the target sequence into a trained text classification model for classification to obtain a classification result includes: The target word sequence is input into the trained text classification model for classification to obtain the classification result.
3. The method according to claim 2, characterized in that After filtering out words in the stop word list from the at least one ordered word to obtain a target word sequence, the method further includes: Input the target word sequence into a pre-trained word vector model to obtain a target word sequence vector; Inputting the target sequence into a trained text classification model for classification to obtain a classification result includes: The target word sequence vector is input into the trained text classification model for classification to obtain the classification result.
4. The method according to claim 3, characterized in that Before inputting the target word sequence into a pre-trained word vector model to obtain the target word sequence vector, the method further includes: Based on the corpus, train the global vector GloVe model to obtain the trained GloVe model; Inputting the target word sequence into a pre-trained word vector model to obtain a target word sequence vector includes: The target word sequence is input into the trained GloVe model to obtain the target word sequence vector.
5. The method according to claim 3, characterized in that The trained text classification model further includes: a recurrent neural network layer, a pooling layer, a fully connected layer, and a classification layer; inputting the target word sequence vector into the trained text classification model for classification to obtain the classification result includes: Inputting the target word sequence vector into the recurrent neural network layer, and outputting the text word sequence corresponding to the target word sequence vector, wherein the text word sequence incorporates context information of the target word sequence vector; Inputting the text word sequence into the dynamic convolution layer for feature extraction and feature mapping, and outputting the text feature matrix; Input the text feature matrix into the pooling layer for dimensionality reduction compression, and output the target feature matrix; Input the target feature matrix into the fully connected layer for feature fusion, and output a target vector; The target vector is input into the classification layer for classification to obtain the classification result.
6. The method according to claim 5, characterized in that The recurrent neural network layer includes any one of the following: a long short-term memory layer, a bidirectional long short-term memory layer.
7. The method according to any one of claims 1 to 6, characterized in that Before inputting the target sequence into a trained text classification model for classification and obtaining a classification result, the method further includes: Constructing the text classification model; Constructing a training set, wherein the training set includes a plurality of sample texts and an actual classification result corresponding to each sample text; Preprocessing each sample text among the plurality of sample texts to obtain a training word sequence corresponding to each sample text to obtain a training word sequence set; The text classification model is trained using the training word sequence set to obtain the trained text classification model; wherein, the multiple sample texts are classified using the trained text classification model, and among the multiple predicted classification results obtained, the proportion of predicted classification results that are the same as the corresponding actual classification results is greater than or equal to an accuracy threshold.
8. A text classification device, characterized in that: include: An acquisition module, configured to acquire a target sequence, wherein the target sequence is used to indicate a text to be classified; An input module is used to input the target sequence into a trained text classification model for classification to obtain a classification result. The trained text classification model includes a dynamic convolution layer. The dynamic convolution layer is used to: determine the weight of each preset convolution kernel in multiple preset convolution kernels based on the text word sequence, and based on the multiple preset convolution kernels and the weight of each preset convolution kernel, perform convolution processing on the text word sequence to obtain a text feature matrix. The text word sequence is obtained based on the target sequence. The text feature matrix is used to determine the classification result. The multiple preset convolution kernels have the same size and different convolution kernel parameters.
9. An electronic device, characterized in that: include: A memory and a processor, wherein the memory is used to store a computer program; and the processor is used to execute the text classification method according to any one of claims 1 to 7 when calling the computer program.
10. A vehicle, characterized in that: include: The text classification device according to claim 8 or the electronic device according to claim 9.
11. A computer-readable storage medium having a computer program stored thereon, characterized in that: When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 7 are implemented.
Citation Information
Patent Citations
Text classification method and device based on attention and computer readable medium
CN114579743A
Text classification method and device and storage medium
CN114860930A
Text feature extraction method and device, model training method and device, equipment and medium
CN115168590A
Cited By
Health data processing method, system, equipment, medium and product
CN122365549A
