Text classification method based on two-channel semantic enhancement and convolutional neural network

Abstract

The invention provides a two-channel semantic enhancement and convolutional neural network-based text classification method, which comprises the following steps of: S1, performing preprocessing and word vector embedding on a text to obtain a text matrix X; s2, performing dual-channel convolution feature extraction on the generated text matrix X by adopting Conv1D and AtrousConv1D respectively to obtain original semantic information C and global text information A; and S3, generating attention scores Cscore and Ascore for the generated original semantic information C and global text information A by adopting weighted average attention, performing semantic enhancement on the text matrix X to obtain yk, finally splicing high-dimensional convolutional feature maps, and then mapping the spliced feature maps into probability distribution of labels through a Linear full-connection layer and a Sofrmax layer. The method can be suitable for Chinese and English text classification tasks, and has good classification performance and generalization ability in both long texts and short texts.

Description

Technical Field

[0001] The present invention relates to the field of text classification, and in particular to a text classification method based on dual-channel semantic enhancement and convolutional neural network. Background Art

[0002] With the popularity of social media, the Internet has generated a large amount of text data. As a key area of ​​natural language processing, text classification plays an important role in tasks such as spam filtering, news classification, and sentiment analysis. However, traditional machine learning models require a lot of manual labor to annotate text features when processing text classification, especially when processing short and long texts. Short texts have poor classification results due to limited context information, while long texts are difficult to capture long-distance dependencies. The emergence of deep learning models has provided a new way to solve these problems, mainly through two stages: pre-trained word vector representation and end-to-end text classification, effectively overcoming the dilemma of manually annotating text information. Among them, word vector embedding methods such as Word2Vec, GloVe, and FastText are widely used, while deep learning models such as TextCNN and TextRNN perform well in sentence semantic mining and capturing long-distance dependency information. Despite this, these models still have certain limitations, such as the gradient vanishing or gradient exploding problems faced by TextRNN.

[0003] Although deep learning models have achieved certain results in text classification tasks, they still face many challenges. First, the generalization ability and universal applicability of existing deep learning models need to be further enhanced when dealing with different languages ​​(such as Chinese and English) and different text lengths (such as short text and long text). Second, although some research works have introduced modules such as attention mechanisms, neighborhood information, feature reconstruction, and adopted integration strategies to capture deep semantic information, these methods still have poor text mining capabilities in practical applications. For example, Ai W, ​​Wei Y, Shao H, et al. developed a graph attention network EMGAN based on the edge enhancement module (see the paper "Edge-enhanced minimum-margin graph attention network for short text classification" in "Expert Systems with Applications" for details). The heterogeneous information graph constructed in this model can effectively represent the complex relationship between text features and enhance semantic information. However, such methods have high complexity and lack good scalability for text mining tasks in different environments to a certain extent. Therefore, how to build a robust text classification model that can be applied to Chinese and English contexts and texts of different lengths is a key issue in current research on text classification tasks. Summary of the invention

[0004] The present invention aims to at least solve the technical problems existing in the prior art, and in particular innovatively proposes a text classification method based on dual-channel semantic enhancement and convolutional neural network.

[0005] In order to achieve the above-mentioned object of the present invention, the present invention provides a text classification method based on dual-channel semantic enhancement and convolutional neural network, comprising the following steps:

[0006] S1, first preprocess the text and divide the text into text units; then perform word vector embedding to convert words or phrases into vector representations; thus, the text matrix X is obtained;

[0007] S2, the generated text matrix X is subjected to dual-channel convolution feature extraction using Conv1D (one-dimensional convolution) and AtrousConv1D (one-dimensional hole convolution) to obtain the original semantic information C and the global text information A;

[0008] S3, the weighted average attention is used to generate the attention score C for the generated original semantic information C and the global text information A score and A score , and semantic enhancement is performed on the text matrix X to obtain y kFinally, the high-dimensional convolutional feature maps are concatenated, and then the concatenated feature maps are mapped to the probability distribution of the label through the Linear fully connected layer and the Sofrmax layer.

[0009] Preferably, the text in step S1 is Chinese text or English text. In this case, the word vector embedding is:

[0010] When the text is Chinese, the pre-trained word vector of the Chinese corpus is used to represent each Chinese character;

[0011] When the text is in English, the pre-trained word vector of the English corpus is used to represent each word.

[0012] Preferably, the word vector embedding is performed to convert the text unit into a vector representation; including: the text S is formed into a text embedding vector X through the corresponding index of the vocabulary table V:

[0013] X=index(S)*V 1)

[0014] Among them, index(S) represents the process of generating X by indexing the pre-training corpus;

[0015] V stands for vocabulary.

[0016] Preferably, using Conv1D (one-dimensional convolution) to extract features, obtaining the original semantic information C includes the following steps:

[0017] For the i-th word in text S, the vector representation is x i , the text embedding vector X is subjected to sliding convolution on X under a one-dimensional convolution kernel w with a convolution kernel size of z, and the feature h is generated by equation (2) i :

[0018] [x i :x i+z-1 ]=[x i ,x i+1 ,...,x i+z-1 ]2)

[0019] h i =σ(w1·[x i :x i+z-1 ]+b1)3)

[0020] where [x i :x i+z-1 ] represents the sliding window range of the convolution kernel;

[0021] x i Represents the word vector of the i-th word in a sentence;

[0022] x i+z-1Represents the word vector of the i+z-1th word in a sentence;

[0023] w1 represents the feature weight, which is an optimizable parameter matrix;

[0024] h i It represents the convolution feature information after the convolution kernel has undergone sliding convolution for the i-th time, which is generated by the dot product operation of the optimizable parameter w1 and the sentence vector plus the bias b1;

[0025] σ is a nonlinear activation function, specifically LeakyReLU; this is because the activation function ReLU has the problem of neurons being discarded. Therefore, it is more appropriate to choose the activation function LeakyReLU when facing the problem of dead neurons in the network.

[0026] The convolution feature h i The mapping is represented as a new convolutional feature H:

[0027] H=[h1,h2,...,h l-z+1 ]5)

[0028] Thus, the extracted convolution feature M is obtained:

[0029] M=[H1,H2,...,H m ]6)

[0030] Where H1 represents the convolution feature generated by the first convolution channel;

[0031] H m Represents the convolution feature generated by the mth convolution channel;

[0032] m is the total number of convolution channels;

[0033] In order to reduce the loss of semantic information of features extracted by the convolutional layer, we use an average pooling operation on the convolutional feature map M to obtain the original semantic information of the one-dimensional convolution channel. The pooling output C of the convolutional feature map M is expressed as:

[0034] C = avgpool(M)7)

[0035] Among them, C represents the original semantic information;

[0036] avgpool() represents average pooling, which is used by Conv1D to capture the original semantic information of the text.

[0037] Preferably, the convolution kernel size z≤4.

[0038] Preferably, the convolution kernel size is set to z={2,3,4}.

[0039] Preferably, AtrousConv1D (one-dimensional hole convolution) is used for feature extraction to obtain global text information A:

[0040] A=maxpool{LeakyReLU(Z)}9)

[0041] Among them, LeakyReLU() represents a nonlinear activation function; this is because the activation function ReLU has the problem of neurons being discarded. Therefore, it is more appropriate to choose the activation function LeakyReLU when facing the problem of dead neurons in the network.

[0042] maxpool() represents the maximum pooling operation;

[0043] Z = AtrConv1D(X)

[0044] Where Z = [z1,z2,...,z K ], in the filter w of length K k After dilation convolution, the global output z is generated i :

[0045]

[0046] Where x[i+p·k] represents the word vector after AtrousConv1D expansion;

[0047] p represents the stride of the dilated convolution on the input vector x[i+p·k], p∈P, P={p1,p2,p3}, and p1, p2, and p3 represent the three semantic receptive field expansion rates.

[0048] The advantage of using AtrousConv1D is that it integrates the global semantics of one-dimensional convolution and can skip the sequence of p·k to obtain long-distance semantic dependencies. The feature map M after AtroussConv1D and LeakyReLU activation a A maxpool operation is performed to obtain the maximum feature map A of sentence-level category preference.

[0049] Preferably, step S3 comprises the following steps:

[0050] S3-1, use weighted average attention to generate attention score C for the generated original semantic information C and global text information A score and A score :

[0051]

[0052] Among them C score and A scoreThe weighted attention scores obtained for Conv1D and AtrousConv1D convolutions respectively;

[0053] softmax(v c T u c ) i v c T with u c The matrix product probability mapping of ;

[0054] softmax(v a T u a ) i v a T with u a The matrix product probability mapping of ;

[0055] T is the transposition symbol;

[0056] C i is the i-th convolution feature map of C;

[0057] A i is the i-th convolution feature map of A;

[0058] l is the sentence length;

[0059] z is the convolution kernel size;

[0060] v c and v a The eigenvectors H and H are a , H is the mapping feature of the text matrix X after Conv1D convolution; H a It is the mapping feature of the text matrix X after AtrousConv1D convolution;

[0061] u c and u a Then it is the high-dimensional feature score after linear change, as shown in formula (12):

[0062]

[0063] Among them, M a They are the convolution features after the one-dimensional convolution channel and the convolution features after the AtrousConv1D convolution channel respectively;

[0064] Tanh is the activation function to calculate the attention score;

[0065] w1 and w2 represent the feature weights of the convolution channels of Conv1D and AtrousConv1D respectively;

[0066] b1 and b2 represent the bias of the convolution channels of Conv1D and AtrousConv1D respectively;

[0067] S3-2, using the semantic enhancement module, semantically enhances the text matrix X and activates the generated y k , expressed as:

[0068] y k =σ1(conv(X;w k ,b k ))13)

[0069] Where σ1 represents the ReLU activation function;

[0070] y k Indicates enhanced features;

[0071] conv() represents the convolution operation, which is a one-dimensional convolution;

[0072] w k Represents the feature weight of the convolution channel;

[0073] b k Represents the bias of the convolution channel;

[0074] k represents the optimizable feature weight of the convolution channel (Conv1D or AtrousConv1D);

[0075] S3-3, y k After maxpool1D, the enhanced feature y is obtained by division. k c and k a ; Then y k c and k a Concatenate with the weighted attention features extracted by dual-channel convolution:

[0076]

[0077] Where C and A are the original semantic information and global text information obtained by the convolution of Conv1D and AtrousConv1D respectively;

[0078] C score and A score The weighted attention scores obtained by Conv1D and AtrousConv1D convolutions respectively;

[0079] y k c Representation and high-dimensional volume feature map C·C scoreSemantically enhanced features generated by splicing;

[0080] y k a Representation and high-dimensional volume feature map A·A score Semantically enhanced features generated by splicing;

[0081] Represents a splicing operation;

[0082] S3-4, the concatenated feature map H cla It is mapped to the probability distribution of labels through the Linear fully connected layer and the Sofrmax layer.

[0083] Preferably, a cross entropy loss function is used to measure the difference between the predicted probability distribution and the true label. The cross entropy loss function is expressed as;

[0084]

[0085] in represents the loss used for classification;

[0086] n is the number of test samples;

[0087] y is the true value label;

[0088] is the predicted label.

[0089] In summary, due to the adoption of the above technical solution, the beneficial effects of the present invention are:

[0090] 1) A method that can be applied to Chinese and English text classification tasks is proposed, which has good classification performance and generalization ability for both long and short texts.

[0091] 2) A dual-channel text convolutional neural network structure with the synergy of semantic enhancement and weighted attention is constructed, which has stronger semantic mining ability and longer distance dependency exploration ability in sentence-level text classification tasks.

[0092] Additional aspects and advantages of the present invention will be given in part in the following description and in part will be obvious from the following description, or will be learned through practice of the present invention. BRIEF DESCRIPTION OF THE DRAWINGS

[0093] The above and / or additional aspects and advantages of the present invention will become apparent and easily understood from the description of the embodiments in conjunction with the following drawings, in which:

[0094] Figure 1 It is a schematic diagram of the framework of the DcSeCNN model of the present invention.

[0095] Figure 2It is a schematic diagram of the one-dimensional convolution and AtrousConv1D convolution of the present invention.

[0096] Figure 3 It is a schematic diagram of model comparison (Acc) of all data sets.

[0097] Figure 4 Schematic diagram of the effect of dropout size (Acc) in DcSeCNN.

[0098] Figure 5 Schematic diagram of the effect of λ size (Acc) in DcSeCNN.

[0099] Figure 6 This is a schematic diagram of the impact of padsize on short text in three comparison models.

[0100] Figure 7 This is a schematic diagram of the impact of padsize on long text in three comparison models. DETAILED DESCRIPTION

[0101] Embodiments of the present invention are described in detail below, examples of which are shown in the accompanying drawings, wherein the same or similar reference numerals throughout represent the same or similar elements or elements having the same or similar functions. The embodiments described below with reference to the accompanying drawings are exemplary and are only used to explain the present invention, and cannot be understood as limiting the present invention.

[0102] The present invention aims to build a lightweight text classification model with high robustness, called DcSeCNN; the model uses TextCNN for feature enhancement to meet the performance challenges in complex text mining tasks. With the explosive growth of social media data, tasks in the fields of news annotation and information retrieval have put forward higher requirements on the accuracy of short text classification technology. However, in changeable and complex situations, deep learning models based on deep semantic mining often find it difficult to achieve ideal performance. To solve this problem, the present invention patent proposes an innovative solution, which is to apply effective semantic enhancement and attention mechanism variants to TextCNN to develop a new lightweight robust text classification model. With this method, the model can more effectively mine the deep semantic information in the text and improve the classification accuracy under complex conditions.

[0103] The present invention not only utilizes the advantages of TextCNN in feature extraction, but also enhances the robustness and accuracy of the model in complex text classification tasks by introducing semantic enhancement and attention mechanism variants, providing strong support for the efficient processing of social media data.

[0104] The framework diagram of the DCSeCNN model used for text classification tasks in the present invention is as follows: Figure 1As shown. The framework mainly consists of three modules: preprocessing stage for Chinese and English, dual-channel convolution, semantic enhancement, and weighted attention. This simple and lightweight DCSeCNN structure can perform effective semantic mining and knowledge capture in a variety of text environments (different text scales, different text lengths, Chinese and English). Specifically: first perform text preprocessing and word vector embedding on the text (Chinese or English). For Chinese text, the pre-trained word vector of the SougouNews corpus is used to represent each Chinese character, and for English, the pre-trained word vector of the GloVe corpus is used to represent each word. The generated text matrix X uses Conv1D and AtrousConv1D for dual-channel convolution feature extraction, and the generated C and A use weighted average attention to generate the attention score C. score and A score At the same time, semantic enhancement is performed on X to obtain y k Finally, the high-dimensional convolutional feature maps are concatenated and generated through the Linear fully connected layer and Sofrmax.

[0105] 1 Text preprocessing stage

[0106] Text preprocessing is the process of converting the pre-trained word vectors of sentences in the dataset into a real-valued matrix. The input sentence can be represented as S = {s1, s2, ..., s l}, S∈S; where s i represents each word or Chinese character in the sentence, l is the length of a sentence, and can also be regarded as l tokens of this text representation. Here we use GloVe and pre-trained embedding from the SougouNews corpus, and the dimension d of each word embedding vector is 300. Finally, after embedding all the sentences in the dataset, a vocabulary can be formed v represents the size of the vocabulary, and d represents the embedding dimension of 300. All word vectors of the vocabulary V constructed by a dataset can be understood as the word vector corresponding to each Chinese character or word in this dataset. i After word vector embedding, it is converted into vector x i . At this time, the text S is represented as a matrix x l n The word vector corresponding to the last word or Chinese character of the nth sentence is 1*300. Its mapping relationship is shown in formula (1).

[0107] X=index(S)*V 1)

[0108] Where n represents the total number of sentences;

[0109] index(S) represents the process of generating X from the original text by indexing the pre-trained corpus;

[0110] V stands for vocabulary;

[0111] The text S is converted into a computer-recognizable text embedding vector X through the corresponding index of the vocabulary V.

[0112] Divide the text embedding vector X into training sets X train , validation set X dev and the test set X test , where X train and X dev The model input is used for the model after training and hyperparameter optimization. test To simplify the complexity of the model principle, we use X in the following introduction to represent the sentence-level embedding vector involved in the training.

[0113] 2Dual channel convolution module

[0114] The matrix X output from the previous stage is fed forward to the dual channels of the classic one-dimensional convolution and the Atrous convolution to obtain the feature sequence. The one-dimensional convolution can obtain the local original semantic information through the sentence sequence, while the Atrous convolution can expand the global text information at the sentence level, increasing the probability of avoiding overall semantic noise. The convolution process of the two channels is as follows: Figure 2 shown.

[0115] 2.1 One-dimensional convolution channel

[0116] For the original text S (i.e. text S), the i-th word representation vector is x i , with input text vector (i.e. text embedding vector X) in a one-dimensional convolution kernel with a convolution kernel size of z In this case, we can use formula (2) to perform sliding convolution on X to generate higher quality features h i .

[0117] [x i :x i+z-1 ]=[x i ,x i+1 ,...,x i+z-1 ]2)

[0118] h i =σ(w1·[x i :x i+z-1 ]+b1)3)

[0119] where [x i :x i+z-1 ] represents the sliding window range of the convolution kernel;

[0120] x i Represents the word vector of the i-th word in a sentence;

[0121] x i+z-1 Represents the word vector of the i+z-1th word in a sentence;

[0122] h i It represents the convolution feature information after the convolution kernel has undergone sliding convolution for the i-th time, which is generated by the dot product operation of the one-dimensional convolution kernel and the sentence vector plus the bias b1.

[0123] σ uses LeakyReLU, a variant of ReLU, as the nonlinear activation function.

[0124] The initial values ​​of w1 and b1 are system pseudo-random numbers, which are gradually optimized during the training process.

[0125] At this time, the convolution kernel is used for the word sliding window of the sentence:

[0126] {h1,h2,...,h l-z+1}={[x1:x z ],[x2:x z+1 ],...,[x l-z+1 :x l ]}4)

[0127] It means that the convolution kernel generates an h each time it performs a convolution operation. i .

[0128] The feature map after the convolution can be represented as a new convolution feature H.

[0129] H=[h1,h2,...,h l-z+1 ]5)

[0130] It is found that when the convolution kernel is too large in text classification, it not only increases the number of parameters, but also may incorporate the information of noise words into the phrase representation, resulting in much worse classification results. When the convolution kernel size z≤4 is the most conducive to the feature extraction of text information. Therefore, the convolution kernel size is set to z={2,3,4} in this paper to increase the diversity of feature extraction. The extracted convolution feature can be represented as M.

[0131] M=[H1,H2,...,H m ]6)

[0132] Where H1 represents the convolution feature generated by the first convolution channel;

[0133] H m Represents the convolution feature generated by the mth convolution channel;

[0134] m is the total number of convolution channels.

[0135] In order to reduce the loss of semantic information of features extracted by the convolution layer, we use average pooling on the convolution feature map M to obtain the original semantic information of the one-dimensional convolution channel. The pooled output C of the convolution feature map M = {C1 avg ,C2 avg ,...,C m avg}As shown in formula (7).

[0136] C i avg =avgpool{H i}

[0137] C = avgpool(M)7)

[0138] C i avg Represents the high-dimensional convolution feature map generated by the i-th H after the average pooling operation;

[0139] The one-dimensional convolution channel extracted C is used as the input of the weighted average attention module to generate the attention score C score , after the dot product operation, it is used as one of the final fully connected input features and is used for the final probability distribution output classification prediction through the fully connected layer.

[0140] 2.2 Atrous Convolution Channel

[0141] Atrous convolutional neural networks were originally used to expand the receptive field in image semantic segmentation to obtain the global semantics of the image. In text classification tasks, when a larger one-dimensional convolution kernel (sliding window width z) is used, it is impossible to obtain maximized semantic information. Therefore, the Atrous convolution channel with an expanded receptive field p provides a new path for extracting global semantics at the sentence level. For short or long texts processed by the present invention, we set the expansion rate corresponding to the convolution kernel w to P = {p1, p2, p3}, where p1, p2, p3 represent three semantic receptive field expansion rates, p1 = p2 = p3 = 1. When extended to sentence convolution, Atrous convolution will also perform semantic extraction in a jumpy manner in a one-dimensional text sequence.

[0142] Specifically, the principle of Atrous convolution is similar to that of one-dimensional convolution. The difference is that the process of performing Atrous on the input sequence will extract the global convolution feature map. Combining equations (2) to (6), the Atrous one-dimensional convolution module can be summarized as shown in equation (9).

[0143]

[0144] A=maxpool{LeakyReLU(Z)}9)

[0145] LeakyReLU() represents a nonlinear activation function;

[0146] maxpool represents the maximum pooling operation; trConv1D uses maximum pooling to obtain the most important semantic information of the text.

[0147] All variables in formula (8) have the same meaning as one-dimensional convolution, and w2 and b2 are parameters that can be optimized by dilated convolution. The difference is that the three-stage high-dimensional features obtained by expanding the Atrous convolution kernel w with different convolution kernel sizes are respectively added with superscript a to distinguish them from traditional convolution. Formula (9) uses its Atrous convolution to operate on the input vector X In a filter w of length K k After dilation convolution, the global output z is generated i The definition is as in formula (10).

[0148]

[0149] Where p represents the stride of the dilated convolution of the input vector x[i+p·k], and i represents the i-th word. Each layer of convolution can be evaluated in P. The standard Atrous convolution dilation rate is 1. Figure 2 (b) is a schematic diagram, i.e., Z = AtrConv1D(X). The advantage of using Atrous convolution is that it integrates the global semantic acquisition of one-dimensional convolution and can skip the sequence of p·k to obtain long-distance semantic dependencies.

[0150] Feature map M after Atrous convolution and LeakyReLU activation a A maxpool operation is performed to obtain the maximum feature map A of sentence-level category preference.

[0151] 3 Semantic Enhancement Module

[0152] The attention mechanism has a good effect in obtaining key category vocabulary and contextual semantic information of sentence-level text. Therefore, we perform feature concatenation with the high-dimensional feature map generated by weighted average attention after the semantic enhancement module. The combination with the semantic enhancement module improves the model's semantic perception of sentences. Weighted attention is mainly used to assign new weights to different parts of the input sequence to highlight the information of more important parts of the sentence. Different weighted attention scores are generated by inputting C and A respectively, as shown in formula (11).

[0153]

[0154] Among them C score and A score The weighted attention scores obtained by one-dimensional convolution and atrous convolution are respectively: and

[0155] softmax(v c T u c ) i v c T with u c The matrix product probability mapping of ;

[0156] softmax(v a T u a ) i v a T with u a The matrix product probability mapping of ;

[0157] · T is the transpose symbol;

[0158] C i is the i-th convolution feature map of C;

[0159] A i is the i-th convolution feature map of A;

[0160] l is the sentence length;

[0161] z is the convolution kernel size;

[0162] v c and v a are the eigenvectors H and H a In order to distinguish the operations of different modules, different characters are used to represent them. c and u a Then the high-dimensional feature score after linear change is shown in formula (12).

[0163]

[0164] Among them, M a They are the convolution features after the one-dimensional convolution channel and the convolution features after the Atrous convolution channel respectively;

[0165] Tanh is the activation function to calculate the attention score;

[0166] w1, w2, b1, and b2 represent the feature weights and biases of one-dimensional convolution and Atrous convolution channels, and are optimizable parameters.

[0167] The semantic enhancement module is used to perform convolution operation on the original patent vector X and then activate it to generate y k , expressed as formula (13):

[0168] y k =σ1(conv(X;w k ,b k ))13)

[0169] Where σ1 represents the ReLU activation function,

[0170] k represents the convolution kernel;

[0171] Then y k After maxpool1D, the enhanced feature y is obtained by division. k c and k a ,

[0172] Finally, y k c and k a They are concatenated with the dual-channel convolution features after weighted attention processing, as shown in formula (14).

[0173]

[0174] Where C and A are the original semantic information and global text information obtained by one-dimensional convolution and Atrous convolution respectively;

[0175] C score and A score The weighted attention scores obtained for one-dimensional convolution and atrous convolution respectively;

[0176] y k c Representation and high-dimensional volume feature map C·C score Semantic enhancement features for splicing;

[0177] y k a Representation and high-dimensional volume feature map A·A score Semantic enhancement features for splicing;

[0178] It represents the concatenation operation, which concatenates the feature map and semantic enhancement features with weighted average attention through one-dimensional convolution and Atrous convolution, and finally maps them into the probability distribution of labels for classification prediction through the fully connected softmax layer. This paper adopts the cross entropy loss function:

[0179]

[0180] in represents the loss used for classification;

[0181] n is the number of test samples;

[0182] y is the ground-truth label;

[0183] is the predicted label.

[0184] The cross entropy loss function can effectively represent the difference between the estimated distribution of samples and the distribution of actual labels.

[0185]

[0186]

[0187] 4 Experimental Proof

[0188] In order to verify the text mining performance of the DCSeCNN model, six different real-world text datasets and four classic evaluation indicators were used to compare the proposed DCSeCNN method with other competitive models. The server configuration information of the experiment is: Intel(R) Xeon(R) Gold 6226R CPU@3.90GHz (32 cores and 64 threads), 128-GB memory, and 2 NVIDIA RTX A6000 GPUs.

[0189] 4.1 Datasets and Evaluation Metrics

[0190] 4.1.1 Dataset Introduction

[0191] The performance verification experiment of the model uses five English datasets including MR, R8, R52, TREC and IMDBR, and the THUCNews Chinese dataset, of which there are three long texts and three short texts. This is conducive to using the same model to train and test text classification models of different languages ​​and lengths, which has verified that the DCSeCNN we proposed has a wide range of applicability for text mining. The specific text dataset information is shown in Table 1.

[0192] Table 1 Experimental dataset information

[0193] Dataset #text #Training set #Validation set / #Test set #Average length #category MR 10662 6396 2133 20.35 2 R8 7674 4604 1534 103.36 8 R52 9100 5460 1820 110.56 52 TREC 5952 3570 1191 9.81 6 IMDBR 50000 30000 10000 240.56 2 THUCNews 200000 120000 40000 16.10 10

[0194] The dataset is divided into training set: validation set: test set = 3:1:1. In order to solve the problem of inconsistent texts of different lengths, we truncate or pad short and long texts to a certain length. GloVe embedding vectors and SougouNews embedding vectors are used for the above 6 datasets (GloVe is used for English texts, SougouNews is used for Chinese texts), and the maximum embedding vector dimension is 300.

[0195] MR1 (Movie Review) is a movie review text dataset, including two labels, pos and neg, collected by Pang and Lee for sentiment analysis experiments

[44] . We select 6396 samples as the training set, and the validation set and test set samples are both 2133. This dataset is used for binary text classification, with short texts with an average sentence length of 20.35.

[0196] R8 and R52 2 (Reuters-21578) is one of the text datasets widely used by researchers in the TC field, which was collected from the Reuters financial news wire service in 1987. For R8, we selected 4604 samples as the training set, 1534 samples for the validation set and the test set, with 8 categories and long texts with an average sentence length of 103.36. For R52, our training set, validation set, and test set samples are 5460, 1820, and 1820, respectively, with 52 categories and an average sentence length of 110.56.

[0197] TREC 3 (Text REtrieval Conference) is a classic dataset for information retrieval and text classification, collected and maintained by the National Institute of Standards and Technology (NIST). We selected 3570 samples as the training set, and the validation set and test set samples are both 1191. It is a short text dataset with an average sentence length of 9.81 and 6 categories.

[0198] IMDBR 4 It is a movie review dataset for polarity sentiment analysis. We select 30,000 samples as the training set, and 10,000 samples for the validation set and the test set. The categories of this dataset are positive and negative emotions, and it is a long text with an average length of 240.56 words.

[0199] THUCNews 5 It is a Chinese news annotation data set filtered and selected based on Sina News. The data downloaded by the patent of this invention is a data set of 200,000 pieces of 10 news types, with 120,000 training set samples, 40,000 validation set samples and 40,000 test set samples, which are Chinese short texts with a sentence length of 16.10.

[0200] Dataset URL:

[0201] 1.https: / / www.cs.cornell.edu / people / pabo / movie-review-data /

[0202] 2.https: / / huggingface.co / datasets / reuters21578

[0203] 3.https: / / trec.nist.gov / data.html

[0204] 4.http: / / ai.stanford.edu / ~amaas / data / sentiment /

[0205] 5.http: / / thuctc.thunlp.org / sendMessage

[0206] 4.1.2 Introduction to evaluation indicators

[0207] The four classic evaluation indicators, Accuracy, Precision, Recall and F1-Score, are used to measure the text classification performance of the comparison model, which are defined as Equation (16) to Equation (19). In order to simplify the description of the performance indicators of the experimental analysis part, we use Acc, Pre, Rec and F1 instead of the above evaluation indicators.

[0208]

[0209] Among them, TP, TN, FP, and FN represent the statistics of the four cases where the prediction and label are both 1, the prediction and label are both 0, the prediction and label are 0 and 1 respectively, and the prediction and label are 1 and 0 respectively. Finally, each indicator and confusion matrix are obtained. In order to ensure the reproducibility of the model and reduce the randomness of the experiment, the model is run independently 5 times. The model is trained with the training set, and then the validation set is used for hyperparameter optimization. Finally, the test set is used for performance testing and statistics of different comparison models.

[0210] 4.2 Baseline Methods

[0211] We selected 7 models from both classic deep learning models and deep learning model variants as baseline methods for the experiment. To ensure a fair comparison, each comparison model was reproduced and a reasonable algorithm performance comparison was performed according to our experimental setting conditions. The information of each baseline method is as follows:

[0212] TextCNN is the first to use CNN for text classification tasks. It is a high-level text classification model that needs to consider text order and long-distance semantic dependencies. It is also used by relevant personnel for comparison and optimization research due to its accelerable performance.

[0213] TextRNN is a text model that uses RNN to capture the sequential dependencies of sentences. The capture of temporal information effectively overcomes the long-distance dependencies of TextCNN and can overcome the gradient explosion problem to a certain extent.

[0214] FastText is a word embedding n-gram feature model developed by Facebook AI Research. This model is superior to traditional text representation methods such as TF-IDF and NBOW, and is superior to similar Word2Vec and GloVe pre-training tools in terms of representation ability and computing speed.

[0215] Transformers abandoned the traditional feature extraction method. The introduction of the Attention mechanism enabled AI to have excellent performance in multiple tasks in multiple fields. It also greatly promoted the development of the NLP field and was a major leap in the field of artificial intelligence.

[0216] TextRCNN combines TextCNN with TextRNN. The combination of RNN and CNN can effectively improve the computing performance of the model and at the same time have advanced text mining capabilities in long texts.

[0217] TextRNN_Att is a text recurrent neural network with an attention mechanism. The introduction of the attention mechanism helps the model focus on the important feature content of the input sequence, and adopts the strategy of survival of the fittest to improve the computational performance and interpretability of the model.

[0218] DPCNN was proposed by Johnson et al.

[46] and is a variant of the TextCNN architecture mainly used for text data mining tasks. It has excellent performance in text classification tasks through pyramid convolutional layer aggregation and deeper network structure.

[0219] 4.3 Model Execution Details

[0220] In the text classification task, the first step is to perform text preprocessing, including word segmentation, padding, and word embedding vectors. For English, space characters are used for word segmentation, and for Chinese, the Jieba library is used to divide a paragraph into each Chinese character. Then, each sample is padded using the short-fill-long-truncation strategy to obtain a standardized data sample. Finally, GloVe and SougouNews are used to represent the word vectors of Chinese and English, respectively. Second, the word vectors are input into the DcSeCNN model for classic convolution and Atrous convolution dual-channel convolution. Third, the extracted dual-channel high-dimensional text features are connected and used as the input of the semantic enhancement module for feature enhancement. This process uses weighted average attention to adjust the global text semantics. Finally, under the condition of fixed parameters set by the system, the learning rate attenuation coefficient is introduced to enhance the adaptability of the learning rate in the optimization process.

[0221] Specifically, the model's operating environment is set as follows: 20 epochs, 0.0001 learning rate, (2, 3, 4) traditional convolution kernel size, (1, 1, 1) Atrous convolution kernel expansion rate, 256 one-dimensional filters, 512 semantic enhancement channels. Each model is run independently for 5 times and the average value is taken as the final statistical evaluation index. During this process, the program is terminated when the model optimization iteration is 1000 times without effect improvement. In addition, according to the scale and sentence length of different datasets in Table 1, we set BatchSize and PadSize as follows: 1) BatchSize of MR, R8, R52, and TREC datasets is 64, and BatchSize of IMDBR and THUCNews datasets is 128; 2) PadSize of MR, TREC, and THUCNews datasets is 32, and PadSize of R8, R52, and IMDBR datasets is 128.

[0222] Table 2 Parameter settings

[0223] parameter default value scope dropout 0.5 0.3,0.4,0.5,0.6,0.7 λ 0.95 0.91,0.93,0.95,0.97,0.99 pad 32 / 128 16 / 32 / 64,64 / 128 / 256

[0224] In the model experiment and performance analysis, we will conduct the following experiments: 1) Study the system parameter settings and conduct experiments on all the comparison models to verify the effectiveness of DcSeCNN; 2) Explore the two models of TextCNN and DcSeCNN to conduct experiments on four datasets: MR, R8, IMDBR and THUCNews to determine the best dropout and learning rate decay coefficient λ to set the optimal hyperparameter combination. The above four datasets contain most text classification situations such as Chinese and English, texts of different scales and lengths, which are used to establish parameters and the portability of the model; 3) Verify the impact of the short-filling and long-cutting strategy on dataset processing through different pad parameter performance; 4) Use ablation experiments to verify the contribution of each module in DcSeCNN to the proposed model. Table 2 shows the information on the settings of the three parameters of dropout, λ and pad.

[0225] 4.4 Experimental Results and Analysis

[0226] 4.4.1 Analysis of comparative experimental results

[0227] In order to verify the performance of the DcSeCNN model in the TC task, we conducted a systematic control experiment on our method and 7 classic baseline models on 6 datasets. The experimental results are shown in Table 3 (the bold labeled data is the first in performance, and the underlined labeled data is the second in performance). Among the 18 statistical data including Pre, Rec and F1, 72% of the statistical indicators of the DcSeCNN model ranked first and 28% of the statistical indicators ranked second, which fully verifies that our model has a very good effect in text data mining tasks under various conditions of different scales and lengths.

[0228] In Table 3, the performance of the models is compared and analyzed from the perspective of data scale. For the two larger datasets IMDBR and THUCNews, the models with better performance in each indicator are TextRCNN, TextRNN_Att and our model. Specifically: DcSeCNN improved by 1.22%, 1.44% and 1.59% on the three indicators of IMDBR, and the Pre of THUCNews was as high as 90.71%, and the other two indicators ranked second. It can be seen that the variant performance of the classic text classification model is generally better than the basic model in such datasets. In the classification performance of the other four smaller data scales, the performance of other models is generally lower than our model.

[0229] Table 3 Pre, Rec and F1 results of different models on 6 text datasets

[0230]

[0231]

[0232] The performance of the models is compared and analyzed from the perspective of text length. For the three short texts MR, TREC, and THUCNews, the DCSeCNN performance indicators rank first and second half each. This shows that for short text datasets, the lack of semantic information will cause the model to have poor ability to mine text information, but there are certain indicators that have been improved on the variants of the deep model. In the remaining three long text datasets, DCSeCNN ranks first in all indicators and is better than all the comparison models, with F1 indicators of 94.07%, 88.42%, and 70.34 respectively. This verifies that our method has stronger text classification capabilities when semantic information is sufficient. In addition, the model comparison of the statistical Acc indicator, such as Figure 3 shown.

[0233] like Figure 3As shown in the figure, the DcSeCNN model has an excellent classification ability in terms of Acc. Specifically, the improvement effect is more obvious in MR. In R8 and R52, TextCNN, DPCNN, and DcSeCNN have higher Acc than other comparison models. All models in TREC and THUCNews tend to the same classification level, but our method has better Acc. In IMDBR, Transformer and TextRCNN have poorer effects, and DcSeCNN is also higher in other models. Through the above model performance comparison and visualization analysis, it can be seen that DcSeCNN has greater advantages in text mining capabilities under various conditions, which greatly verifies the effectiveness and generalization ability of DcSeCNN in text mining tasks.

[0234] 4.4.2 Analysis of the influence of hyperparameters

[0235] (1) Impact of dropout size in DCSeCNN

[0236] DcSeCNN uses different dropouts to screen out a certain ratio of dual-channel convolution feature extraction, effectively ensuring the simplicity of the network structure and the effectiveness of feature representation learning. The method proposed in the article is trained with different dropout settings, and the setting options are shown in Table 2. Then, the effect of the hyperparameter on the classification performance of the model is analyzed from the result visualization, and the dropout value corresponding to the best classification performance of different text datasets is obtained.

[0237] like Figure 4 As shown in the figure, when the dropout size is 0.5, DCSeCNN obtains better Acc on the four datasets of MR, R8, TREC and THUCNews. On the R52 dataset, when the parameter size is 0.7, the classification effect is far worse than other values, resulting in excessive dropout, which will cause the network structure to be unable to fit the high-dimensional features with more categories. At the same time, the performance on other values ​​is also unstable, and the best classification performance of R52 is achieved when the dropout size is 0.4. This means that it is inappropriate to set the dropout too large or too small for classification tasks with more categories. On the larger IMDBR dataset, as the dropout increases, the classification effect gradually decreases, and a large performance difference occurs when the dropout spans from 0.6 to 0.7. This shows that for the classification tasks of large-scale datasets, the dropout setting of DCSeCNN between 0.3-0.5 can better learn and represent high-dimensional features for more samples. Therefore, we believe that excessive dropout will cause the network to fit the data poorly, and a smaller dropout will increase the burden on computing resources, and its setting of 0.5 is appropriate.

[0238] (1) The influence of λ size in DCSeCNN

[0239] For the DCSeCNN model that changes the network structure, whether the learning rate is a fixed parameter or has a certain degree of adaptability is crucial to the classification performance of the model. A fixed learning rate that is too large will cause the model optimization process to converge early, while a learning rate that is too small will slow down the gradient convergence. Therefore, introducing a learning rate with an adaptive attenuation mechanism is of great significance to the performance improvement and downstream task expansion of DCSeCNN. The results obtained within the parameter range given in Table 2 are visualized as follows: Figure 5 shown.

[0240] from Figure 5 It can be seen that, except for the IMDBR dataset, the sensitivity of the other five datasets to the learning rate decay coefficient is in the same distribution trend, showing a performance effect that is high in the middle and low on both sides. Among them, λ is 0.95 on the three datasets of MR, R52 and YHUCNews, which is the best performance, while the classification performance on R8 and TREC is lower than the classification effect when λ is 0.97. On IMDBR, the classification effect of the model within a certain range is negatively correlated with λ, which shows that our learning rate setting is too small on large-scale datasets, but the consistency with the results obtained by dropout verifies the rationality of our experimental work. In general, when the adaptive change rate of λ is too fast, it is not conducive to the semantic mining and representation of text, and it will lead to performance degradation when it is too slow. It is appropriate to set it to 0.95 to decay the learning rate with moderate adaptivity.

[0241] 4.4.3 Analysis of the impact of filling scale

[0242] In order to explore the impact of the padsize short padding and long truncation strategy classification performance of the DCSeCNN model in different languages ​​(Chinese and English) and texts of different lengths. In this paper, the average lengths of the data sets given in Table 1 are processed into short texts and long texts with padsizes of 32 and 128 respectively. When the padsize is set to 16 and 64 for long truncation and 64 and 256 for short padding, the Acc comparison of each model can be obtained through system simulation experiments as shown in Table 4.

[0243] Table 4 Effect of padsize in DcSeCNN (Acc)

[0244]

[0245] For short texts MR, TREC, and THUCNews, our model still has better classification performance on MR after short padding and long truncation. On TREC, it is consistent with TextRCNN when it is long truncated, and ranks second with default settings, which is 0.02% lower than TextRCNN's Acc, and is about 0.8% lower than DPCNN and TextRCNN when it is long truncated. In long texts, DcSeCNN has the best text classification performance under all conditions.

[0246] In addition, we select TextCNN and its variant DPCNN to compare with our DcSeCNN, and visualize the classification performance of the three models on the same dataset, and obtain the visualization of short and long texts with different padsizes. Figure 6 and Figure 7 shown.

[0247] like Figure 6 As shown in the figure, the comparison of different padsizes of MR clearly shows the effectiveness of our solution, while the overall classification ability on TREC is the same, both about 97% higher. Compared with the comparison model in the Chinese text THUCNews, DcSeCNN ranks second when padsize is 32 and 64, and is at the same performance level as the best. Figure 7 As shown in the figure, the performance of each model on IMDBR is improved with the increase of padsize, and our solution has better performance. On R8 and R52, there is also a slight performance improvement with the increase of padsize. At the same time, it can be found that the overall performance of DPCNN is lower than that of the basic TextCNN and our model. This means that we choose the default settings of padsize to be 32 and 128, which is appropriate in terms of both computing performance improvement and computing resource balance. Based on the above performance analysis, it can be shown that the DcSeCNN model can perform text mining tasks under various conditions and has good classification performance.

[0248] 4.4.4 Ablation Experiment

[0249] The main contributions of this patent include three modules: dual-channel convolution, semantic enhancement, and weighted average attention. Dual-channel convolution is to overcome the problem of insufficient semantic information mining of traditional convolution kernel sliding window, and to capture long-distance semantic connections with new Atrous convolution. Semantic enhancement and attention modules can enhance the strong correlation semantics and reduce the noise of weak correlation semantics in the extraction process. Specifically, we classify and remove the learning rate attenuation factor λ, Atrous convolution channel (Dc), semantic enhancement (Se), and weighted average attention mechanism (Att) in DcSeCNN. The Acc and F1 indicators of the statistical experimental results under the default parameter settings in Table 1 are shown in Table 5.

[0250] Table 5 Ablation experiment

[0251]

[0252] Table 5 shows that removing any module will cause the performance of the model to decline to varying degrees. This shows that the various modules or parameter tuning schemes introduced by the model are effective. However, from the underlined data of the experimental results, it can be seen that for MR, IMDBR and THUCNews, the Att module has the least impact on them, while the Se module has the greatest impression on them. In the two same series of R8 and R52, λ has the least impression on them, and TextCNN ranks second, indicating that the introduction of a single module cannot have a profound impression on them, but the synergy of multiple modules has a certain improvement in classification performance. In addition, due to the strong standardization of the data, the attenuation coefficient has the least impact on them, while semantic enhancement has the greatest contribution to the model on TREC. In summary, each module has a positive effect on the optimization of the model to varying degrees, and through synergy, the DcSeCNN model has reliable performance improvement and good applicability in processing text classification tasks in various situations.

[0253] It can be seen that DCSeCNN demonstrates advanced text classification performance on six text classification datasets (three short texts, three long texts, four small-scale datasets, two large-scale datasets, five English datasets and one Chinese dataset), with significant performance improvements on four datasets, namely MR, R8, R52 and TREC, and the best performance on two datasets, namely IMDBR and THUCNews, approaching the same level.

[0254] In summary, the present invention proposes a text classification model based on DcSeCNN. First, the Atrous convolution with long-distance semantic mining capability is introduced and combined with the traditional one-dimensional convolution, which can effectively capture the local and global semantic information of sentence-level text. In order to improve the quality of the convolution feature map, we use a weighted average attention with an attention mechanism to enhance the features of the convolution feature map. Finally, the initial input text vector is subjected to a semantic enhancement operation, and two different original semantics are obtained and fused with the enhanced features to obtain a high-dimensional feature vector that is closer to the original sentence semantics. In addition, the DcSeCNN model is obtained by dynamically tuning the model training process with adaptive decay learning rate λ and drouput. A large number of experiments and data analysis have been carried out on the model, and the results show that the method has excellent text classification performance and good versatility under different conditions (different languages, different lengths, and different scales).

[0255] Although the embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions and variations may be made to the embodiments without departing from the principles and spirit of the present invention, and that the scope of the present invention is defined by the claims and their equivalents.

Claims

1. A text classification method based on dual-channel semantic enhancement and convolutional neural network, characterized in that: The following steps are involved: S1, first preprocess the text and divide the text into text units; then perform word vector embedding to convert words or phrases into vector representations; thus, the text matrix X is obtained; S2, the generated text matrix X is subjected to dual-channel convolution feature extraction using Conv1D and AtrousConv1D to obtain the original semantic information C and global text information A; S3, the weighted average attention is used to generate the attention score C for the generated original semantic information C and the global text information A score and A score , and semantic enhancement is performed on the text matrix X to obtain y k Finally, the high-dimensional convolutional feature maps are concatenated, and then the concatenated feature maps are mapped to the probability distribution of the label through the Linear fully connected layer and the Sofrmax layer.

2. A text classification method based on dual-channel semantic enhancement and convolutional neural network according to claim 1, characterized in that: The text in step S1 is Chinese text or English text. At this time, the word vector embedding is: When the text is Chinese, the pre-trained word vector of the Chinese corpus is used to represent each Chinese character; When the text is in English, the pre-trained word vector of the English corpus is used to represent each word.

3. The text classification method based on dual-channel semantic enhancement and convolutional neural network according to claim 1, characterized in that: The word vector embedding is performed to convert the text unit into a vector representation; including: The text S is converted into a text embedding vector X through the corresponding index of the vocabulary V: X=index(S)*V 1) Among them, index(S) represents the process of generating X by indexing the pre-training corpus; V stands for vocabulary.

4. The text classification method based on dual-channel semantic enhancement and convolutional neural network according to claim 1, characterized in that: Using Conv1D for feature extraction, obtaining the original semantic information C includes the following steps: For the i-th word in text S, the vector representation is x i , the text embedding vector X is subjected to sliding convolution on X under a one-dimensional convolution kernel w with a convolution kernel size of z, and the feature h is generated by equation (2) i : [x i :x i+z-1 ]=[x i ,x i+1 ,...,x i+z-1 ]2) h i =σ(w1·[x i :x i+z-1 ]+b1)3) where [x i :x i+z-1 ] represents the sliding window range of the convolution kernel; x i Represents the word vector of the i-th word in a sentence; x i+z-1 Represents the word vector of the i+z-1th word in a sentence; w1 represents the feature weight; b1 represents bias; h i Represents the convolution feature information after the convolution kernel has undergone sliding convolution for the i-th time; σ is a nonlinear activation function, specifically LeakyReLU; The convolution feature h i The mapping is represented as a new convolutional feature H: H=[h1,h2,...,h l-z+1 ]5) Thus, the extracted convolution feature M is obtained: M=[H1,H2,...,H m ]6) Where H1 represents the convolution feature generated by the first convolution channel; H m Represents the convolution feature generated by the mth convolution channel; m is the total number of convolution channels; The average pooling operation is used on the convolution feature map M to obtain the original semantic information of the one-dimensional convolution channel. The pooling output C of the convolution feature map M is expressed as: C = avgpool(M)7) Among them, C represents the original semantic information; avgpool() means average pooling.

5. A text classification method based on dual-channel semantic enhancement and convolutional neural network according to claim 4, characterized in that: The convolution kernel size z≤4.

6. A text classification method based on dual-channel semantic enhancement and convolutional neural network according to claim 5, characterized in that: The convolution kernel size is set to z = {2, 3, 4}.

7. The text classification method based on dual-channel semantic enhancement and convolutional neural network according to claim 1, characterized in that: AtrousConv1D is used for feature extraction to obtain global text information A: A=maxpool{LeakyReLU(Z)}9) Among them, LeakyReLU() represents a nonlinear activation function; maxpool() represents the maximum pooling operation; Z = AtrConv1D(X) Where Z = [z1,z2,...,z K ], in the filter w of length K k After the dilated convolution, the global output z is generated i : Where x[i+p·k] represents the word vector after AtrousConv1D expansion; p represents the stride of the dilated convolution on the input vector x[i+p·k], p∈P, P={p1,p2,p3}, and p1, p2, and p3 represent the three semantic receptive field expansion rates.

8. The text classification method based on dual-channel semantic enhancement and convolutional neural network according to claim 1, characterized in that: Step S3 includes the following steps: S3-1, use weighted average attention to generate attention score C for the generated original semantic information C and global text information A score and A score : Among them C score and A score The weighted attention scores obtained for Conv1D and AtrousConv1D convolutions respectively; softmax(v c T u c ) i v c T with u c The matrix product probability mapping of ; softmax(v a T u a ) i v a T with u a The matrix product probability mapping of ; T is the transpose symbol; C i is the i-th convolution feature map of C; A i is the i-th convolution feature map of A; l is the sentence length; z is the convolution kernel size; v c and v a The eigenvectors H and H are a , H is the mapping feature of the text matrix X after Conv1D convolution; H a It is the mapping feature of the text matrix X after AtrousConv1D convolution; u c and u a Then it is the high-dimensional feature score after linear change, as shown in formula (12): Among them, M a They are the convolution features after the one-dimensional convolution channel and the convolution features after the AtrousConv1D convolution channel respectively; Tanh is the activation function to calculate the attention score; w1 and w2 represent the feature weights of the convolution channels of Conv1D and AtrousConv1D respectively; b1 and b2 represent the bias of the convolution channels of Conv1D and AtrousConv1D respectively; S3-2, using the semantic enhancement module, semantically enhances the text matrix X and activates the generated y k , expressed as: y k =σ1(conv(X;w k ,b k ))13) Where σ1 represents the ReLU activation function; y k Indicates enhanced features; conv() represents the convolution operation; w k Represents the feature weight of the convolution channel; b k Represents the bias of the convolution channel; S3-3, y k After maxpool1D, the features are divided to obtain enhanced features. and Then and Concatenate with the weighted attention features extracted by dual-channel convolution: Where C and A are the original semantic information and global text information obtained by the convolution of Conv1D and AtrousConv1D respectively; C score and A score The weighted attention scores obtained by Conv1D and AtrousConv1D convolutions respectively; y k c Representation and high-dimensional volume feature map C·C score Semantically enhanced features generated by splicing; y k a Representation and high-dimensional volume feature map A·A score Semantically enhanced features generated by splicing; Represents a splicing operation; S3-4, the concatenated feature map H cla It is mapped to the probability distribution of labels through the Linear fully connected layer and the Sofrmax layer.

9. The text classification method based on dual-channel semantic enhancement and convolutional neural network according to claim 1, characterized in that: The cross entropy loss function is used to measure the difference between the predicted probability distribution and the actual label. The cross entropy loss function is expressed as; in represents the loss used for classification; n is the number of test samples; y is the true value label; is the predicted label.

Images