Deep learning based multi-label text classification method
By combining Bi-LSTM and graph convolutional networks, a deep learning approach is adopted to address the problem of neglecting label semantics and relationships in traditional multi-label text classification, thereby improving the classification performance of the model.
Patent Information
- Application Number
- CN202310792349.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- Filing Date
- 2023-06-30
- Publication Date
- 2025-12-16
- Estimated Expiration
- 2043-06-30
AI Technical Summary
Traditional multi-label text classification methods ignore the semantics of the labels themselves and the relationships between the labels, leading to a decline in model classification performance.
We employ a deep learning-based approach, using a Bi-LSTM model to learn word vectors from document text, combining this with a structured self-attention mechanism to extract high-level semantics, optimizing label features through a graph convolutional network, utilizing a cross-attention mechanism for information interaction, and finally obtaining the text representation through weighted calculation.
The performance of multi-label text classification models has been improved by focusing on the relationships between labels and the overall semantics of the text to obtain more comprehensive text representation features, thereby enhancing the model's classification ability.
Smart Images
Figure CN116821340B_ABST
Abstract
Description
TECHNICAL FIELD
[0001] The application relates to the technical field of multi-label text classification, in particular to a multi-label text classification method based on deep learning. BACKGROUND
[0002] Text data usually has non-structural and redundant characteristics, and in a multi-label text data set, each document can have a large number of corresponding labels, which determines that in the multi-label text classification problem, there are usually three research and optimization directions of label information utilization, processing of potential relationships between labels and text feature representation.
[0003] At present, the mainstream multi-label text classification benchmark model usually focuses on the text feature representation problem, for example, a convolutional neural network can effectively extract local features of text, and a recurrent neural network can learn long-distance dependency relationships in text. However, the traditional multi-label text classification method ignores the semantics of the label itself and the relationship between the labels, thereby reducing the performance of the model classification. SUMMARY
[0004] The technical problem of the application is that the traditional multi-label text classification method ignores the semantics of the label itself and the relationship between the labels, thereby reducing the performance of the model classification.
[0005] The basic scheme provided by the application is a multi-label text classification method based on deep learning, which comprises the following steps:
[0006] S1: performing word vector training on a corpus of a training set;
[0007] S2: learning word vectors of each word in a document text based on a Bi-LSTM model, extracting high-level semantics in the text through a structured self-attention mechanism, and obtaining document text representation;
[0008] S3: optimizing label features through a graph convolution network;
[0009] S4: performing information interaction on text semantic features output by the Bi-LSTM model and label features output by the graph convolution network based on a cross-attention mechanism, and obtaining label text representation;
[0010] S5: obtaining final text representation through weighted calculation;
[0011] S6: inputting the final text representation into a fully connected layer after pooling, performing linear mapping of a label dimension vector, and calculating a prediction probability of the label.
[0012] Further, the S1 comprises the following steps:
[0013] S101: obtaining an original training set, and preprocessing the original training set;
[0014] S102: word vector training is performed on the corpus of the training set based on the Glove model.
[0015] Further, after word vector training, the document text vector W and the label text vector L are obtained:
[0016] W=(w1, w2,..., wn) n )
[0017] L=(l1, l2,..., lk) k )
[0018] where n is a self-defined text length, W is a fixed number of word vectors, k is the number of labels, respectively, are the text word vector representation and the label word vector representation.
[0019] Further, in the S2, the steps include:
[0020] S201: forward and backward semantics of the document text are obtained through the Bi-LSTM network, and each hidden state expression form is:
[0021]
[0022]
[0023] where w t is the word vector representation of the word, h t is the hidden state vector at the corresponding moment, each hidden state vector forms a basic text representation with text forward semantic features and backward semantic features, and the text feature representation after the Bi-LSTM network is denoted as H=(h1, h2,..., hn) n ;
[0024] S202: high-level semantics in the text are extracted through the structured self-attention mechanism, and the document text representation is obtained.
[0025] Further, the calculation process of the structured attention mechanism in the S202 includes the steps of:
[0026] S2021: determine the attention weight corresponding to each hidden state in the Bi-LSTM:
[0027] a=softmax(w s2 tanh(W s1 H T ))
[0028] where a is the weight matrix, W s1 is a matrix with dimension d aThe weight matrix of w s2 is a parameter vector with dimension d a .
[0029] S2022: Expand the dimension of w s2 to a weight matrix with dimension c x n:
[0030] A s = softmax(W s2 tanh(W s1 H T ))
[0031] That is, A s = (a1, a2,..., a c ), and the final text vector is represented as M W = A s H.
[0032] Further, the S3 includes the steps of:
[0033] S301: Construct a label graph based on prior knowledge of label co-occurrence:
[0034]
[0035] In the formula, M co is a k x k label co-occurrence matrix, is the adjacency matrix of the label, and F is the frequency vector of the label. Let F = (f1, f2,..., f k ), then f i is the frequency of the i-th label, and each element a i，j in A is the probability of the i-th label existing when the j-th label exists.
[0036] S302: Optimize label features through a graph convolution network.
[0037] Further, in the graph convolution network in the S3, the input of each graph convolution layer can be the input of the previous graph convolution layer:
[0038]
[0039] In the formula, l is the l-th layer of the graph convolution network, σ is the ReLU activation function, is a trainable transformation matrix, in the first layer of the graph convolution network, L (l) is the original embedding vector, is the normalized adjacency matrix, and the normalization method is:
[0040]
[0041]
[0042] In the formula, is a degree matrix, and after a plurality of graph convolution layers, a label feature M with label relationship information is obtained G .
[0043] Further, the S4 comprises the steps of:
[0044] S401: multi-head attention is used to map input data into a plurality of subspaces respectively;
[0045] S402: an independent attention mechanism calculation is performed on each subspace;
[0046] S403: the attention results of different subspaces are combined and linearly transformed to obtain a label text representation.
[0047] Further, the S5 comprises the steps of:
[0048] S501: feature fusion is performed based on a gating mechanism to obtain a text feature representation;
[0049] S502: the weight t of the text feature is calculated, and a weighted calculation is performed to obtain a final text representation.
[0050] Further, the S6 comprises the steps of:
[0051] S601: the vector dimension is mapped to the label number dimension through a full connection layer;
[0052] S602: the probability of each label is calculated through a sigmoid activation function:
[0053] y=sigmoid(WM+b) F
[0054] In the formula, M F is a final text representation.
[0055] The principle and advantages of the present application are that: from the perspective of label text, in a multi-label text classification task, some labels often appear in the same document at the same time, and have high co-occurrence. Therefore, the present scheme represents the relationship between labels based on a graph structure, extracts the co-occurrence matrix of labels in the label text of the training set, processes the co-occurrence frequency after obtaining the label co-occurrence matrix, constructs the adjacency graph of the labels, and introduces a graph convolution network on this basis, and by implicitly assigning different weights to the nodes and adjacent nodes in the graph structure based on a convolution kernel, the features between the nodes are fused.
[0056] From the perspective of document text representation, the scheme adopts a Bi-LSTM network to train text features, and adds a structured self-attention mechanism on this basis, sets an attention weight for each hidden state of the Bi-LSTM network during its cyclic iteration, forms a learnable attention mechanism, and pays more attention to the relationship between words and the whole text. Avoid the problem that in the case of using only LSTM, each hidden state can only provide short-term context information around each word, resulting in the loss of part of the semantics. Therefore, the scheme adds a structured self-attention mechanism in the process of training document text representation, which can extract high-level semantics with long-term dependencies between words in the text.
[0057] After obtaining the document text representation and the label text representation fused with the label correlation, since the two kinds of texts have different focuses, the document text representation based on the self-attention mechanism focuses on the overall semantics of the text, and the label text representation based on the graph convolution network fused with the label semantics focuses more on the relationship between the labels and the labels themselves. The scheme performs feature fusion based on the gating mechanism, and performs weighted calculation on the two kinds of text representations to obtain more comprehensive text representation features and improve the classification ability of the model. BRIEF DESCRIPTION OF DRAWINGS
[0058] Figure 1 The step flowchart of the multi-label text classification method based on deep learning of the application.
[0059] Figure 2 The structural diagram of the multi-label text classification model based on graph convolution and cross-attention in embodiment one.
[0060] Figure 3 The cross-attention mechanism diagram of the multi-label text classification method based on deep learning of the application DETAILED DESCRIPTION
[0061] The following will be further described in detail through specific embodiments:
[0062] The specific implementation process is as follows:
[0063] Embodiment one
[0064] Embodiment one is basically as shown in the accompanying drawings Figure 1 The multi-label text classification method based on deep learning comprises the following steps:
[0065] S1: performing word vector training on the corpus of the training set;
[0066] S2: learning the word vector of each word in the document text based on the Bi-LSTM model, extracting high-level semantics in the text through the structured self-attention mechanism, and obtaining the document text representation;
[0067] S3: optimizing label features through a graph convolution network;
[0068] S4: information interaction between text semantic features output by the Bi-LSTM model and label features output by the graph convolution network based on a cross-attention mechanism, to obtain label text representation;
[0069] S5: obtaining final text representation through weighted calculation;
[0070] S6: performing linear mapping of label dimension vectors by inputting the final text representation into a fully connected layer after pooling, to calculate prediction probability of the label.
[0071] Specifically, a multi-label text classification model based on graph convolution and cross-attention is constructed in the embodiment, and the model includes a word embedding module, a text representation module based on structured self-attention, a label feature optimization module based on graph convolution, a cross-attention module, a semantic fusion module, and a classification module, and the overall structure is as shown in Figure 2 The model can be divided into the following six modules:
[0072] Word embedding module: in the model, label text information is needed, but such information does not exist in most conventional pre-training corpus. Therefore, the module uses 300-dimensional pre-trained Glove word vectors for text word embedding, and performs trainable random initialization for label embedding. The document text D={word1, word2,..., word n} and label text Label={label1, label2,..., label n} are represented by vectors.
[0073] Text representation module based on structured self-attention: the module learns each word embedding in the document text based on a Bi-LSTM model, extracts high-level semantics in the text through a structured self-attention mechanism, and obtains document text representation.
[0074] Label feature optimization module based on graph convolution: based on a label co-occurrence matrix and label text vectors, the module implicitly learns the relationship between labels through a graph convolution network, to optimize label features.
[0075] Cross-attention module: based on a cross-attention mechanism, the module performs information interaction between text semantic features output by the Bi-LSTM and label features output by the GCN, to obtain label text representation.
[0076] The semantic fusion module and the classification module: after obtaining the text representation focusing on the document text and the label text representation focusing on the label relationship and the label semantics by the text representation module based on the structured self-attention, the label feature optimization module based on the graph convolution and the cross-attention module, adaptive weights are established based on the gating mechanism, and the final text representation is obtained by weighted calculation of the two features.
[0077] The classification module: after the obtained final text representation is transmitted into the full connection layer after pooling, linear mapping of the label dimension vector is performed, the prediction probability of the label is calculated through the sigmoid function, and the multi-label classification of the text is realized.
[0078] Specifically, S1 includes the following steps:
[0079] S101: obtaining an original training set and pre-processing the original training set;
[0080] S102: performing word vector training on the corpus of the training set based on the Glove model.
[0081] Since the text data in the real world is composed of words and sentences, it belongs to natural language. However, computers cannot directly understand and process natural language, so we need to pre-process the original text data and represent it in the form of vectors that computers can understand.
[0082] Therefore, the embodiment first performs word segmentation, truncation and unknown word padding on the text through the word embedding module. Since the original text data itself is mostly noisy, for example, some stop words: the, have, etc., which do not have important meaning, these relatively unimportant text information is first filtered out in the pre-processing process. At the same time, the number of words in each document is uncertain, in order to fix the dimension of the document text vector unchanged, a dimension value needs to be fixed, and it is considered that there are only this number of words in each document. If the number of words in the document exceeds the limited value, it is truncated, and if the number is less than the limited value, it is filled with <pad>. If a word not in the dictionary is encountered, it is filled with <unk>. The role of the dictionary is to map the word to the word id based on the corpus, because only numbers can be input into the computer, and each number represents the corresponding word.
[0083] After obtaining the preprocessed text, we train the word vectors of the training corpus based on the Glove model. In the conventional scenario, pre-trained Glove word vectors are generally considered, such as the Glove word vectors of various dimensions based on English Wikipedia pre-trained on the Glove website. However, our model needs to use label word vectors for label semantic fusion, and the label words of several real-world datasets such as AAPD, Reuters21578, RCV1-V2, etc. are usually not in the published pre-trained Glove model, so this module uses 300-dimensional Glove pre-trained word vectors for text words, randomly initializes the label embedding, and sets it to a trainable mode, which is continuously optimized during the training process.
[0084] The trained document text vector W and label text vector L are:
[0085] W = (w1, w2,..., wn) n )
[0086] L = (l1, l2,..., lk) k )
[0087] where n is the self-defined text length (Sequence Length), W is a fixed number of word vectors, and k is the number of labels because there are k words in the label text, respectively, the text word vector representation and the label word vector representation, in order to facilitate the subsequent module processing of the label word vector and the document word vector, this module sets w i and l i to the same dimension.
[0088] Further, S2 includes the following steps:
[0089] S201: Obtain the forward and backward semantics of the document text through the Bi-LSTM network, and each hidden state expression form is:
[0090]
[0091]
[0092] where w t is the word vector representation of the word, h t is the hidden state vector at the corresponding moment, and each hidden state vector forms a basic text representation with text forward semantic features and backward semantic features. The text feature representation after the Bi-LSTM network is denoted as
[0093] S202: Extract high-level semantics in the text through a structured self-attention mechanism to obtain a document text representation.
[0094] This step takes the text representation with context semantics extracted by LSTM as the basis through the structured self-attention-based text representation module for constructing the document text representation and the label text representation in the subsequent components. Although LSTM has the ability to model the extracted context semantics within a certain distance, it is often difficult to extract the global features of the text for too long sequences. Therefore, this step considers using a structured attention mechanism for optimization to perceive long-distance dependency information of the text sequence. This attention mechanism is initially applied to sentence-level embedding representation. This attention does not perform dot product operation, but similarity measurement through a layer of feedforward neural network, which improves the operation speed of the self-attention layer to some extent. In this embodiment, the attention mechanism is applied to the optimization of the text representation, enhancing the long-distance modeling capability of the BiLSTM network.
[0095] Specifically, in the calculation process of the structured self-attention mechanism, it is necessary to first determine the attention weight corresponding to each hidden state in the Bi-LSTM, which is actually to find the linear combination parameters of the set of hidden state vectors, i.e., the weight matrix a, and the calculation formula is:
[0096] a = sof max(w s2 tanh(W s1 H T ) )
[0097] In the formula, a is the weight matrix, W s1 is a weight matrix with a dimension of d a xd, and w s2 is a parameter vector with a dimension of d a . The softmax activation function is set here to ensure that the attention weight sum is 1. Since this mechanism often causes the text representation attention to focus on a specific part, in a longer sequence, a document usually has multiple keywords, in order to extract these features, multiple self-attention processing needs to be performed on the text vector, i.e., the dimension of w s2 is expanded, and global features are extracted in multiple dimensions, converting the weight matrix into a dimension of c x n:
[0098] A s = sof max(W s2 tanh(W s1 H T ) )
[0099] That is, A s = (a1, a2,..., a c ), which is essentially composed of multiple attention weights, in order to focus on different parts of the document, the final text vector Representation M W =A s H.
[0100] Furthermore, S3 includes the following steps:
[0101] S301: Constructing a label graph based on prior knowledge of label co-occurrence;
[0102] S302: Optimize label features using graph convolutional networks.
[0103] This step primarily addresses the issue of traditional models failing to incorporate label relationships by employing a graph convolution-based label feature optimization module. In the real world, labels in multi-label text datasets typically exhibit hierarchical relationships or correlations. For instance, in the Arxiv Academic Literature Dataset (AAPD), the labels cs.it and cs.ai frequently appear together. This module constructs a label relationship graph based on label co-occurrence information as prior knowledge. It then enhances the label vectors using a graph convolutional network, aggregating co-occurring label features strongly correlated with each label. In graph convolutional network models, adjacency relationships between nodes are required for feature fusion of nodes and their neighbors. However, in multi-label text datasets, labels often lack explicit relationships. Therefore, this module establishes label adjacency relationships based on a co-occurrence matrix.
[0104] Given that the number of tags is k, then the tag co-occurrence matrix M co Given a k×k tag co-occurrence matrix, the element in the i-th row and j-th column is represented as m. ij This element represents how many samples contain both label i and label j, i.e., the number of times they co-occur in the corpus. However, the label co-occurrence matrix is not suitable for directly measuring the adjacency relationship between labels. Therefore, this module constructs a label graph based on prior knowledge of label co-occurrence. The calculation process is shown below:
[0105]
[0106] In the formula, Let F be the adjacency matrix of the tags, and F be the frequency vector of the tags. Let F = (f1, f2, ..., f...). k ), then f i Let a be the frequency of the i-th label, and let a be the frequency of each element a in A. i，j This represents the probability that the j-th label also exists.
[0107] Graph convolutional networks are often used for node feature aggregation, while in this module, the label is actually considered as a node, and the label feature information is aggregated according to the label adjacency matrix and the graph convolutional network. Through the GCN and the label co-occurrence information, the model further learns the deep relationship between the labels. The graph convolutional network is a model that works based on the graph structure, which enhances the representation of node features by propagating information between adjacent nodes. The graph convolutional layer actually still performs convolution operation, but the window for convolution is slightly different.
[0108] In the multi-layer graph convolutional network, the input of each graph convolutional layer can be the input of the previous graph convolutional layer:
[0109]
[0110] In the formula, l is the l-th layer of the graph convolutional network, σ is the ReLU activation function, is a trainable transformation matrix, in the first layer of the graph convolutional network, L (l) is the original embedding vector, is the normalized adjacency matrix, and the normalization method is:
[0111]
[0112]
[0113] In the formula, is the degree matrix, after passing through multiple graph convolutional layers, the neighborhood of the adjacent node feature of each node can be expanded, and the features of more nodes in the adjacent network are fused, so as to obtain the label feature M with label relationship information G After introducing the label co-occurrence relationship information into the label feature, the cross attention can be used to link the label feature and the text feature, and obtain the text representation feature with label semantics.
[0114] Further, S4 includes the steps of:
[0115] S401: The multi-head attention maps the input data into multiple subspaces respectively;
[0116] S402: An independent attention mechanism calculation is performed on each subspace;
[0117] S403: The attention results of different subspaces are combined and linearly transformed to obtain the label text representation.
[0118] In this embodiment, information interaction is carried out through the cross attention module, the multi-head attention mechanism can learn different dimensional or different level representations, and the expression ability and generalization ability of the model to the input are improved. At the same time, the multi-head attention can also improve the robustness and anti-noise ability of the model, the number of multi-head cross attention heads in this module is set to 6, and the calculation process is as shown in Figure 3
[0119] The multi-head attention maps the input data into multiple subspaces respectively, then performs independent attention mechanism calculation on each subspace, and finally splices and linearly transforms the attention results of different subspaces to obtain the final output, and the calculation formula is as follows:
[0120] MultiHead(Q, K, V) = Concat(head1,..., head h )W o
[0121] head i = Attention(QW i Q , KW i K , VW i V )
[0122] In the formula, W o , W i Q , W i K , W i V are parameter matrices, that is, full connection layers in the model, used for mapping different dimensional spaces. The above is actually mapping M a and H to multiple subspaces, where K and V are different linear space mappings of H, and the multi-head cross attention output is as shown below:
[0123] M′ G = MultiHead(M a , H, H)
[0124] This module uses the Norm structure and the feedforward neural network structure, increases the depth and width of the nonlinear mapping to enhance the expression ability and adaptability of the model, and increases the normalization processing in the attention mechanism to help the network better process the noise and redundant information in the input data, and improve the robustness and generalization ability of the model. The calculation process is as shown below:
[0125] M L = LayerNorm(M′ G + FFN(M′ G ))
[0126] M L is the text representation feature with label semantics. The feedforward neural network is composed of two fully connected layers and a nonlinear activation function, and the calculation process is as follows:
[0127] FFN(M′ G )=max(0,M′ G W1+b1)W2+b2
[0128] wherein, are trainable parameters of the fully connected layer.
[0129] Further, S5 includes the following steps:
[0130] S501: Feature fusion based on a gating mechanism to obtain a text feature representation;
[0131] S502: Calculate the weight t of the text feature and perform weighted calculation to obtain the final text representation.
[0132] After obtaining the text representation of the interaction between the label and the document, the label feature M L captures the important content of the label for the text, and the text feature M W is mainly used for feature extraction of the document text, so this step mainly performs feature fusion based on a gating mechanism through a semantic fusion module to extract and fuse relatively important parts in different text representations to obtain a more comprehensive text representation feature. The specific steps are as follows:
[0133] t=σ(tanh(M W W2+M L W3)W1)
[0134] wherein, are parameter matrices for mapping the text representation to the same dimension. The weight t of the text feature is calculated, and based on this, weighted calculation is performed as follows:
[0135]
[0136] M F is the final text representation, which is the input of the classification module.
[0137] Further, S6 includes the following steps:
[0138] S601: Map the vector dimension to the label number dimension through a fully connected layer;
[0139] S602: Calculate the probability of each label through a sigmoid activation function.
[0140] After the classification module obtains the final text representation vector, it maps the vector dimension to the label number dimension through a fully connected layer, and finally calculates the probability of each label through the sigmoid activation function:
[0141] y = sigmoid(WM F +b)
[0142] The classification module in this embodiment uses a binary cross-entropy loss function (BCELoss) for calculation. This loss function is one of the commonly used loss functions in the field of text classification, and the calculation process is as follows:
[0143]
[0144] In the formula, N is the sample input of the document, is the true value of the jth label corresponding to the ith sample, y ij is the prediction probability of the jth label in the ith sample by the model.
[0145] In addition, two publicly available data sets are used to test the multi-label text classification model based on graph convolution and cross attention proposed in this scheme, aiming to verify the universality and reliability of the model on different types of data sets.
[0146] In summary, from the perspective of label text, in the multi-label text classification task, some labels often appear in the same document at the same time, with high co-occurrence. Therefore, this scheme represents the relationship between labels based on graph structure, extracts the co-occurrence matrix of labels in the label text of the training set. After obtaining the label co-occurrence matrix, the co-occurrence frequency is processed, the adjacency graph of the label is constructed, and the graph convolution network is introduced on this basis, which implicitly assigns different weights to the nodes and adjacent nodes in the graph structure based on the convolution kernel to fuse the features between the nodes.
[0147] From the perspective of text representation, this scheme uses a Bi-LSTM network to train text features, and adds a structured self-attention mechanism on this basis, which sets an attention weight for each hidden state of the Bi-LSTM network during its loop iteration, forming a learnable attention mechanism that pays more attention to the relationship between words and the whole text. Avoid the problem of missing part of the semantics caused by only using LSTM, which can only provide short-term context information around each word for each hidden state. Therefore, the structured self-attention mechanism added in the process of training the document text representation can extract high-level semantics with long-term dependencies between words in the text.
[0148] After obtaining the document text representation and the label text representation with the fusion of label correlation, since the two kinds of texts have different focuses, the document text representation based on the self-attention mechanism focuses on the overall semantics of the text, and the label text representation based on the graph convolution network fusing the label semantics focuses more on the label itself and the relationship between the labels. The scheme performs feature fusion based on a gating mechanism, performs weighted calculation on the two kinds of text representations, obtains more comprehensive text representation features, and improves the classification ability of the model.
[0149] The above is only an embodiment of the present application, and the common knowledge of specific structures and characteristics in the scheme is not described in detail here. The ordinary skilled person in the art knows all the ordinary technical knowledge in the field of the present application before the filing date or the priority date, can know all the prior art in the field, and has the ability to apply conventional experimental means before that date. The ordinary skilled person in the art can improve and implement the present scheme based on their own ability under the guidance of the present application. Some typical known structures or known methods should not be an obstacle for the ordinary skilled person in the art to implement the present application. It should be noted that, for those skilled in the art, without departing from the structure of the present application, a number of modifications and improvements can be made, which should also be considered as the protection scope of the present application. These will not affect the effect and practicality of the present application. The protection scope of the present application should be subject to the content of its claims, and the specific implementation mode and the like in the specification can be used to explain the content of the claims.
Claims
1. A multi-label text classification method based on deep learning, characterized by, The method comprises the steps of: S1: word vector training is performed on a corpus of a training set; S2: word vectors of each word in the document text are learned based on a Bi-LSTM model, high-level semantics in the text are extracted through a structured self-attention mechanism, and document text representation is obtained; S3: label features are optimized through a graph convolution network; S4: information interaction is performed on text semantic features output by the Bi-LSTM model and label features output by the graph convolution network based on a cross-attention mechanism, and label text representation is obtained; S5: final text representation is obtained through weighted calculation; S6: the final text representation is input into a fully connected layer after being pooled, linear mapping of label dimension vectors is performed, and prediction probability of the label is calculated; The calculation process of the structured self-attention mechanism comprises the steps of: S2021: attention weights corresponding to each hidden state in the Bi-LSTM are determined: wherein is a weight matrix, is a weight matrix of dimension is a weight matrix of dimension is a parameter vector of dimension is a parameter vector of dimension S2022: expand the dimension of the to a weight matrix of the dimension. That is , final text vector , representation ; The S3 comprises the steps of: S301: a label graph is constructed based on prior knowledge of label co-occurrence: In the formula, For one The label co-occurrence matrix, Let the adjacency matrix of the labels be... Let be the frequency vector of the label. ,but For the first The frequency of each label, Each element That is, the existence of the first When the first tag exists, the second tag also exists. The probability of each label; S302: label features are optimized through the graph convolution network. 2.The deep learning-based multi-label text classification method of claim 1, wherein, The S1 comprises the steps of: S101: an original training set is acquired, and the original training set is preprocessed; S102: word vector training is performed on a corpus of the training set based on a Glove model. 3.The deep learning-based multi-label text classification method of claim 2, wherein: Document text vectors W and label text vectors L are obtained after word vector training: In the formula, n is the length of the defined text, W is a fixed number of word vectors, k is the number of labels, Respectively, the text word vector representation and the label word vector representation. 4.The deep learning-based multi-label text classification method of claim 3, wherein, In the S2, the steps comprise: S201: forward semantics and backward semantics of the document text are obtained through the Bi-LSTM network, and each hidden state is expressed in the form of: In the formula, is a word vector representation of a word, is a hidden state vector of a corresponding moment, each hidden state vector forms a basic text representation with forward and backward semantic features of the text, and the text feature representation after the Bi-LSTM network is denoted as , ; S202: high-level semantics in the text are extracted through the structured self-attention mechanism, and document text representation is obtained. 5.The deep learning-based multi-label text classification method of claim 1, wherein: In the S302, the input of each graph convolution layer of the graph convolution network can be the input of the previous graph convolution layer: In the formula, is the first layer graph convolution network, is an activation function, is a trainable transformation matrix, in the first layer graph convolution network, is the original embedding vector, is the normalized adjacency matrix, and the normalization method is: In the formula, is a degree matrix, and after a plurality of graph convolution layers, a label feature with label relationship information is obtained .
6. The deep learning-based multi-label text classification method of claim 5, wherein The S4 comprises the steps of: S401: input data is respectively mapped into multiple subspaces through multi-head attention; S402: independent attention mechanism calculation is performed on each sub-space; S403: attention results of different subspaces are combined and linearly transformed to obtain label text representation.
7. The deep learning-based multi-label text classification method of claim 6, wherein The S5 comprises the steps of: S501: feature fusion is performed based on a gating mechanism to obtain text feature representation; S502: the weight t of the text feature is calculated, and final text representation is obtained through weighted calculation. 8.The deep learning-based multi-label text classification method of claim 7, wherein The S6 comprises the steps of: S601: the vector dimension is mapped to the label number dimension through the fully connected layer; S602: the probability of each label is calculated through a sigmoid activation function: In the formulae, is the final text representation.