Multi-label text classification method and device based on graph neural network

Abstract

The invention provides a multi-label text classification method and device based on a graph neural network. According to the method, the relationship between the tags is dynamically adjusted through the dynamic graph convolutional network, so that the tag relationship graph can dynamically change according to contexts of different texts, and the change and interaction between the tags in different scenes can be better captured. When the input data has different domain backgrounds or context changes, the content of the corresponding target text feature part can be combined to effectively capture different associations between the tags in a specific context, so that the classification and adaptive capacity of diversified tags in a complex scene is effectively improved.

Description

TECHNICAL FIELD

[0001] The present application relates to the technical field of label classification, and particularly relates to a multi-label text classification method and device based on a graph neural network. BACKGROUND

[0002] Since the graph convolutional network (GCN) was proposed in 2016, the graph neural network has become a hot research topic in the field of deep learning. Traditional neural networks such as convolutional neural networks usually rely on regular grid structures, and due to the non-Euclidean nature and irregularity of graph structures, traditional methods are difficult to apply to the processing of graph data.

[0003] Graph-based methods such as TextGCN and HAN utilize static co-occurrence matrices or knowledge graphs to construct label relationships and model the relationships between labels with graph convolution. The advantage of this method is that it can effectively mine the potential relevance between labels, but since the constructed graph structure is static, the establishment of label relationships is often too simple and rough, lacking deep semantic understanding. This coarse-grained modeling makes the model unable to capture more subtle and variable semantic associations when processing complex texts. Therefore, as the diversity of input texts and the change of context, the static graph cannot be dynamically adjusted according to the semantics of specific texts. When the input data has different domain backgrounds or context changes, the static graph often cannot effectively capture the different associations between labels in a specific context. SUMMARY

[0004] To solve the technical problems in the prior art that the graph structure is static, the establishment of the label relationship is often too simple and rough, lacks deep semantic understanding, and such coarse-grained modeling makes the model unable to capture more subtle and variable semantic associations when processing complex texts, and therefore the static graph cannot dynamically adjust according to the semantics of the specific text, and when the input data has different domain backgrounds or context changes, the static graph often cannot effectively capture the different associations between labels in a specific context, the application provides a multi-label text classification method and device based on a graph neural network, different types of features are extracted from the text to be classified based on different extraction rules to obtain target text features, and after a label co-occurrence matrix is constructed, a dynamic weight matrix is introduced to obtain dynamically changing label features; and the target text features and the label features are fused based on a cross-attention mechanism to obtain dynamic fusion features, and finally the fusion features are weighted to obtain independent prediction probabilities of each label. Due to the introduction of the dynamic weight matrix, the dynamically changing label features are obtained, the establishment of the label relationship is more accurate and practical, and therefore the graph structure constructed is also dynamic, and the final label prediction is based on the fusion features, the establishment of the fusion features combines the target text features and the label features, and the target text features are extracted from the text to be classified according to the preset multi-level extraction rules, so that the corresponding fusion features obtained finally dynamically change and adjust with the diversity of the input text and the change of the context. When the input data has different domain backgrounds or context changes, the corresponding target text feature part can also be combined to effectively capture the different associations between labels in a specific context. Therefore, the adaptability of the diversified labels in a complex scene is effectively improved. The application solves the technical problems in the prior art that the graph structure is static, the establishment of the label relationship is often too simple and rough, and lacks deep semantic understanding, and such coarse-grained modeling makes the model unable to capture more subtle and variable semantic associations when processing complex texts, and therefore the static graph cannot dynamically adjust according to the semantics of the specific text, and when the input data has different domain backgrounds or context changes, the static graph often cannot effectively capture the different associations between labels in a specific context.

[0005] The multi-label classification model fuses dynamic graph convolution and multi-level feature fusion. The core idea of the model is to realize the collaborative optimization of text semantics and label relationship through a dynamic parameterization mechanism and multi-level text feature coding. Specifically, the model first dynamically adjusts the relationship between labels through a dynamic graph convolution network, so that the label relationship graph can dynamically change according to the context of different texts, thereby better capturing the changes and interactions between labels in different scenarios. In addition, by hybridly using Bi-LSTM and CNN and combining a self-attention mechanism, the deep semantic representation of the text is captured based on different fine-grained feature levels, forming hierarchical semantic representations, and the completeness of feature expression is significantly improved.

[0006] According to an embodiment of the present application, a multi-label text classification method based on a graph neural network comprises: performing feature extraction on a text to be classified based on preset multi-level extraction rules to obtain target text features; constructing a label co-occurrence matrix based on the text to be classified, introducing a dynamic weight matrix, and performing convolution to obtain label features; fusing the target text features and the label features based on a cross-attention mechanism to obtain fused features; and performing weighted processing on the fused features to obtain independent prediction probabilities of each label.

[0007] In some embodiments, the step of performing feature extraction on a text to be classified based on preset multi-level extraction rules to obtain target text features further comprises: using different encoders to respectively read out global sequence level features and local phrase level features from the text to be classified; splicing the global sequence level features and the local phrase level features to obtain initial text features; and performing feature extraction on the initial text features based on a self-attention mechanism to obtain target text features.

[0008] In some embodiments, the encoders comprise a bidirectional encoder and a convolutional encoder; and the step of using different encoders to respectively read out global sequence level features and local phrase level features from the text to be classified further comprises: using a bidirectional encoder to read the text to be classified in a forward and backward direction to obtain global sequence level features of the text to be classified; and using a convolutional encoder to read the text to be classified based on multi-layer convolution to obtain local phrase level features of the text to be classified.

[0009] In some embodiments, the step of constructing a label co-occurrence matrix based on the text to be classified, introducing a dynamic weight matrix, and performing convolution to obtain label features further comprises: counting the frequency of occurrence of each label in the text to be classified, and constructing a label co-occurrence matrix according to the frequency of occurrence of each label; introducing a dynamic matrix in the label co-occurrence matrix, and dynamically adjusting based on a dynamic gating mechanism to perform convolution to obtain label features.

[0010] In some embodiments, the step of fusing the target text features and the label features based on a cross-attention mechanism to obtain fused features further comprises: embedding the label features to obtain an embedding vector; performing linear projection according to the embedding vector to obtain a query vector; establishing a projection matrix of the text features, and mapping to obtain a key vector and a value vector according to the projection matrix; obtaining a relevance matrix between the label features and the text features according to the query vector and the key vector; and performing weighted aggregation on the value vector based on the relevance matrix to obtain fused features.

[0011] In some embodiments, the step of obtaining, according to the query vector and the key vector, a correlation matrix between the label feature and the text feature further comprises: performing dot product calculation according to the query vector and the key vector; and scaling the dot product calculation by using a scaling factor to obtain the correlation matrix.

[0012] In some embodiments, the step of weighting the fusion feature to obtain an independent prediction probability of each label further comprises: weighting the fusion feature by using a gating mechanism to dynamically adjust the weight of the fusion feature; and outputting an independent prediction probability of each label based on the weight by using a preset activation function.

[0013] In some embodiments, the step of weighting the fusion feature by using a gating mechanism to dynamically adjust the weight of the fusion feature further comprises: constructing a gating weight matrix based on the gating mechanism; performing nonlinear transformation on the fusion feature according to the gating weight matrix to obtain a gating vector; and dynamically adjusting the fusion weight of the fusion feature based on the gating vector.

[0014] In some embodiments, the step of outputting an independent prediction probability of each label based on the weight by using a preset activation function further comprises: establishing a fusion formula based on the dynamically adjusted fusion weight; performing label space mapping on the fusion formula to obtain an unactivated score value of each label; and mapping the unactivated score value of each label to a preset interval by using an activation function to obtain an independent prediction probability of each label.

[0015] In another aspect, according to an embodiment of the present application, a multi-label text classification device based on a graph neural network is also provided, which is characterized in that the device comprises: a multi-level text feature encoding module, configured to perform feature extraction on a text to be classified based on a preset multi-level extraction rule to obtain a target text feature; a dynamic label embedding learning module, configured to construct a label co-occurrence matrix based on the text to be classified and introduce a dynamic weight matrix to obtain a label feature by convolution; a cross-attention module, configured to fuse the target text feature and the label feature based on a cross-attention mechanism to obtain a fusion feature; and a classification module, configured to weight the fusion feature to obtain an independent prediction probability of each label.

[0016] The technical principle of the present application is that: based on a preset multi-level extraction rule, the features of the text to be classified are extracted to obtain target text features, and the features extracted by the multi-level extraction rule are more complete and complete than the features extracted by a single rule; based on the text to be classified, a label co-occurrence matrix is constructed, and a dynamic weight matrix is introduced, and the label features are obtained by convolution, so that the correlation strength between labels can be adaptively corrected to form a more complete label feature representation; based on the cross attention mechanism, the target text features and the label features are fused to obtain the fusion features, the text features and the label features are fused to strengthen the interaction between the text and the label, and the complex relationship between the text and the label is captured; the fusion features are weighted to obtain the independent prediction probability of each label.

[0017] Compared with the prior art, the present application has the following beneficial effects: the features extracted by the multi-level extraction rule are more complete and complete than the features extracted by a single rule; at the same time, by introducing a dynamic weight matrix in the label co-occurrence matrix, the correlation strength between labels can be adaptively corrected to form a more complete label feature representation; by fusing the text features and the label features, the interaction between the text and the label is strengthened, and the complex relationship between the text and the label is captured, and finally by weighting the fusion features, the independent prediction probability of each label is obtained, which effectively improves the adaptability of diversified labels in complex scenarios. It solves the technical problems existing in the prior art that the graph structure constructed is static, the label relationship is too simple and rough, and lacks deep semantic understanding, so that the model cannot capture more subtle and variable semantic associations when processing complex texts, and the static graph cannot be dynamically adjusted according to the semantics of specific texts, nor can it effectively capture different associations between labels in a specific context. The present application introduces a dynamic weight matrix to obtain dynamically changing label features, making the establishment of label relationships more accurate and practical, enabling the label relationship graph to dynamically change according to the context of different texts, thereby better capturing the changes and interactions between labels in different scenarios. When the input data has different domain backgrounds or context changes, it can also effectively capture the different associations between labels in a specific context by combining the content of the corresponding target text feature part. Thus, the adaptability of diversified labels in complex scenarios is effectively improved. BRIEF DESCRIPTION OF DRAWINGS

[0018] Figure 1 The flowchart of a multi-label text classification method based on a graph neural network according to an embodiment of the present application.

[0019] Figure 2 The flowchart of a multi-label text classification method based on a graph neural network according to another embodiment of the present application.

[0020] Figure 3A flowchart of a multi-label text classification method based on a graph neural network according to another embodiment of the present application.

[0021] Figure 4 A flowchart of a multi-label text classification method based on a graph neural network according to another embodiment of the present application.

[0022] Figure 5 A schematic diagram of a dynamic weight gating formula according to an embodiment of the present application.

[0023] Figure 6 A schematic diagram of a total loss function formula according to an embodiment of the present application.

[0024] Figure 7 A schematic diagram of a gradient calculation formula of Lambda according to an embodiment of the present application.

[0025] Figure 8 A flowchart of a multi-label text classification method based on a graph neural network according to another embodiment of the present application.

[0026] Figure 9 A flowchart of a multi-label text classification method based on a graph neural network according to another embodiment of the present application.

[0027] Figure 10 A flowchart of a multi-label text classification method based on a graph neural network according to another embodiment of the present application.

[0028] Figure 11 A flowchart of a multi-label text classification method based on a graph neural network according to another embodiment of the present application.

[0029] Figure 12 A flowchart of a multi-label text classification method based on a graph neural network according to another embodiment of the present application.

[0030] Figure 13 A structural schematic diagram of a multi-label text classification device based on a graph neural network according to an embodiment of the present application.

[0031] Figure 14 A structural schematic diagram of a computer device according to an embodiment of the present application. DETAILED DESCRIPTION

[0032] The technical solutions in the present application will be further described below with reference to the accompanying drawings and embodiments.

[0033] The in-depth exploration of multi-label text classification methods has significant implications in both theoretical and applied practice. In terms of academic research, the study of multi-label text classification algorithms has driven innovation in NLP algorithms and provided new research directions and inspiration for other tasks in the field of artificial intelligence. In terms of practical applications, multi-label text classification can be widely applied in various fields such as news recommendation and sentiment analysis, enabling the extraction of multi-dimensional information from text data and more accurately meeting user needs.

[0034] Since the introduction of Graph Convolutional Network (GCN) in 2016, graph neural networks have become a popular research topic in the field of deep learning. Traditional neural networks such as convolutional neural networks typically rely on regular grid structures. However, due to the non-Euclidean nature and irregularity of graph structures, traditional methods are difficult to apply to the processing of graph data. GCN can capture dependencies in graph structures through the propagation of information between nodes, providing an effective framework for the processing of graph data for the first time. Through this framework, researchers can effectively extract meaningful features from graph data, greatly promoting the development of deep learning for graph data and providing new approaches for modeling and task execution of graph data. In the task of multi-label text classification, existing methods often struggle to balance the representation of text features and the modeling of label relationships, limiting the accuracy and generalization ability of the model in complex scenarios.

[0035] Traditional text encoders such as TextCNN and Bi-LSTM can capture local keywords and global sequence dependencies, respectively. However, these methods typically focus on single-level feature extraction and lack the collaborative use of multi-granularity semantics.

[0036] TextCNN, a convolutional neural network (CNN) model for text classification. Bi-LSTM, Bidirectional Long Short-Term Memory, is a recurrent neural network (RNN) architecture developed based on the traditional long short-term memory network (LSTM).

[0037] TextCNN extracts local n-gram features through multi-scale convolution kernels, although it has good ability to capture local semantics, but it performs poorly in modeling long-distance context, and cannot effectively capture the semantic information of cross-sentence or long-distance dependence in the text. At the same time, Bi-LSTM can capture the temporal dependence relationship in the text, and capture the context information through the forward and reverse recurrent neural network, but it has limitations in processing non-continuous key fragments, especially in long text, the model may ignore those important but discontinuous semantic information. This fragmented feature representation makes it difficult for the model to fully capture and integrate the semantic information in the text when dealing with diverse texts. At the same time, the existing methods generally lack the utilization of label semantics, and many methods ignore the semantic information contained in the labels.

[0038] In addition, the graph-based method, such as TextGCN and HAN, uses a static co-occurrence matrix or a knowledge graph to construct label relationships, and models the relationships between labels with graph convolution. The advantage of these methods is that they can effectively mine the potential relevance between labels, but because the constructed graph structure is static, the establishment of label relationships is often too simple and rough, lacking deep semantic understanding. This coarse-grained modeling makes it difficult for the model to capture more subtle and variable semantic associations when dealing with complex texts. Therefore, as the diversity of input text and the change of context, the static graph cannot be dynamically adjusted according to the semantics of the specific text. When the input data has different domain backgrounds or context changes, the static graph often cannot effectively capture the different associations between labels in a specific context.

[0039] To solve the above problems, the present application provides a multi-label text classification method and device based on graph neural network. The core idea of the model is to realize the cooperative optimization of text semantics and label relationships through dynamic parameterization mechanism and multi-level text feature coding. Specifically, the model first dynamically adjusts the relationships between labels through a dynamic graph convolution network, so that the label relationship graph can dynamically change according to the context of different texts, thereby better capturing the changes and interactions between labels in different scenarios. In addition, by hybridly using Bi-LSTM and CNN, combined with the self-attention mechanism, the deep semantic representation of the text is captured based on different fine-grained feature levels, forming hierarchical semantic representation, which significantly improves the completeness of feature expression. In order to better illustrate the technical effects of the present application, the following embodiments are provided for detailed description.

[0040] As shown in Figure 1 The present application provides a multi-label text classification method based on graph neural network, the method comprising:

[0041] Step S110: performing feature extraction on the text to be classified based on a preset multi-level extraction rule to obtain target text features.

[0042] The preset multi-level extraction rule is respectively extracting the global sequence level feature and the local phrase level feature by using corresponding encoders.

[0043] Specifically, the global sequence level feature is extracted by Bi-LSTM first, so as to capture the long-range dependency in the text, and then the local phrase level feature is extracted by CNN. The multi-head self-attention mechanism is introduced to focus on important parts of the text, and the understanding and representation ability of the model to the text are enhanced. The multi-head self-attention mechanism breaks through the limitation of local window and sequence length by calculating the correlation strength of all position pairs in parallel, and establishes a long-range semantic bridge explicitly. The multi-head design allows the model to learn diverse interaction patterns in different subspaces: for example, some attention heads focus on syntactic structure, and others focus on semantic roles, and finally the multi-dimensional semantic fusion is realized through concatenation and linear transformation.

[0044] Step S120: constructing a label co-occurrence matrix based on the text to be classified, introducing a dynamic weight matrix, and convolving to obtain label features.

[0045] Specifically, based on the label co-occurrence matrix, a learnable dynamic weight matrix is introduced, the dependency between labels is dynamically adjusted, and convolution is performed, so as to adaptively correct the correlation strength between labels and form more rich label feature representation.

[0046] Step S130: based on the cross-attention mechanism, the target text feature and the label feature are fused to obtain the fusion feature.

[0047] Specifically, based on the cross-attention mechanism, the text feature and the label feature are fused to strengthen the interaction between the text and the label, so as to capture the complex relationship between the text and the label.

[0048] Step S140: weighting the fusion feature to obtain the independent prediction probability of each label.

[0049] Specifically, the fused feature is weighted by the gating mechanism, the fusion weight is dynamically adjusted, and the independent prediction probability of each label is output by combining the Sigmoid function. The Sigmoid function is a kind of activation function.

[0050] The technical principle of the present application is: based on a preset multi-level extraction rule, features of the text to be classified are extracted to obtain target text features, and the features extracted by the multi-level extraction rule are more complete and richer than the features extracted by a single rule; based on the text to be classified, a label co-occurrence matrix is constructed, and a dynamic weight matrix is introduced, and label features are obtained by convolution, so that the correlation strength between labels can be adaptively corrected to form a more rich label feature representation; based on the cross attention mechanism, the target text features and the label features are fused to obtain fusion features, the text features and the label features are fused to strengthen the interaction between the text and the label and capture the complex relationship between the text and the label; and the fusion features are weighted to obtain the independent prediction probability of each label.

[0051] Compared with the prior art, the present application has the following beneficial effects: the features extracted by the multi-level extraction rule are more complete and richer than the features extracted by a single rule; at the same time, by introducing a dynamic weight matrix in the label co-occurrence matrix, the correlation strength between labels can be adaptively corrected to form a more rich label feature representation; by fusing the text features and the label features, the interaction between the text and the label is strengthened, the complex relationship between the text and the label is captured, and finally the fusion features are weighted to obtain the independent prediction probability of each label, which effectively improves the adaptability of diversified labels in complex scenarios.

[0052] It solves the technical problems in the prior art that the graph structure constructed is static, the establishment of label relationship is too simple and rough, and lacks deep semantic understanding, so that the model cannot capture more subtle and variable semantic associations when processing complex texts, and the static graph cannot be dynamically adjusted according to the semantics of specific texts, nor can it effectively capture different associations between labels in a specific context.

[0053] By dynamically adjusting the relationship between labels through a dynamic graph convolution network, the label relationship graph can dynamically change according to the context of different texts, thereby better capturing the changes and interactions between labels in different scenarios. When the input data has different domain backgrounds or context changes, it can also effectively capture the different associations between labels in a specific context by combining the content of the corresponding target text feature part. Thus, the classification and adaptability of diversified labels in complex scenarios are effectively improved.

[0054] In some embodiments, as shown in Figure 2 Step S110 further includes:

[0055] Step S111: different encoders are used to read out global sequence level features and local phrase level features from the text to be classified respectively.

[0056] Specifically, Bi-LSTM and CNN are used in parallel to capture the global context dependency and local word-level features of the text, respectively.

[0057] Step S112: splice the global sequence-level features and the local phrase-level features to obtain initial text features.

[0058] Specifically, the global sequence-level features and the local phrase-level features are spliced to obtain the input of the self-attention mechanism, i.e., the initial text features.

[0059] Step S113: performing feature extraction on the initial text features based on the self-attention mechanism to obtain target text features.

[0060] Specifically, after completing the Bi-LSTM and CNN feature extraction, the multi-head self-attention mechanism is introduced to realize cross-level semantic interaction and dynamic feature enhancement. The basic self-attention mechanism process is to realize the context-aware feature representation by dynamically modeling the correlation strength between elements in the sequence. First, generate query vector (Query, Q), key vector (Key, K) and value vector (Value, V) for each position in the sequence X. The similarity between the query vector K and all key vectors Q is calculated, and after Softmax normalization, the value vector is weighted and summed as a weight. Specifically, given the input sequence X ∈ R n×d , Q, K, V are obtained through linear transformation W q ,W k ,W v ∈R d×dk The multi-head self-attention mechanism projects the query, key, and value into h independent subspaces to obtain key information in different dimensions in parallel and learn diverse correlation patterns. The specific implementation process is first to split the input sequence into h groups of features through the parameter matrix W l Q ,W l K ,W i V ∈R d×dh (dh=d / h):

[0061]

[0062] Then calculate each group of features through dot product:

[0063]

[0064] Finally, splice all attention head outputs, and through linear transformation W O ∈R hdh×d fuse into the final representation:

[0065] MultiHead(Q,K,V)=Concat(head1,...,head h )W O

[0066] This design enables the model to focus on local syntax structure, long-range semantic association and other heterogeneous features in different subspaces. In an embodiment of the present application, the obtained Bi-LSTM output and CNN output are first spliced as an input sequence X={x1,x2,...,x n} into a multi-head self-attention module, and 8 attention heads are used for feature extraction, and finally the output Htextatt, i.e., MultiHead(Q,K,V), is obtained.

[0067] Beneficial effects: By mixed use of Bi-LSTM and CNN, combined with the self-attention mechanism, the deep semantic representation of the text is captured based on different fine-grained feature levels, forming hierarchical semantic representation, which significantly improves the completeness of feature expression.

[0068] In some embodiments, as shown in Figure 3 , the encoder includes a bidirectional encoder and a convolutional encoder; step S111 further includes:

[0069] Step S210: using a bidirectional encoder to obtain global sequence-level features of the text to be classified based on forward and backward bidirectional reading of the text to be classified.

[0070] Specifically, the specific work of Bi-LSTM is to realize forward and backward propagation through two LSTMs, so that the model can simultaneously perceive the historical context and future context of each word and update the hidden state thereof using a threshold mechanism, and finally splice the hidden states.

[0071] For a given input text sequence representation X=[x1,x2,...,x L ]∈R L×300 , where each position x i corresponds to a 300-dimensional pre-trained Glove word vector. Bi-LSTM first uses two LSTMs to realize forward and backward propagation: word-by-word processing from the beginning to the end of the sequence, calculating the hidden state, and capturing the relationship between the current word and the previous context:

[0072]

[0073] At the same time, the sequence is transmitted in reverse from the end to the beginning, generating a hidden state that captures the association between the current word and the following context:

[0074]

[0075] Finally, the bidirectional hidden states are concatenated by position to obtain the final Bi-LSTM text feature H LSTM ∈

[0076] R L×2dh (dh is the hidden layer dimension):

[0077]

[0078] Step S220: using a convolutional encoder to read the text to be classified based on multiple layers of convolution to obtain local phrase-level features of the text to be classified.

[0079] Specifically, CNN focuses on the extraction of local semantic patterns. By scanning the text sequence with a convolution kernel, CNN can effectively capture key phrase combinations, such as "quantum entangled state", "deep neural network" and other professional terms, which usually carry strong classification signals. The max-pooling operation further filters significant patterns and suppresses non-key vocabulary interference. In addition, compared with recurrent networks, the translational invariance of CNN makes it more robust to word order changes. Therefore, the parallel of the two can extract text features from different fine-grained levels, making the obtained text features more optimal.

[0080] The model uses 2-3 layers of convolutional layers according to different data sets. Taking a two-layer CNN as an example, its specific workflow is to realize feature refinement through two-stage convolution-pooling: the first layer of convolution captures the basic n-gram pattern, and the second layer of convolution learns the composite semantic combination on the fixed length sequence, and finally outputs a 300-dimensional vector.

[0081] For a given input text sequence representation X = [x1, x2,..., x L ] ∈ R L×300 , where each position x i corresponds to a 300-dimensional pre-trained Glove word vector. To adapt to the characteristics of convolution operation, first perform dimension permutation on the input to obtain X ′ ∈ R 300×L , taking the word embedding dimension as the input channel:

[0082]

[0083] Then use the first layer of convolution to extract local features, with a convolution kernel k = 3 and an input channel of 300:

[0084] H (1) = ReLU (W1 x X' + b1) ∈ R 256×L

[0085] where w (1) is a trainable parameter, and an output of 256 channel feature maps.

[0086] Then, in order to reduce the sequence dimension and enhance the local invariance, the sequence length is compressed to a fixed dimension through max-pooling:

[0087] H pool =AdaptiveMaxPool1D(H (1) )∈R 256×100

[0088] Then, the second layer of convolution kernel size remains k=3, the input channel is 256, the output channel is restored to 300, the activation function is ReLU, and high-order semantic aggregation is performed.

[0089] H (2) =ReLU(W2×P+b2)∈R 300×100

[0090] Then, the channel dimension is restored through the inverse permutation operation, as shown in the following formula, to obtain the time sequence feature aligned with the original input.

[0091] F=Permute(H (2) )∈R 100×300

[0092] Finally, the document local phrase-level semantic representation vector is generated through dimension restoration and global average pooling.

[0093] R CNN =MeanPool(H (2)T )∈R 300 .

[0094] Beneficial effects: The technical scheme of using different encoders to read global sequence-level features and local phrase-level features from the text to be classified is further refined, enriching the content of the application. By respectively using a bidirectional encoder and a convolutional encoder, the text to be classified is read to obtain corresponding global sequence-level features and local phrase-level features. Thus, hierarchical semantic representation is formed, significantly improving the completeness of feature expression.

[0095] In some embodiments, as shown in Figure 4 , step S120 further includes:

[0096] Step S121: Count the frequency of each label in the text to be classified, and construct a label co-occurrence matrix according to the frequency of each label.

[0097] Specifically, in the dynamic adjacency matrix construction phase, since there is often also an association between labels, with a hierarchical structure or correlation, first, based on the training set, the label co-occurrence frequency matrix Aco∈R L×L , element A ij represents the co-occurrence frequency of labels l i and l jThe number of common occurrences in all samples is calculated as follows.

[0098]

[0099] where Y k is the label set of the k-th sample; I(·) is an indicator function that outputs 1 when the condition in the parentheses is true, and otherwise outputs 0. Specifically, I(l i ∈Y k ∧l j ∈Y k ) = 1 if and only if the labels l i and l j co-occur in the k-th sample.

[0100] Step S122: Introduce a dynamic matrix in the label co-occurrence matrix, and dynamically adjust based on a dynamic gating mechanism to obtain label features by convolution.

[0101] Specifically, the model then introduces a dynamic weight matrix Λ, and uses a Sigmoid function to perform nonlinear mapping on Λ and the identity matrix (I) (representing label self-connection) to generate a dynamic weight gate, as shown in the following equation. Figure 5

[0102] where σ represents the sigmoid function, σ(Λ ij ) = 1 / (1 + e-Λ ij ) constrains the weight to the interval (0, 1). This design makes the strength of each edge A dyn [i,j] a differentiable function, preserving the original co-occurrence information while allowing the model to enhance or suppress specific associations by adjusting Λ ij . For example, when Λ ij → +∞, σ(Λ ij ) → 1, and A dyn [i,j] ≈ A co [i,j], emphasizing the prior co-occurrence relationship; conversely, when Λ ij → -∞, A dyn [i,j] → I[i,j], then the external association is weakened, and the focus is on the label's own features.

[0103] Subsequently, the label embedding is propagated through two layers of GCN, as shown in the following equation.

[0104]

[0105] where E label ∈ R L×d is the initial label embedding, is the projection matrix. The propagation of each layer is realized through the dynamic adjacency matrix A dyn to aggregate neighborhood information, making the label embedding Z​(2) Encoding optimized semantic relations. For example, if the model learns the implicit association of labels in a specific context, even if its co-occurrence frequency is low, it can be enhanced by increasing Λ ij to improve the connection strength of both in A dyn , thereby enhancing feature interaction during graph convolution.

[0106] The update of the key parameter Λ relies on the composite gradient mechanism. The total loss function is defined as shown in Figure 6 During the backpropagation phase, the classification loss gradient is first passed through the cross-attention module to the label embedding Z (2) , and then the gradient of A dyn is calculated along the GCN propagation path. Subsequently, according to the chain rule and the Sigmoid function, the gradient of Λ is calculated as shown in Figure 7 .

[0107] This formula consists of three parts: (1) the semantic error signal driven by the classification task; (2) the local curvature adjustment of the Sigmoid function; (3) the difference weight between the co-occurrence matrix and the identity matrix. Taking the label pair (y p , y q ) as an example, if the model overestimates its relevance and makes a classification error, the gradient is negative, pushing Λ pq to decrease, thereby reducing the proportion of co-occurrence weight in A dyn [p,q].

[0108] For example, assume that the initial co-occurrence weight of label y i (“virus”) and y j (“vaccine”) is A co [i,j] = 0.3, and the initial Λ ij = 0 corresponds to A dyn [i,j] = 0.5 x 0.3 + 0.5 x 0 = 0.15. If the model discovers during training that the co-occurrence of the two is crucial for classification, the backpropagation will produce a positive gradient driving Λ ij to increase. After several iterations, when A ij = 2.0, σ(2.0) ≈ 0.88, and the dynamic weight is increased to A dyn [i,j] = 0.88 x 0.3 + 0.12 x 0 = 0.264, which is significantly higher than the initial value. Through this process of dynamic updating of the adjacency matrix, the model can capture the potential semantic synergy between labels, improving the accuracy and interpretability of multi-label text classification.

[0109] Beneficial effects: further refine the steps of constructing a label co-occurrence matrix based on the text to be classified, introducing a dynamic weight matrix, and convolving to obtain label features, enriching the embodiments of the application. By introducing a dynamic weight matrix in the label co-occurrence matrix, the potential semantic synergy between labels can be captured, improving the accuracy and interpretability of multi-label text classification.

[0110] In some embodiments, as shown in FIG. 1, Figure 8 Step S130 further comprises:

[0111] Step S131: embedding processing is performed on the label features to obtain embedding vectors.

[0112] Step S132: linear projection is performed according to the embedding vectors to obtain a query vector.

[0113] Specifically, first, the GCN-optimized label embedding E label ∈R C×d is projected by a linear projection W Q ∈R d×d to generate Q as a query vector. These label embeddings represent high-order semantic information of the labels and can guide the model to focus on the text part related to each label.

[0114] Step S133: a projection matrix of the text features is established, and a key vector and a value vector are obtained according to the projection matrix.

[0115] Specifically, in addition to the text features H text ∈R L×d , the projection matrix W K ∈R d×d and W V ∈r d×d are used to map the text features into a key vector K and a value vector V, respectively.

[0116] Step S134: according to the query vector and the key vector, a correlation matrix between the label features and the text features is obtained.

[0117] Specifically, then the label-text association matrix is calculated, and a weight is assigned to the association degree between each label and each position in the text. By calculating the dot product of the query vector Q and the key vector K of the text, and then scaling by a scaling factor, the correlation matrix between the labels and the text is obtained:

[0118]

[0119] Step S135: based on the correlation matrix, the value vector is weighted and aggregated to obtain a fusion feature.

[0120] Specifically, based on the calculated attention weight matrix A, the value vector V of the text is weighted and aggregated to obtain the label-related context representation H text2label . Specifically, the label features focus on the text features related to them through attention weights to generate the context representation H text2label , as shown below.

[0121] H text2label =A V ∈R C×d .

[0122] where the matrix multiplication A V performs weighted aggregation on each position of the text to obtain the context representation of each label. This process enables the feature representation of each label to actively focus on the text segment related to the semantic of the label, thereby improving the model's understanding and representation ability of label-specific semantics.

[0123] Beneficial effects: further refine the cross-attention mechanism-based step of fusing target text features and label features to obtain fused features, enriching the embodiments of the present application. Through the cross-attention mechanism, the label-related semantic features and the global text features (i.e., target text features) are fused to form a joint representation that takes into account both the global context and the label characteristics, providing a more discriminative input for the classification layer.

[0124] In some embodiments, as shown in Figure 9 , step S134 further includes:

[0125] Step S310: Perform dot product calculation according to the query vector and the key vector.

[0126] Specifically, the dot product calculation is performed by multiplying the query vector Q and the key vector K of the text.

[0127] Step S320: Scale the dot product calculation using a scaling factor to obtain a relevance matrix.

[0128] Specifically, the relevance matrix between the label and the text is obtained by scaling with a scaling factor:

[0129]

[0130] where, the scaling factor is used to prevent gradient anomalies, and the Softmax operation normalizes the weights of each row to ensure that the sum of the relevance of each label to all text positions is 1, thereby obtaining the normalized attention weight matrix A. This process can be regarded as label-guided text attention, i.e., each label learns how to focus on the text part related to its semantics.

[0131] Beneficial effects: further refine the step of obtaining the correlation matrix between the label features and the text features according to the query vector and the key vector, enrich the application scheme. Through the dot product calculation according to the query vector and the key vector, and then scaling the dot product calculation by using the scaling factor, the correlation matrix is obtained. Thus, each label can learn to focus on the text part related to its semantics, thereby improving the correlation between the label and the text features.

[0132] In some embodiments, as shown in FIG. 13, step S140 further includes: Figure 10

[0133] Step S141: using a gating mechanism to perform weighted processing on the fused features, and dynamically adjusting the weight of the fused features.

[0134] Specifically, the fused features are weighted processed by the gating mechanism to dynamically adjust the fusion weight.

[0135] Step S142: based on the weight, using a preset activation function to output the independent prediction probability of each label.

[0136] Specifically, the independent prediction probability of each label is output by combining the dynamically adjusted fusion weight and the Sigmoid function.

[0137] Beneficial effects: further refine the step of weighting the fused features to obtain the independent prediction probability of each label, enrich the embodiments of the application. By using the gating mechanism to perform weighted processing on the fused features, the weight of the fused features is dynamically adjusted, and then based on the weight, the independent prediction probability of each label is output by using the preset activation function, thereby realizing the prediction of the probability of each label, and further realizing the classification of the label.

[0138] In some embodiments, as shown in FIG. 13, step S141 further includes: Figure 11

[0139] Step S410: based on the gating mechanism, constructing a gating weight matrix.

[0140] Specifically, the gating mechanism uses a learnable gating weight matrix W g ∈R 2d×d .

[0141] Step S420: performing nonlinear transformation on the fused features according to the gating weight matrix to obtain a gating vector.

[0142] Specifically, the nonlinear transformation is performed on the spliced label features and text features to generate a gating vector g:

[0143] g=σ([Z||H]W g )∈R d ​​.

[0144] where σ is a Sigmoid activation function, ensuring that the gating vector g is between [0, 1];

[0145] Step S430: dynamically adjusting the fusion weight of the fusion feature based on the gating vector.

[0146] Specifically, [Z||H] represents splicing the label feature H text2label and the text feature H textatt along the feature dimension. Each element g i in the gating vector g controls the fusion weight of the label feature and the text feature.

[0147] Beneficial effects: further refine the step of dynamically adjusting the weight of the fusion feature by using the gating mechanism to weight the fusion feature, enrich the embodiments of the present application. By constructing a gating weight matrix based on the gating mechanism, and then performing nonlinear transformation on the fusion feature according to the gating weight matrix to obtain a gating vector, and finally dynamically adjusting the fusion weight of the fusion feature based on the gating vector, the dynamic adjustment of the fusion feature weight is realized, which prepares for subsequent label prediction classification based on dynamic adjustment.

[0148] In some embodiments, as shown in Figure 12 , step S142 further includes:

[0149] Step S510: establishing a fusion formula based on the dynamically adjusted fusion weight.

[0150] Specifically, the gating vector g adjusts the fusion ratio of the label feature and the text feature, so that the model can flexibly adjust the attention degree to the two types of features. The fusion formula is as follows:

[0151] F = g o Z + (1-g) o H e R d

[0152] where F is the fused feature, representing the weighted combination of the label feature and the text feature; o represents element-wise multiplication, i.e. weighting each feature dimension.

[0153] Step S520: performing label space mapping on the fusion formula to obtain the unactivated score value of each label.

[0154] Specifically, F is mapped to the label space through a fully connected layer to obtain the unactivated score value logits:

[0155] logits = F W c + b c (W c e R d×C , b c e RC )

[0156] wherein W c is the weight matrix of the fully connected layer, b c is the bias term; logits∈R C represents the unactivated score of each label.

[0157] Step S530: mapping the unactivated score of each label to a preset interval by using an activation function to obtain the independent prediction probability of each label.

[0158] Specifically, the score logits of each label is finally mapped to the interval [0, 1] by the Sigmoid activation function, and the independent probability of each label is output:

[0159]

[0160] The final output layer generates the final multi-label prediction result by threshold determination. Therefore, the final label prediction can be represented as follows.

[0161]

[0162] wherein, represents the prediction value of label l i , P i is the prediction probability of the corresponding label.

[0163] Beneficial effects: further refine the step of outputting the independent prediction probability of each label based on the weight and using the preset activation function, enrich the embodiments of the application. By establishing a fusion formula based on the dynamically adjusted fusion weight, and then performing label space mapping on the fusion formula, the unactivated score of each label is obtained; finally, the unactivated score of each label is mapped to a preset interval by using an activation function to obtain the independent prediction probability of each label. Thus, the independent probability of the label is predicted, and the classification of the label is realized.

[0164] On the other hand, as shown in Figure 13 , the embodiments of the application also provide a multi-label text classification device 600 based on a graph neural network, the device 600 comprising:

[0165] A multi-level text feature encoding module 610 is configured to extract features from a text to be classified based on a preset multi-level extraction rule to obtain target text features.

[0166] A dynamic label embedding learning module 620 is configured to construct a label co-occurrence matrix based on the text to be classified, and introduce a dynamic weight matrix to obtain label features by convolution.

[0167] The cross-attention module 630 is configured to fuse the target text features and the label features based on a cross-attention mechanism to obtain fused features.

[0168] The classification module 640 is configured to perform weighted processing on the fused features to obtain independent prediction probabilities of the labels.

[0169] In the multi-level text feature encoding module 610, Bi-LSTM and CNN are used in parallel to capture global context dependency and local word-level features of the text, respectively, and then multi-head self-attention is used to further optimize the extraction of the text features, so as to realize multi-granularity text feature extraction. When processing the text data, RNN can process the text word by word and retain historical information through the cyclic transmission of the state, and is suitable for modeling long-distance context association. LSTM solves the problem of gradient disappearance, and Bi-LSTM further proposed by LSTM has the ability of bidirectional time sequence modeling of sequence data. Through the double transmission mechanism of forward and reverse, Bi-LSTM can capture the forward and reverse dependencies of the word sequence in the text, such as identifying the context association in the resolution of anaphora or long-span logical connection. Compared with unidirectional LSTM, the bidirectional structure significantly improves the parsing accuracy of complex sentence structures, especially for long text scenes such as academic literature and legal texts.

[0170] The dynamic label embedding learning module 620 is mainly used for extracting label features. In traditional methods, the information value contained in the label is often ignored. The model uses GCN to utilize the label features, and introduces a learnable weight matrix to overcome the shortcomings of the static graph in traditional models that cannot capture the more subtle and variable semantic associations between labels. The core process can be divided into three stages: dynamic adjacency matrix construction, graph convolution feature propagation, and parameter gradient calculation.

[0171] The cross-attention module 630 establishes a connection between the label features and the text features through the cross-attention mechanism, dynamically adjusts the attention of the text segments, and thus enhances the coverage and learning of low-frequency labels. Specifically, the module jointly represents the label features and the text features, so that each label can focus on the text part related to its semantics, thereby improving the ability to capture label-specific information.

[0172] The classification module 640 first integrates the label-related semantic features from the cross-attention and the global text features generated by the self-attention based on a gating mechanism, performs weighted processing, dynamically adjusts the fusion weight, and finally outputs the independent prediction probability of each label in combination with the Sigmoid function.

[0173] The working principle of the embodiment of the application is as follows: feature extraction is performed through the multi-level text feature coding module 610 to obtain target text features; then, the dynamic label embedding learning module 620 constructs a co-occurrence matrix and further introduces a dynamic weight matrix to obtain label features; then, the cross-attention module 630 is used to realize fusion of the target text features and the label features to obtain fused features; finally, the classification module 640 is used to perform weighted processing on the fused features to obtain independent prediction probabilities of each label, so as to realize label classification of the text to be classified.

[0174] Beneficial effects: the dynamic graph convolution network dynamically adjusts the relationship between labels, so that the label relationship graph can dynamically change according to the context of different texts, thereby better capturing the changes and interactions between labels in different scenarios. When the input data has different domain backgrounds or context changes, the corresponding target text feature part can also be combined to effectively capture different associations between labels in a specific context. Thus, the classification and adaptability of diversified labels in complex scenarios are effectively improved.

[0175] The applicant has proved through experiments that the application performs best compared with other prior art. The specific experimental results are described in detail.

[0176] The experimental results are shown in Tables 1 and 2 as follows:

[0177] Table 1

[0178]

[0179] Table 2

[0180]

[0181] All the baseline model results are from the original papers of the models. For the convenience of result comparison, the optimal performance indicators in the experimental results are marked in bold font, and the suboptimal indicators are presented in underlined form.

[0182] On the AAPD dataset, the HL value of the model is 0.021, which is the best performance, and the Micro-precision is the second best among the models. Although its Micro-precision value is slightly lower than that of Text-CNN, the Recall value is significantly higher than that of Text-CNN, indicating that the model has found a better balance between precision and coverage. Text-CNN relies on CNN to focus on local features, and although it can accurately match some explicit terms, it lacks global semantic modeling, resulting in more missed cases. The model dynamically adjusts the correlation strength between labels through dynamic graph convolution, and simultaneously uses Bi-LSTM and CNN to extract features and combine multi-head attention mechanism to realize multi-level text feature encoding, and through cross-attention mechanism to realize the interaction between labels and text features, thereby improving the robustness of prediction. Compared with the LP-MLM model, it can be seen that the model does not consider the long-tail distribution, and lacks the ability to identify low-frequency labels, which performs poorly in recall rate; although it performs well in precision, the overall performance is still unbalanced.

[0183] On the RCV1-V2 dataset, the Micro-F1 value of the model reaches 0.884, and the HL value is 0.0072, which is better than the rest of the baseline models. Unlike AAPD, RCV1-V2 dataset comes from news reports, and there are cross-domain associations between labels, which requires higher dynamic adaptation ability of the model in timeliness and diversity. Dynamic graph convolution updates the weight of the co-occurrence matrix in real time, which improves the response speed of the model to emerging label combinations, and thus makes the model performance better than other models. In addition, multi-level text encoding enables the model to accurately capture keywords while effectively discovering implicit associations. This synergistic effect allows the model to maintain high precision while having one of the highest recall rates among existing methods, thereby improving overall performance.

[0184] The comparison experiment with multiple benchmark models on the AAPD and RCV1-V2 datasets proves that the model has certain improvement in classification performance. In terms of Micro-Precision, Micro-Recall, Micro-F1 value and Hamming Loss, it respectively reaches 0.824, 0.653, 0.724, 0.0210; 0.908, 0.860, 0.884, 0.0072; achieving good classification effect. It has certain guiding significance and practical utility value for multi-label text classification tasks.

[0185] Figure 14A structural diagram of an embodiment of the computer device of the present application is shown, which shows a structural diagram of a computer system of the computer device suitable for implementing the embodiment of the present application, and the embodiment of the present application does not limit the specific implementation of the computer device.

[0186] Please refer to Figure 14 As shown, the computer device includes a controller, a memory for storing one or more programs, when the one or more programs are executed by the controller, to execute the above-mentioned multi-label text classification method based on a graph neural network.

[0187] Please continue to refer to Figure 14 As shown, the computer system 700 of the computer device includes a central processing unit (CPU) 701, which can perform various appropriate actions and processes according to the programs stored in the read-only memory (ROM) 702 or the programs loaded from the storage portion 708 into the random access memory (RAM) 703, such as performing the method in the above-mentioned embodiments. In the RAM 703, various programs and data required for system operation are also stored. The CPU 701, the ROM 702, and the RAM 703 are connected to each other through a bus 704. An input / output (I / O) interface 705 is also connected to the bus 704.

[0188] The following components are connected to the I / O interface 705: an input portion 706 including a keyboard, a mouse, and the like; an output portion 707 including a cathode ray tube (CRT), a liquid crystal display (LCD), and the like, and a speaker, and the like; a storage portion 708 including a hard disk, and the like; and a communication portion 709 including a network interface card such as a LAN (Local Area Network) card, a modem, and the like. The communication portion 709 performs communication processing via a network such as the Internet. A drive 710 is also connected to the I / O interface 705 as needed. A removable medium 711 such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, and the like is mounted on the drive 710 as needed, so that a computer program read therefrom is installed in the storage portion 708 as needed.

[0189] In particular, according to embodiments of the present application, the processes described above with reference to the flowcharts can be implemented as a computer software program. For example, embodiments of the present application include a computer program product comprising a computer program carried on a computer readable medium, the computer program comprising computer programs for executing the methods illustrated by the flowcharts. In such embodiments, the computer program can be downloaded and installed from a network by the communication section 709, and / or installed from the detachable medium 711. When the computer program is executed by the central processing unit (CPU) 701, various functions defined in the system of the present application are executed.

[0190] Another aspect of the present application also provides a computer readable storage medium, the computer readable storage medium storing at least one executable instruction, the executable instruction causing a computer device to execute a multi-label text classification method based on a graph neural network when the computer device runs the executable instruction.

[0191] Beneficial effects: The relationship between labels is dynamically adjusted by the dynamic graph convolution network, so that the label relationship graph can dynamically change according to the context of different texts, thereby better capturing the changes and interactions between labels in different scenarios. When the input data has different domain backgrounds or context changes, it can also effectively capture the different associations between labels in a specific context by combining the content of the corresponding target text feature part. Thus, the classification and adaptability of diversified labels in complex scenarios are effectively improved.

[0192] It should be noted that the computer-readable medium in the embodiments of the present application can be a computer-readable signal medium or a computer-readable storage medium or any combination thereof. The computer-readable storage medium may, for example, be an electronic, magnetic, optical, electromagnetic, infrared, or semiconductor system, device or apparatus, or any combination thereof. More specific examples of the computer-readable storage medium can include, but are not limited to, an electrical connection having one or more wires, a portable computer diskette, a hard disk, a random access memory (RAM), a read-only memory (ROM), an erasable programmable read-only memory (EPROM), a flash memory, an optical fiber, a portable compact disc read-only memory (CD-ROM), an optical storage device, a magnetic storage device, or any suitable combination thereof. In this application, the computer-readable storage medium can be any tangible medium that contains or stores a program used by an instruction execution system, apparatus or device, and can be used or combined with the same. In this application, the computer-readable signal medium can include a data signal carried in a baseband or as a part of a carrier wave, which carries computer-readable computer programs. Such a propagated data signal can take various forms, including, but not limited to, an electromagnetic signal, an optical signal, or any suitable combination thereof. The computer-readable signal medium can also be any computer-readable medium other than the computer-readable storage medium, which can send, transmit, propagate or transport a program for use by or in connection with an instruction execution system, apparatus or device. The computer programs contained in the computer-readable medium can be transmitted by any suitable medium, including, but not limited to, wireless, wired, or the like, or any suitable combination thereof.

[0193] The flowcharts and block diagrams in the drawings illustrate the possible implementation architectures, functions and operations of the systems, methods and computer program products according to various embodiments of the present application. In the flowcharts or block diagrams, each block can represent a module, a program segment or a part of code, which contains one or more executable instructions for implementing the specified logical functions. It should also be noted that in some alternative implementations, the functions noted in the blocks can occur in a different order from that noted in the drawings. For example, two blocks represented in succession can actually be executed substantially in parallel, and sometimes in reverse order, depending on the functions involved. It should also be noted that each block in the block diagrams or flowcharts, and the combination of blocks in the block diagrams or flowcharts, can be implemented by a dedicated hardware-based system for implementing the specified functions or operations, or can be implemented by a combination of special-purpose hardware and computer instructions.

[0194] The units described in the embodiments of the present application can be implemented by software, or by hardware, or by a combination of software and hardware. The units described can also be located in a single processor. In some cases, the names of the units do not limit the units themselves.

[0195] According to an aspect of the embodiments of the present application, a computer system is also provided, which includes a central processing unit (CPU) that can perform various appropriate actions and processes according to a program stored in a read-only memory (ROM) or a program loaded from a storage section into a random access memory (RAM), such as performing the method in the above embodiments. In the RAM, various programs and data required for system operation are also stored. The CPU, the ROM, and the RAM are connected to each other through a bus. An input / output (I / O) interface is also connected to the bus.

[0196] The following components are connected to the I / O interface: an input section including a keyboard, a mouse, etc.; an output section including a display such as a cathode ray tube (CRT), a liquid crystal display (LCD), etc., and a speaker, etc.; a storage section including a hard disk, etc.; and a communication section including a network interface card such as a LAN (Local Area Network) card, a modem, etc. The communication section performs communication processing via a network such as the Internet. A drive is also connected to the I / O interface as necessary. A removable medium such as a magnetic disk, an optical disk, a magneto-optical disk, a semiconductor memory, etc. is attached to the drive as necessary, so that a computer program read out from the removable medium is installed into the storage section as necessary.

[0197] Finally, it should be noted that the above embodiments are merely intended to illustrate the technical solutions of the present application, not to limit the present application. Although the present application is described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present application can be modified or replaced equivalently without departing from the spirit and scope of the present application, and all of them should be covered in the scope of the claims of the present application.

Claims

1. A multi-label text classification method based on graph neural network, characterized in that: The method comprises: Based on the preset multi-level extraction rules, feature extraction is performed on the classified text to obtain the target text features; Constructing a label co-occurrence matrix based on the text to be classified, introducing a dynamic weight matrix, and convolving to obtain label features; Based on the cross attention mechanism, the target text feature and the label feature are fused to obtain a fused feature; The fusion features are weighted to obtain the independent prediction probability of each label.

2. The method according to claim 1, characterized in that The step of extracting features from the text to be classified based on the preset multi-level extraction rules to obtain target text features further includes: Different encoders are used to read global sequence-level features and local phrase-level features from the text to be classified; Concatenating the global sequence-level features and the local phrase-level features to obtain initial text features; The initial text features are extracted based on the self-attention mechanism to obtain target text features.

3. The method according to claim 2, characterized in that The encoder includes a bidirectional encoder and a convolutional encoder; the step of using different encoders to read global sequence-level features and local phrase-level features from the text to be classified further includes: A bidirectional encoder is used to read the text to be classified in a forward and backward bidirectional manner to obtain a global sequence-level feature of the text to be classified; A convolutional encoder is used to read the text to be classified based on multi-layer convolution to obtain local phrase-level features of the text to be classified.

4. The method according to claim 1, wherein The step of constructing a label co-occurrence matrix based on the text to be classified, introducing a dynamic weight matrix, and convolving to obtain label features further includes: Counting the frequency of occurrence of each tag in the text to be classified, and constructing a tag co-occurrence matrix based on the frequency of occurrence of each tag; A dynamic matrix is ​​introduced into the tag co-occurrence matrix, and is dynamically adjusted based on a dynamic gating mechanism to obtain tag features through convolution.

5. The method according to claim 1, wherein The step of fusing the target text feature and the label feature based on the cross attention mechanism to obtain a fused feature further includes: Embedding the label features to obtain an embedding vector; Performing linear projection on the embedding vector to obtain a query vector; Establishing a projection matrix of the text features, and obtaining a key vector and a value vector according to the projection matrix mapping; Obtaining a correlation matrix between the tag features and the text features according to the query vector and the key vector; The value vectors are weighted and aggregated based on the correlation matrix to obtain fusion features.

6. The method according to claim 5, characterized in that The step of obtaining a correlation matrix between the tag features and the text features based on the query vector and the key vector further includes: Performing a dot product calculation based on the query vector and the key vector; The dot product calculation is scaled using a scaling factor to obtain the correlation matrix.

7. The method according to claim 1, characterized in that The step of performing weighted processing on the fused features to obtain independent prediction probabilities of each label further includes: A gating mechanism is used to perform weighted processing on the fusion features, and the weights of the fusion features are dynamically adjusted; Based on the weights, a preset activation function is used to output the independent prediction probability of each label.

8. The method according to claim 7, characterized in that The step of using a gating mechanism to perform weighted processing on the fused features and dynamically adjusting the weights of the fused features further includes: Based on the gating mechanism, constructing a gating weight matrix; Performing a nonlinear transformation on the fusion feature according to the gating weight matrix to obtain a gating vector; Based on the gating vector, the fusion weight of the fusion feature is dynamically adjusted.

9. The method according to claim 8, characterized in that The step of outputting the independent prediction probability of each label using a preset activation function based on the weight further includes: Establishing a fusion formula based on the dynamically adjusted fusion weight; Perform label space mapping on the fusion formula to obtain an inactivated score value for each label; An activation function is used to map the inactivated score values ​​of the labels to a preset interval to obtain the independent prediction probability of each label.

10. A multi-label text classification device based on graph neural network, characterized in that: The device comprises: A multi-level text feature encoding module is used to extract features from the text to be classified based on preset multi-level extraction rules to obtain target text features; A dynamic label embedding learning module is used to construct a label co-occurrence matrix based on the text to be classified, introduce a dynamic weight matrix, and convolve to obtain label features; A cross attention module is used to fuse the target text features and the label features based on a cross attention mechanism to obtain a fused feature; The classification module is used to perform weighted processing on the fusion features to obtain the independent prediction probability of each label.

Images