An extreme multi-label learning method based on space-time network clustering reduction ensemble

Abstract

The present invention discloses an extreme multi-label learning method based on space-time network clustering reduction integration in the field of multi-label text mining technology, comprising the following steps: space-time network attention integration representation; adaptive label relationship enhancement and clustering reduction learning; weighted reduced label set imbalance learning; the present invention integrates the interactive attention among words, phrases and labels in multi-label text, explores the dependency relationship among words, phrases and labels, and effectively improves the extreme multi-label text representation capability; proposes an adaptive label relationship enhancement and clustering reduction learning mechanism, through which the dependency relationship between labels can be effectively mined and the generalization of the model can be improved; through clustering reduction learning, labels of different magnitudes can be effectively adapted to the existing model for training; proposes a weighted reduced label set imbalance learning mechanism, which solves the problems of poor model generalization and scalability caused by label sparsity and imbalance.

Description

Technical Field

[0001] The present invention relates to the technical field of multi-label text mining, and in particular to an extreme multi-label learning method based on spatiotemporal network clustering reduction integration. Background Art

[0002] With the continuous development of internet technology, more and more labels are becoming multi-label. Traditional multi-label learning focuses on a relatively small number of labels. However, with the ever-increasing amount of internet data, the number of labels has exceeded tens of thousands or even millions. Extreme multi-label text classification is a widely used application, such as the classification of short text such as Taobao and web links, which has over ten thousand labels, and Wikipedia text classification, which has over one million labels. When faced with such a large number of labels, traditional multi-label learning methods are no longer applicable. Therefore, addressing the application limitations of traditional multi-label learning, it is of great practical significance to propose methods suitable for extreme multi-label learning.

[0003] Existing multi-label learning models have the following shortcomings:

[0004] (1) The traditional multi-label model cannot adapt to the shortcomings of extreme multi-label scenarios

[0005] Traditional multi-label learning focuses on a relatively small number of labels, such as less than 100. However, with the ever-increasing amount of Internet data, the number of labels has exceeded tens of thousands or millions. Due to the huge number of labels, traditional multi-label learning methods have a high time complexity and cannot adapt to extreme multi-label learning scenarios.

[0006] (2) The disadvantage of low generalization performance of existing extreme multi-label learning models

[0007] Existing extreme multi-label learning methods are mainly based on tree ensembles, embeddings, and deep network end-to-end methods. These methods simply use linear or nonlinear mapping for multi-label learning, ignoring the dependencies between labels. Especially when the number of labels is huge, the labels are usually presented in a graphical form, which makes the generalization performance of existing multi-label learning models low.

[0008] (3) The existing extreme multi-label learning has poor scalability

[0009] Existing extreme multi-label learning methods are mainly developed for specific label levels and cannot adapt to the training of models with different label levels. They also ignore problems such as label sparsity and imbalance, resulting in poor model generalization and scalability.

[0010] Therefore, there is an urgent need to design an extreme multi-label learning method based on spatiotemporal network clustering reduction integration. Summary of the Invention

[0011] The purpose of the present invention is to provide an extreme multi-label learning method based on spatiotemporal network clustering reduction integration to solve the problems raised in the above background technology.

[0012] To achieve the above objectives, the present invention provides the following technical solution: an extreme multi-label learning method based on spatiotemporal network clustering reduction integration, characterized by comprising the following steps:

[0013] S1: Spatiotemporal network attention integrated representation;

[0014] S2: Adaptive label relationship enhancement and cluster reduction learning;

[0015] S3: Weighted reduced label set imbalance learning.

[0016] Furthermore, in the extreme multi-label learning method based on spatiotemporal network clustering reduction integration, the specific steps of S1 are as follows:

[0017] S11: Acquisition of original extreme multi-label data; learning based on the extreme multi-label data acquired for different actual application scenarios;

[0018] S12: phrase-level representation CNN and word-level representation RNN; given a document representation x i ∈R d×n , for phrase-level representation, CNN convolution kernel W can be used i ∈R ωd and the deviation term b i Learn the phrase-level representation of ω-grams, let the vector c i Expressive word (e i-ω+1 ,...,e i ), then the feature p i Expressed as: p i =δ(Conv1D(W i , c i )+b i ); for word-level representation; RNN can be used to learn bidirectional word-level information, expressed as:

[0019] S13: Spatial semantic information representation: Through a hybrid attention mechanism, including a convolutional multi-head self-attention module and a convolutional interactive attention module, we finally obtain a spatial semantic information representation that not only considers the relationship between phrases, but also the relationship between phrases and labels.

[0020] S14: Temporal Semantic Information Representation; Use a hybrid attention mechanism to capture temporal semantic information representation, including a cyclic self-attention module and a cyclic interactive attention module;

[0021] S15: Integrated representation of spatiotemporal network attention.

[0022] Furthermore, in the extreme multi-label learning method based on spatiotemporal network clustering reduction integration, the specific steps of the multi-head self-attention module and the convolutional interactive attention are as follows:

[0023] S131: Specific technical steps of the multi-head self-attention module:

[0024] S1311: Single-head attention calculation; dot-product-based attention, Q∈R 2r×l , K∈R 2r×l and V∈R 2r×l Represent the three embedding matrices of query, key, and value respectively, and the attention output matrix is expressed as:

[0025] S1312: Multi-head attention calculation: Based on the single-head attention calculated in S1311, multi-head attention can be calculated, which can be expressed as:

[0026] P=Muti-head Attention(Q,K,V)=Concat(head1,head2,...,head h )wherehead i =Attention(Q i ,K i ,V i );

[0027] S1313: Convolutional multi-head self-attention calculation; In multi-label text learning, since each document can be assigned to multiple labels, we use a multi-label attention mechanism to focus on different label relationships, based on the matrix P∈R calculated in S1312 2r×l , and finally output multi-label attention S j (j=1,2,...,k) is expressed as:

[0028] S132: Specific technical steps for interactive attention of convolution:

[0029] S1321: Tag graph embedding; Use Node2Vec [6] to generate tag co-occurrence graph vectors to explore the structural information of tags, that is, each tag can be regarded as a node. If any two tags appear in the same document, they are connected by an edge. Based on random walks, high-order tag dependencies are captured through graph embedding technology. Each tag can be expressed as a 2r-dimensional vector, that is, L j ∈R 2r (j=1,2,...k) represents the i-th label, so the entire label embedding is represented as L∈Rk×2r ;

[0030] S1322: Convolutional interactive attention calculation; based on S13, the representation L∈R is obtained k×2r , through the matrix K∈R 2r×l and V∈R 2r×l , the interactive attention of convolution can be expressed as: I1=V×softmax(LK) T ; Matrix S∈R obtained based on S131 2r×k and I1∈R obtained based on S132 2r×k , then S13 can be expressed as: C = Concat(S, I1).

[0031] Furthermore, in the extreme multi-label learning method based on spatiotemporal network clustering reduction integration, the specific steps of the cyclic self-attention module and the cyclic interactive attention module are as follows:

[0032] S141: Cyclic self-attention module; In order to better model the contextual word-level dependencies, a weighted self-attention mechanism is used to focus on different aspects of the document, which can not only learn long-term temporal dependencies but also capture various dense parts of the document. Cyclic self-attention U∈R 2r×k It can be described as: T = tanh(W1H)A = softmax(W2T) T U = HA;

[0033] S142: Cyclic interactive attention module; Similar to the convolutional interactive attention, the interactive attention is introduced to capture fine-grained word-level signals and calculate the matching score between words and labels. The cyclic interactive attention can be described as The matrix U∈R obtained based on S141 2r×k and I2∈R obtained by S142 2r×k , then S14 can be expressed as: R = Concat(U, I2).

[0034] Furthermore, in the extreme multi-label learning method based on spatiotemporal network clustering reduction integration, the specific steps of S2 are as follows:

[0035] S21: Label tree clustering; by summing the inner product of the sparse text features containing the label and the label text features, and then normalizing them to obtain the feature representation of each label, and then using balanced k-means (k = 2) for recursive clustering, iterate until the following conditions are met: given the maximum number of labels for each cluster, the labels are required to be divided into S clusters, and the number of labels contained in each label cluster is less than the maximum number of labels or greater than half of the maximum number of labels; when S clusters are obtained, based on the representation M obtained by the S1 module, M can be mapped to an S-dimensional vector P through a fully connected layer;

[0036] S22: Label relationship enhancement: By conveying the label relationship, a bottleneck layer is added on top of the original prediction P to adaptively achieve label enhancement, which can be formally described as:

[0037] S23: Cluster reduction learning; the S cluster indexes obtained based on the S21 label tree clustering are represented as y S ∈{0,1} S , and the representation obtained by label relationship enhancement An adaptive clustering reduction learning mechanism is proposed, which is described as:

[0038]

[0039] Furthermore, in the extreme multi-label learning method based on spatiotemporal network clustering reduction integration, the specific steps of S3 are as follows:

[0040] S31: Reduced label set embedding; Based on the k clusters obtained in S21, we can get a reduced label set U, and based on the representation M obtained in S1, we can get a reduced label set embedding vector Q, which is described as: Q = σ(W Q M+b Q );

[0041] S32: Weighted imbalanced learning; Based on the k clusters obtained in S21, we can get the reduced set U of labels, and find the labels y corresponding to these labels. U ∈{0,1} U ,To solve the multi-label imbalance problem, a weighted imbalance loss is used for learning, which is described as:

[0042]

[0043] in is the true label corresponding to the sample in the reduced label set U, Qk is the embedding obtained using S31, γ+ and γ- express the contribution of different positive and negative sample weights. Usually, γ->γ+, usually set γ+ to 0 and γ- to 1;

[0044] Therefore, based on the above three modules, an extreme multi-label learning method based on spatiotemporal network clustering reduction integration is proposed. The overall model training can be described as:

[0045] L=L S +L Q

[0046] Furthermore, in the extreme multi-label learning method based on space-time network clustering reduction integration, the specific steps of S15 are: based on C∈R obtained in S13 2r×k and R∈R obtained based on S14 2r×k , an adaptive weighted integration strategy is proposed, which first uses l2 normalization C∈R 2r×k and R∈R 2r×k , and then transform it into weights α∈R through an MLP layer and a fully connected layer k×1 and β∈R k×1 , which can be expressed as: Finally, the final spatial-temporal network attention integration representation is obtained by normalizing the weights, which is described as: M=α×C+β×R, and finally the spatial-temporal network attention integrated representation M is obtained.

[0047] Compared with the prior art, the present invention has the following beneficial effects:

[0048] 1. Space-time network attention integrated representation: This paper proposes an extreme multi-label text representation method based on space-time network attention integration, which integrates the interactive attention between words, phrases, and labels in multi-label text, explores the dependency relationship between words, phrases, and labels, and effectively improves the extreme multi-label text representation capability. Compared with existing extreme multi-label text representation methods, the space-time network attention integrated representation proposed in this paper is not only compatible with existing CNN and RNN text representation networks, but also simultaneously considers the interactive attention between words, phrases, and labels in multi-label text, and explores the dependency relationship between words, phrases, and labels.

[0049] 2. Adaptive label relationship enhancement and cluster simplification learning: The present invention proposes an adaptive label relationship enhancement and cluster simplification learning mechanism. Through adaptive label relationship enhancement, the dependency relationship between labels can be effectively mined and the generalization of the model can be improved. Through cluster simplification learning, labels of different magnitudes can be effectively adapted to the existing model for training, solving the problems of time and space consumption caused by huge labels. Although the existing extreme multi-label learning method reduces the time consumption caused by huge labels by simplifying labels, the reduction in the number of labels also leads to a reduction in accuracy. The adaptive label relationship enhancement and cluster simplification learning mechanism proposed in the present invention solves the time and space consumption caused by huge labels through label tree clustering, and then adaptively enhances the labels based on a simple label relationship enhancement module. Finally, the accuracy of the clustered label cluster is improved based on cluster simplification learning. While ensuring low time and space consumption caused by the reduction in the number of labels, the accuracy of the clustered label cluster is also guaranteed.

[0050] 3. Weighted reduced label set imbalance learning: The present invention proposes a weighted reduced label set imbalance learning mechanism, which solves the problems of poor model generalization and scalability caused by label sparsity and imbalance, and effectively improves the accuracy of extreme multi-label text learning. Existing extreme multi-label learning either does not consider the label imbalance problem, or considers the label imbalance on the original label set. However, due to the huge amount of original labels, the imbalance is large and the model learning effect is not good. The weighted reduced label set imbalance learning mechanism proposed in the present invention has two advantages: First, to prevent the lack of spatial-temporal network attention integration representation in imbalanced learning, the reduced label set here not only integrates the S2 module, but also considers the S1 module at the same time, which effectively guarantees the representation quality of imbalanced learning; second, by using two parameters and expressing the contribution between different positive and negative sample weights, imbalanced learning is performed on the reduced label set, which enables the model to better learn the impact of imbalance. BRIEF DESCRIPTION OF THE DRAWINGS

[0051] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative work.

[0052] Figure 1 This is a flow chart of the extreme multi-label learning method based on spatiotemporal network clustering reduction integration of the present invention;

[0053] Figure 2 This is a schematic diagram of the spatiotemporal network attention integration representation submodule of the present invention;

[0054] Figure 3 Schematic diagram of the adaptive tag relationship enhancement module of the present invention;

[0055] Figure 4 This is a schematic diagram of the test results of extreme multi-label learning based on spatiotemporal network clustering reduction integration of the present invention;

[0056] Figure 5 Schematic diagram of the training accuracy change curve of the two parts of the Amazon-670K dataset adaptive label relationship enhancement and cluster reduction learning and weighted reduced label set imbalance learning of the present invention;

[0057] Description of the drawings: Figure 2 (a) Convolutional multi-head self-attention module; (b) interactive attention module; (c) recurrent self-attention module; Figure 5 (a) Spatial semantic information representation; (b) temporal semantic information representation; (c) spatiotemporal network attention integration representation. DETAILED DESCRIPTION

[0058] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making any creative efforts shall fall within the scope of protection of the present invention.

[0059] The present invention provides a technical solution: an extreme multi-label learning method based on spatiotemporal network clustering reduction integration, comprising the following steps:

[0060] S1: Spatiotemporal network attention integrated representation;

[0061] S11: Acquisition of original extreme multi-label data; learning based on the extreme multi-label data acquired for different actual application scenarios;

[0062] S12: phrase-level representation CNN and word-level representation RNN; given a document representation x i ∈R d×n , for phrase-level representation, CNN convolution kernel W can be used i ∈R ωd and the deviation term b i Learn the phrase-level representation of ω-grams, let the vector c i Expressive word (e i-ω+1 ,...,e i ), then the feature p i Expressed as: p i =δ(Conv1D(W i , c i )+b i ); for word-level representation; RNN can be used to learn bidirectional word-level information, expressed as:

[0063] S13: Spatial semantic information representation: Through a hybrid attention mechanism, including a convolutional multi-head self-attention module and a convolutional interactive attention module, we finally obtain a spatial semantic information representation that not only considers the relationship between phrases, but also the relationship between phrases and labels.

[0064] S131: Specific technical steps of the multi-head self-attention module:

[0065] S1311: Single-head attention calculation; dot-product-based attention, Q∈R 2r×l , K∈R 2r×l and V∈R 2r×l Represent the three embedding matrices of query, key, and value respectively, and the attention output matrix is expressed as:

[0066] S1312: Multi-head attention calculation: Based on the single-head attention calculated in S1311, multi-head attention can be calculated, which can be expressed as:

[0067] P=Muti-head Attention(Q,K,V)=Concat(head1,head2,...,head h )wherehead i =Attention(Q i ,K i ,V i );

[0068] S1313: Convolutional multi-head self-attention calculation; In multi-label text learning, since each document can be assigned to multiple labels, we use a multi-label attention mechanism to focus on different label relationships, based on the matrix P∈R calculated in S1312 2r×l , and finally the output multi-label attention Sj (j = 1, 2, ..., k) is expressed as:

[0069] S132: Specific technical steps for interactive attention of convolution:

[0070] S1321: Tag graph embedding; Use Node2Vec [6] to generate tag co-occurrence graph vectors to explore the structural information of tags, that is, each tag can be regarded as a node. If any two tags appear in the same document, they are connected by an edge. Based on random walks, high-order tag dependencies are captured through graph embedding technology. Each tag can be expressed as a 2r-dimensional vector, that is, L j ∈R 2r (j=1,2,...k) represents the i-th label, so the entire label embedding is represented as L∈R k×2r ;

[0071] S1322: Convolutional interactive attention calculation; based on S13, the representation L∈R is obtained k×2r , through the matrix K∈R 2r×l and V∈R 2r×l , the interactive attention of convolution can be expressed as: I1=V×softmax(LK) T ; Matrix S∈R obtained based on S131 2r×k and I1∈R obtained based on S132 2r×k , then S13 can be expressed as: C = Concat(S, I1).

[0072] S14: Temporal Semantic Information Representation; Use a hybrid attention mechanism to capture temporal semantic information representation, including a cyclic self-attention module and a cyclic interactive attention module;

[0073] S141: Cyclic self-attention module; In order to better model the contextual word-level dependencies, a weighted self-attention mechanism is used to focus on different aspects of the document, which can not only learn long-term temporal dependencies but also capture various dense parts of the document. Cyclic self-attention U∈R 2r×k It can be described as: T = tanh(W1H)A = softmax(W2T) T U = HA;

[0074] S142: Cyclic interactive attention module; Similar to the convolutional interactive attention, the interactive attention is introduced to capture fine-grained word-level signals and calculate the matching score between words and labels. The cyclic interactive attention can be described as The matrix U∈R obtained based on S141 2r×k and I2∈R obtained by S142 2r×k , then S14 can be expressed as: R = Concat(U, I2).

[0075] S15: Space-time network attention integrated representation. Based on C∈R obtained in S13 2r×k and R∈R obtained based on S14 2r ×k , an adaptive weighted integration strategy is proposed, which first uses l2 normalization C∈R 2r×k and R∈R 2r×k , and then transform it into weights α∈R through an MLP layer and a fully connected layer k×1 and β∈R k×1 , which can be expressed as: Finally, the final spatial-temporal network attention integration representation is obtained by normalizing the weights, which is described as: M=α×C+β×R, and finally the spatial-temporal network attention integrated representation M is obtained.

[0076] S2: Adaptive label relationship enhancement and cluster reduction learning;

[0077] S21: Label tree clustering; by summing the inner product of the sparse text features containing the label and the label text features, and then normalizing them to obtain the feature representation of each label, and then using balanced k-means (k = 2) for recursive clustering, iterate until the following conditions are met: given the maximum number of labels for each cluster, the labels are required to be divided into S clusters, and the number of labels contained in each label cluster is less than the maximum number of labels or greater than half of the maximum number of labels; when S clusters are obtained, based on the representation M obtained by the S1 module, M can be mapped to an S-dimensional vector P through a fully connected layer;

[0078] S22: Label relationship enhancement: By conveying the label relationship, a bottleneck layer is added on top of the original prediction P to adaptively achieve label enhancement, which can be formally described as:

[0079] S23: Cluster reduction learning; the S cluster indexes obtained based on the S21 label tree clustering are represented as y S ∈{0,1} S , and the representation obtained by label relationship enhancement An adaptive clustering reduction learning mechanism is proposed, which is described as:

[0080]

[0081] S3: Weighted reduced label set imbalance learning.

[0082] S31: Reduced label set embedding; Based on the k clusters obtained in S21, we can get a reduced label set U, and based on the representation M obtained in S1, we can get a reduced label set embedding vector Q, which is described as: Q = σ(W Q M+b Q );

[0083] S32: Weighted imbalanced learning; Based on the k clusters obtained in S21, we can get the reduced set U of labels, and find the labels y corresponding to these labels. U ∈{0,1} U ,To solve the multi-label imbalance problem, a weighted imbalance loss is used for learning, which is described as:

[0084]

[0085] in is the true label corresponding to the sample in the reduced label set U, Q k is the embedding obtained using S31, γ+ and γ- express the contribution of different positive and negative sample weights. Usually, γ->γ + , usually set γ +is 0, γ- is 1;

[0086] Therefore, based on the above three modules, an extreme multi-label learning method based on spatiotemporal network clustering reduction integration is proposed. The overall model training can be described as:

[0087] L=L S +L Q

[0088] Table 1 Related terms

[0089]

[0090] Throughout this specification, references to terms such as "one embodiment," "example," or "specific example" indicate that the specific features, structures, materials, or characteristics described in conjunction with that embodiment or example are included in at least one embodiment or example of the present invention. In this specification, schematic representations of these terms do not necessarily refer to the same embodiment or example. Furthermore, the specific features, structures, materials, or characteristics described may be combined in any suitable manner in any one or more embodiments or examples.

[0091] The preferred embodiments of the present invention disclosed above are intended only to help illustrate the present invention. These preferred embodiments do not exhaustively describe all details, nor do they limit the present invention to the specific embodiments described. Obviously, many modifications and variations are possible based on the content of this specification. These embodiments are selected and described in detail in this specification to better explain the principles and practical applications of the present invention, thereby enabling those skilled in the art to better understand and utilize the present invention. The present invention is limited only by the claims and their full scope and equivalents.

Claims

1. An extreme multi-label learning method based on spatiotemporal network clustering reduction integration, characterized by: The following steps are involved: S1, spatial-temporal network attention integrated representation; The specific steps are: S11. Acquisition of original extreme multi-label data; learning based on the extreme multi-label data acquired for different actual application scenarios; S12, phrase-level representation CNN and word-level representation RNN; given document representation For phrase-level representation, use CNN convolution kernel and the deviation term b i Learn the phrase-level representation of ω-grams, let the vector c i Expressive word (e i-ω+1 ,...,e i ), then the feature p i Expressed as: p i =δ(Conv1D(W i , c i )+b i ); For word-level representation, RNN is used to learn bidirectional word-level information, which is expressed as: S13, spatial semantic information representation; through a hybrid attention mechanism, including a convolutional multi-head self-attention module and a convolutional interactive attention module, a spatial semantic information representation is finally obtained. This representation considers the relationship between phrases and the relationship between phrases and labels; S14, temporal semantic information representation; using a hybrid attention mechanism to capture temporal semantic information representation, including a recurrent self-attention module and a recurrent interactive attention module; S15, spatial-temporal network attention integrated representation; based on S13 and obtained based on S14 An adaptive weighted integration strategy is proposed, first using l2 normalization and Then it is converted to weights through an MLP layer and a fully connected layer. and Expressed as: Finally, the final spatial-temporal network attention integration representation is obtained by normalizing the weights, which is described as: M =α× C +β× R , obtain the spatial-temporal network attention integrated representation M; S2, adaptive label relationship enhancement and cluster reduction learning; the specific steps are: S21. Label tree clustering: The feature representation of each label is obtained by summing the inner product of the sparse text features containing the label and the label text features, and then normalizing them. Then, balanced k-means is used with k = 2 for recursive clustering. The iteration is performed until the following condition is met: given the maximum number of labels per cluster, the labels are required to be divided into S clusters, and the number of labels contained in each label cluster is less than the maximum number of labels or greater than half of the maximum number of labels. When S clusters are obtained, the representation M obtained based on the S1 module is mapped to an S-dimensional vector P through a fully connected layer; S22, label relationship enhancement; By conveying the label relationship, a bottleneck layer is added to the original prediction P to adaptively achieve label enhancement, which can be formally described as: S23, cluster reduction learning; The S cluster indexes obtained based on S21 label tree clustering are represented as y S ∈{0,1} S , and the representation obtained by label relationship enhancement An adaptive clustering reduction learning mechanism is proposed, which is described as: S3. Weighted reduced label set imbalance learning; the specific steps are: S31, simplified label set embedding; Based on the k clusters obtained in S21, a reduced label set U is obtained, and based on the representation M obtained in S1, a reduced label set embedding vector Q is obtained, which is described as: Q = σ(W Q M+b Q ); S32, weighted imbalanced learning; Based on the k clusters obtained by S21, we get the reduced set U of labels and find the labels y corresponding to these labels. U ∈{0,1} U ; Using weighted imbalance loss for learning, described as: in is the true label corresponding to the sample in the reduced label set U, Q k is the embedding obtained using S31, γ + and γ - Expresses the contribution of different positive and negative sample weights. Usually, γ - >γ+, usually set γ + is 0, γ - is 1; The overall model training is described as:

2. The extreme multi-label learning method based on spatiotemporal network clustering reduction integration according to claim 1, characterized in that: The specific steps of the multi-head self-attention module and the convolutional interactive attention are: S131. Specific technical steps for the multi-head self-attention module: S1311, single-head attention calculation; attention based on dot product, and Represent the three embedding matrices of query, key, and value respectively, and the attention output matrix is expressed as: S1312, multi-head attention calculation: Based on the single-head attention calculated in S1311, calculate the multi-head attention, expressed as: P=Muti-head Attention(Q,K,V)=Concat(head1,head2,...,head h )where head i =Attention(Q i ,K i ,V i ); S1313, convolutional multi-head self-attention calculation; in multi-label text learning, each document can be assigned to multiple labels, and the multi-label attention mechanism is used to focus on different label relationships. Based on the matrix calculated in S1312 Finally, output the multi-label attention S j (j=1,2,...,k) is expressed as: S132: Specific technical steps for interactive attention of convolution: S1321: Tag graph embedding; Use Node2Vec to generate tag co-occurrence graph vectors to explore the structural information of tags, that is, each tag is regarded as a node. If any two tags appear in the same document, they are connected by an edge. Based on random walks, high-order tag dependencies are captured through graph embedding technology. Each tag can be expressed as a 2r-dimensional vector, that is represents the i-th label, so the entire label embedding is expressed as S1322: Convolutional interactive attention calculation; based on S13, the representation is obtained Through the matrix and The interactive attention of convolution is expressed as: I1 = V × softmax (LK) T ; Matrix obtained based on S131 and based on S132 Then S13 is expressed as: C=Concat(S,I1).

3. The extreme multi-label learning method based on spatiotemporal network clustering reduction integration according to claim 1, characterized in that: The specific steps of the cyclic self-attention module and the cyclic interactive attention module are: S141, cyclic self-attention module; use weighted self-attention mechanism to focus on different aspects of the document, learn long-term temporal dependencies, capture various dense parts of the document, cyclic self-attention It can be described as: T = tanh(W1H)A = softmax(W2T) T U = HA; S142, recurrent interactive attention module; Similar to the interactive attention of convolution, interactive attention is introduced to capture fine-grained word-level signals and calculate the matching score between words and labels. The cyclic interactive attention is described as Matrix obtained based on S141 and S142 Then S14 is expressed as: R=Concat(U,I2).

Images