Multi-label classification methods that incorporate higher-order label correlation

Abstract

This invention discloses a multi-label classification method that integrates higher-order label correlations. The steps include: using a data-driven approach to mine label co-occurrence information in the dataset to define the correlations between labels, and constructing a label hypergraph; for image classification tasks, using a deep residual network (ResNet) as the backbone network to extract image features; performing hypergraph convolution operations on the constructed label hypergraph using hypergraph convolutional layers to extract higher-order correlations between nodes in the hypergraph, obtaining label embedding vectors; sequentially embedding the label embedding vectors output from each hypergraph convolutional layer into the connection points of the backbone feature extraction network, and simultaneously using a LC operation to fuse the label embedding vectors with the ResNet feature vectors at the connection points of the backbone feature extraction network, thereby using higher-order label correlations to guide the learning of image features. This invention can perceive and utilize higher-order label correlations during image feature learning, thereby achieving multi-label classification.

Description

Technical Field

[0001] This invention relates to the field of multi-label classification technology, specifically a multi-label classification method that integrates higher-order label correlations. Background Technology

[0002] With the development of information technology, internet data and resources are characterized by massive volumes. To effectively manage and utilize this vast amount of information, content-based information retrieval and data mining have gradually become areas of great interest. As the volume of data continues to increase, the complexity of data annotation structures is also growing. Research results on single-label data classification can no longer meet the needs of technological development, and the importance of multi-label classification is becoming increasingly prominent. The applications of this technology are also constantly expanding, such as semantic annotation of images and videos, genomics, music sentiment classification, and marketing guidance, which have become a hot research topic today.

[0003] In recent years, a significant trend in multi-label learning research has been the optimization of the learning process by effectively utilizing the correlations between different labels. This correlation mining has been widely recognized in academia and proven to be an effective strategy for significantly improving multi-label classification performance. Currently, a large number of studies attempt to introduce graph structures to capture label relationships, thereby leveraging label correlations to help build classification models. Based on the constructed label graph, some methods use Graph Convolutional Networks (GCNs) to encode label correlations. For example, the ML-GCN method extracts statistical knowledge from label co-occurrence information to construct a directed graph structure of labels, and uses GCNs to map the initial label word embedding vectors to a set of interdependent object classifiers. In the final classification stage, this set of classifiers is directly applied to image features for classification. The ML-GCN method combines CNN image feature recognition networks and GCN graph convolutional neural networks, belonging to the traditional CNN-GCN framework. The KSSNet method improves upon ML-GCN by adding external prior knowledge to the label graph based on label co-occurrence to construct the final label graph, while simultaneously injecting label information into various stages of the CNN backbone network, thereby improving feature learning.

[0004] However, in existing methods based on the CNN-GCN framework, graph structures are used to characterize pairwise relationships between labels. Since each edge in a graph can only connect two vertices, this inherent structural characteristic limits its ability to model higher-order correlations, thus restricting its capacity to capture complex associations between labels. This results in the CNN-GCN framework lacking the ability to represent higher-order label correlations. Furthermore, in the process of modeling label correlations, the aforementioned methods generally employ learning methods based on ordinary graphs. While these methods can capture pairwise relationships between labels to some extent, they neglect the higher-order relationships that may exist between labels. Higher-order label correlations, as a key factor influencing multi-label classification, are complex and important. Therefore, in order to more accurately understand and apply the relationships between labels, in-depth research and focused attention on higher-order label correlations are particularly important. Summary of the Invention

[0005] The purpose of this invention is to provide a multi-label classification method that integrates higher-order label correlations. By embedding labels from different stages of a hypergraph neural network into a feature extraction network, the network can perceive and utilize higher-order label correlations during image feature learning, thereby achieving multi-label classification.

[0006] To solve the above-mentioned technical problems, the present invention provides the following technical solution:

[0007] A multi-label classification method that integrates higher-order label relevance includes the following steps:

[0008] A data-driven approach is used to mine label co-occurrence information in the dataset to define the correlation between labels and construct a label hypergraph.

[0009] For image classification tasks, a deep residual network (ResNet) is selected as the backbone extraction network;

[0010] Hypergraph convolution operations are performed on the constructed label hypergraph using hypergraph convolutional layers to extract higher-order correlations between nodes in the hypergraph, thus obtaining label embedding vectors.

[0011] The label embedding vectors output from each hypergraph convolutional layer are sequentially embedded into the connection points of the backbone feature extraction network. At the same time, the LC operation is used to fuse the label embedding vectors with the ResNet feature vectors at the connection points of the backbone feature extraction network, thereby using high-order label correlation to guide the learning of image features.

[0012] According to the above technical solution, the label hypergraph construction steps include:

[0013] The co-occurrence frequency of each label pair in the dataset is counted to obtain the co-occurrence frequency matrix M∈R. q×q, where q is the total number of all tags, and the co-occurrence matrix M is a symmetric matrix;

[0014] Count the number of times each label is labeled in the sample to obtain a one-dimensional array of label counts N∈R. 1×q , where N i Indicates label l i Number of times the data is labeled in the sample;

[0015] Based on the co-occurrence frequency matrix M and the labeled frequency array N, the probability matrix P is calculated using the conditional probability formula.

[0016] Based on the probability matrix P, a hypergraph association matrix H∈R is constructed using the nearest neighbor search method. q×n In the hypergraph incidence matrix H, rows represent labeled vertices, columns represent hyperedges, and p and n represent the number of vertices and the number of hyperedges, respectively.

[0017] Add an identity matrix I to H q ∈R q×q The hypergraph correlation matrix H is obtained. L H L =H+I q .

[0018] According to the above technical solution, the conditional probability formula is as follows:

[0019]

[0020] The formula for calculating the element at position (i,j) in the probability matrix P is as follows:

[0021]

[0022] Wherein, P(l i |l j ) indicates that when the label l j When it appears, the tag l i The conditional probability of occurrence, P(l) i I l j ) indicates label l i and tag l j The probability of them occurring simultaneously, P(l) j ) indicates label l j The probability of occurrence in the training set, i,j=1,2,K,q,M ij Indicates label l i and l j The number of co-occurrences in the training set, where m is the number of training samples in the training set, and N is the number of co-occurrences. j Indicates label l j The number of times the data is labeled in the training set samples, [P] ijThis represents the element at position (i,j) in the probability matrix P, whose value corresponds to P(l i |l j ).

[0023] According to the above technical solution, the label hypergraph uses the conditional probability of label co-occurrence as a metric for searching the nearest neighbor vertex. That is, each hyperedge selects a vertex as the centroid vertex and connects K nearest neighbor vertices with the conditional probability value.

[0024] The formula for defining the elements of the hypergraph incidence matrix H is as follows:

[0025] ;

[0026] Where Nv(j) represents vertex v j Let the set of K nearest neighbors be i = 1, 2, K, q, j = 1, 2, K, n, [H] ij This represents the element position of the hypergraph incidence matrix H; due to the use of the nearest neighbor search method, a corresponding hyperedge is generated for each vertex, such that q = n;

[0027] When vertex v i Located at v j In the nearest neighbor set, the value at the corresponding position in the hypergraph association matrix H is set to the conditional probability value of the labels corresponding to the two vertices, otherwise it is set to 0.

[0028] According to the above technical solution, the propagation formula between the hypergraph convolutional layers is as follows:

[0029]

[0030] Among them, E (l) Let E be the label embedding vector of the l-th layer, and let E be the initial label embedding. (0) Words can be obtained using word embedding techniques such as GloVe and FastText; H is the hypergraph association matrix; D v and D e H represents the diagonal matrices representing the vertex degree and hyperedge degree of the hypergraph, respectively; T Let W be the transpose of the hypergraph incidence matrix, and let W = diag(w1, w2, K, w n ) represents the weight values ​​of the n hyperedges, W is initially set to the identity matrix I, σ(·) is a non-linear activation function, and θ (l) f represents the weight of the l-th layer. hg (.) represents the function for hypergraph convolution operation, E (l+1) This is the label embedding vector for the (l+1)th layer.

[0031] According to the above technical solution, the backbone feature extraction network adopts the ResNet-101 network; connection points are set between Conv_2, Conv_3, Conv_4 and Conv_5 in the ResNet-101 network.

[0032] According to the above technical solution, the LC operation steps include:

[0033] During the LC operation between the label embedding vector and the ResNet feature vector, the tensor shapes of the label embedding vector and the ResNet feature vector are adjusted to ensure that they remain consistent in dimension.

[0034] The relationship between ResNet feature vectors and label embedding vectors with consistent dimensions is calculated by tensor multiplication, and then the relevant tensors are mapped to the hidden space by 1x1x1 convolution to encode the relationship between ResNet features and label embeddings.

[0035] The relation tensor generated by the 1×1×1 convolution is added to the original ResNet feature tensor to form higher-order label features.

[0036] According to the above technical solution, the formula for the LC operation is:

[0037]

[0038] Where, x (l) The ResNet feature at the l-th connection point, The fused features are obtained after the LC operation, where q is the number of labels, c is the number of channels, h and w represent the height and width of the feature tensor, and E is the value of E. (l) ∈R q×c Let g be the label embedding vector in the convolutional layer, and g(·) represent a 1×1×1 convolution function. The matrix multiplication operation is represented by (·). T Indicates the transpose operation; R q×h×w (·) represents the tensor dimension adjustment operation, used to adjust the input tensor according to the dimensions of the number of labels q, the feature tensor height h, and the feature tensor width w. σ(·) is a non-linear activation function, R hw×c This represents the dimensions of the feature tensor height h, feature tensor width w, and number of channels c after the tensor dimension adjustment operation.

[0039] Compared with the prior art, the beneficial effects achieved by the present invention are as follows: The present invention combines hypergraph neural network with ResNet feature extraction network. By embedding labels from different stages in the hypergraph neural network into the feature extraction network, it can perceive and utilize higher-order label correlations during image feature learning. Compared with the traditional CNN-GCN framework, the present invention can effectively integrate higher-order label correlations and shows superior performance in multi-label image classification. Attached Figure Description

[0040] The accompanying drawings are provided to further illustrate the invention and form part of the specification. They are used in conjunction with embodiments of the invention to explain the invention and do not constitute a limitation thereof. In the drawings:

[0041] Figure 1 This is a flowchart illustrating the steps of the multi-label classification method that integrates higher-order label correlation in this invention.

[0042] Figure 2 This is the pseudocode for the implementation of the label hypergraph construction method in the example. Detailed Implementation

[0043] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.

[0044] The present invention provides the following technical solution:

[0045] Example 1

[0046] A multi-label classification method that integrates higher-order label relevance includes the following steps:

[0047] S1. Employ a data-driven approach to mine label co-occurrence information in the dataset to define the correlation between labels and construct a label hypergraph. The label hypergraph construction steps include:

[0048] S101. Count the co-occurrence frequency of each label pair in the dataset to obtain the co-occurrence frequency matrix M∈R. q×q , where q is the total number of tags;

[0049] S102. Count the number of times each label is labeled in the sample to obtain a one-dimensional array N∈R. 1×q , where N i Indicates label l i Number of times the data is labeled in the sample;

[0050] S103. Calculate the probability matrix P using the conditional probability formula based on the co-occurrence frequency matrix M. L The conditional probability formula is as follows:

[0051]

[0052] The formula for calculating the element at position (i,j) in the probability matrix P is as follows:

[0053]

[0054] Wherein, P(l i |l j ) indicates that when the label l j When it appears, the tag l i The conditional probability of occurrence, P(l) i I l j ) indicates label l i and tag l j The probability of them occurring simultaneously, P(l) j ) indicates label l j The probability of occurrence in the training set, i,j=1,2,K,q,M ij Indicates label l i and l j The number of co-occurrences in the training set, where m is the number of training samples in the training set, and N is the number of co-occurrences. j Indicates label l j The number of times the data is labeled in the training set samples, [P] ij This represents the element at position (i,j) in the probability matrix P, whose value corresponds to P(l i |l j ).

[0055] S104, Based on probability matrix P L Construct the association matrix H∈R using the nearest neighbor search method q×n The correlation matrix H consists of rows representing labeled vertices and columns representing hyperedges. p and n represent the number of vertices and the number of hyperedges, respectively. The labeled hypergraph uses the conditional probability of label co-occurrence as a metric for searching nearest neighbor vertices. That is, each hyperedge selects a vertex as its centroid and connects K nearest neighbor vertices with the conditional probability value.

[0056] S105. Add an identity matrix I to H. q ∈R q×q The hypergraph correlation matrix H is obtained. L ;

[0057] The element definition formula for the hypergraph incidence matrix H is as follows:

[0058] ;

[0059] Where Nv(j) represents vertex v j Let the set of K nearest neighbors be i = 1, 2, K, q, j = 1, 2, K, n, [H] ij This represents the element position of the hypergraph incidence matrix H; due to the use of the nearest neighbor search method, a corresponding hyperedge is generated for each vertex, such that q = n;

[0060] When vertex v i Located at v j In the nearest neighbor set, the value at the corresponding position of the association matrix H is set to the conditional probability value of the labels corresponding to the two vertices, otherwise it is set to 0.

[0061] Examples of building a label hypergraph include... Figure 2 As shown: Based on the label co-occurrence frequency matrix The conditional probability matrix is ​​calculated from the label annotation frequency matrix N = [2530351822]. Then, hyperedge e1 will connect other vertices centered on vertex v1, using conditional probability values ​​as distance indicators. When K is 2, the two nearest neighbors of vertex v1 are vertex v2 with a probability of 0.8 and vertex v5 with a probability of 0.48. Therefore, the hypergraph incidence matrix H... L In column e1, the element values ​​for v2 and v5 are 0.8 and 0.48 respectively, while the element values ​​for the remaining positions are set to 0. (Hypergraph Intent Matrix)

[0062]

[0063] S2. For the image classification task, a deep residual network (ResNet) is selected as the backbone network to extract the image features to be identified; the backbone feature extraction network adopts the ResNet-101 network; connection points are set between Conv_2, Conv_3, Conv_4 and Conv_5 in the ResNet-101 network.

[0064] S3. On the constructed label hypergraph, perform hypergraph convolution operations using hypergraph convolutional layers to extract higher-order correlations between nodes in the hypergraph, obtaining label embedding vectors; where the propagation formula between hypergraph convolutional layers is:

[0065]

[0066] Among them, E (l) Let E be the label embedding vector of the l-th layer, and let E be the initial label embedding. (0) It can be obtained through word embedding technology; H is the hypergraph association matrix; D v and D e H represents the diagonal matrices representing the vertex degree and hyperedge degree of the hypergraph, respectively;T Let W be the transpose of the hypergraph incidence matrix, and let W = diag(w1, w2, K, w n ) represents the weight values ​​of the n hyperedges, W is initially set to the identity matrix I, σ(·) is a non-linear activation function, and θ (l) f represents the weight of the l-th layer. hg (.) represents the function for hypergraph convolution operation, E (l+1) This is the label embedding vector for the (l+1)th layer.

[0067] S4. The label embedding vectors output from each hypergraph convolutional layer are sequentially embedded into the connection points of the backbone feature extraction network. Simultaneously, the LC operation is used to fuse the label embedding vectors with the ResNet feature vectors at the connection points of the backbone feature extraction network, thereby leveraging higher-order label correlation to guide the learning of image features. The LC operation specifically includes:

[0068] S401. During the LC operation between the label embedding vector and the ResNet feature vector, the tensor shapes of the label embedding vector and the ResNet feature vector are adjusted to ensure that they remain consistent in dimension.

[0069] S402. The relationship between ResNet feature vectors and label embedding vectors that are consistent in dimension is calculated by tensor multiplication, and then the relevant tensors are mapped to the hidden space by 1x1x1 convolution to encode the relationship between ResNet features and label embeddings.

[0070] S403. Add the relation tensor generated by the 1×1×1 convolution to the original ResNet feature tensor to form higher-order label features.

[0071] The formula for LC operation is as follows:

[0072]

[0073] Where, x (l) The ResNet feature at the l-th connection point, The fused features are obtained after the LC operation, where q is the number of labels, c is the number of channels, h and w represent the height and width of the feature tensor, and E is the value of E. (l) ∈R q×c Let g be the label embedding vector in the convolutional layer, and g(·) represent a 1×1×1 convolution function. The matrix multiplication operation is represented by (·). T Indicates the transpose operation; R q×h×w (·) represents the tensor dimension adjustment operation, used to adjust the input tensor according to the dimensions of the number of labels q, the feature tensor height h, and the feature tensor width w. σ(·) is a non-linear activation function, Rhw×c This represents the dimensions of the feature tensor height h, feature tensor width w, and number of channels c after the tensor dimension adjustment operation.

[0074] Example 2

[0075] The pseudocode for the implementation of the tag hypergraph construction method is as follows: Figure 2 As shown, where m is the number of samples in the multi-label dataset, q is the number of labels, and I q This represents the identity matrix with q rows and q columns.

[0076] Figure 2 Lines 5 to 16 in the code calculate the label co-occurrence matrix M and the label annotation frequency matrix N. The time complexity of calculating M and N is O(m·q). 2 Lines 17 to 21 calculate the probability matrix P. The time complexity of solving the probability matrix P is O(q). 2 Lines 22 to 34 are used to calculate the hypergraph incidence matrix H. L In solving the hypergraph incidence matrix H L The time complexity of the process is O(q(q+qlogq+K)) = O(q 2 +q 2 logq + qK) = O(q 2 Therefore, the total time complexity of the label hypergraph construction method is O(m·q). 2 +q 2 The total space complexity is O(q). 2 ).

[0077] Example 3

[0078] On the COCO2004 dataset, the multi-label classification method that incorporates high-order label correlation (HOLC-MLC) was compared with CNN-RNN, SRN, Multi-Evidence, ML-GCN, and KSSNet in terms of mAP, CP, CR, CF1, and OP. The experimental results of the six models on the COCO2014 validation set are shown in Table 1.

[0079] Table 1. Experimental results of the six models on the COCO2014 validation set.

[0080]

[0081]

[0082] As shown in Table 1, our method achieved the best performance in three of the five evaluation metrics, with mAP, OP, and CP scores of 85.6%, 88.3%, and 85.3%, respectively. Compared to KSSNet, which has the second-best performance, the HOLC-MLC method demonstrates a significant advantage.

[0083] The parameter settings for HOLC-MLC are as follows:

[0084] HOLC-MLC uses ResNet-101 as the backbone network for image feature extraction; connection points are set between Conv_2, Conv_3, Conv_4 and Conv_5 in the ResNet-101 network.

[0085] During the construction of the label hypergraph, K was set to 5. During model training, the input image was randomly cropped and resized to 448×448, and randomly horizontally flipped for data augmentation. During testing, the test image was simply resized to 448×448 for evaluation. For network optimization, SGD was used as the optimizer; LeakyReLU was used as the non-linear activation function for the hypergraph convolutional layers with a slope of -0.2; the initial learning rate was set to 0.01; the learning rate decayed by a factor of 10 every 40 epochs, and the network was trained for a total of 100 epochs.

[0086] The label hypergraph uses three hypergraph convolutions to extract higher-order correlations between nodes in the hypergraph and obtains the label embedding vector; the number of output channels of the three consecutive hypergraph convolution layers are 256, 512 and 1024, respectively.

[0087] For the initial label representation, a GloVe model pre-trained on the Wikipedia dataset is used to extract 300-dimensional label semantic embedding vectors. When a label contains multiple words, the mean of all word embeddings is used as the label embedding.

[0088] The label embeddings output by the hypergraph convolutional layers HG_Conv_1, HG_Conv_2, and HG_Conv_3 are sequentially injected into Conv_2, Conv_3, Conv_4, and Conv_5 in ResNet-101 to couple feature vectors.

[0089] Example 4

[0090] Based on the COCO2014 dataset, the performance of mAP, CP, CR, CF1, and OP was compared using experimental results from a multi-label classification method that incorporates high-order label correlation (HOLC-MLC) and one that uses only the ResNet-101 backbone network. The results are shown in Table 2.

[0091] Table 2. Experimental Results of the Effectiveness of Higher-Order Relation Modeling Networks in COCO2014

[0092]

[0093]

[0094] Based on the VOC2007 dataset, the mAP performance of a multi-label classification method that incorporates high-order label correlation (HOLC-MLC) and one that uses only the ResNet-101 backbone network were compared, as shown in Table 3:

[0095] Table 3. Experimental Results of the Effectiveness of High-Order Relationship Modeling Networks in VOC2007

[0096]

[0097] As shown in Tables 2 and 3, HOLC-MLC achieves an improvement in mAP of up to 8.3% and 4.55% compared to its degenerate version, ResNet-101. This demonstrates that the label embedding of the Hypergraph Neural Network (HGNN) can effectively utilize higher-order label relationships, thus verifying the importance of introducing higher-order label correlations.

[0098] It should be noted that, in this document, relational terms such as "first" and "second" are used only to distinguish one entity or operation from another, and do not necessarily require or imply any such actual relationship or order between these entities or operations. Furthermore, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such process, method, article, or apparatus.

[0099] Finally, it should be noted that the above descriptions are merely preferred embodiments of the present invention and are not intended to limit the present invention. Although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art can still modify the technical solutions described in the foregoing embodiments or make equivalent substitutions for some of the technical features. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the protection scope of the present invention.

Claims

1. A multi-label classification method that integrates higher-order label relevance, characterized in that, The steps include: A data-driven approach is used to mine label co-occurrence information in the dataset to define the correlation between labels and construct a label hypergraph. For image classification tasks, a deep residual network (ResNet) is selected as the backbone extraction network; Hypergraph convolution operations are performed on the constructed label hypergraph using hypergraph convolutional layers to extract higher-order correlations between nodes in the hypergraph, thus obtaining label embedding vectors. The label embedding vectors output by each hypergraph convolutional layer are sequentially embedded into the connection points of the backbone feature extraction network. At the same time, the LC operation is used to fuse the label embedding vectors with the ResNet feature vectors at the connection points of the backbone feature extraction network, thereby using high-order label correlation to guide the learning of image features. The label hypergraph construction steps include: The co-occurrence frequency of each label pair in the dataset is counted to obtain a co-occurrence frequency matrix. , where M represents a q-row, q-column matrix, R indicates that the elements in matrix M are real numbers, and q is the number of labels in the multi-label dataset; Count the number of times each label is labeled in the sample to obtain a one-dimensional array. ,in Indicates label Number of times the data is labeled in the sample; Based on the co-occurrence frequency matrix M and the labeled frequency array N, the probability matrix P is calculated using the conditional probability formula. Based on the probability matrix P, a hypergraph association matrix is ​​constructed using the nearest neighbor search method. In the hypergraph incidence matrix H, rows represent labeled vertices, columns represent hyperedges, and p and n represent the number of vertices and the number of hyperedges, respectively. Add an identity matrix to H The hypergraph correlation matrix H is obtained. L .

2. The multi-label classification method that integrates higher-order label correlations according to claim 1, characterized in that, The conditional probability formula: ； The position in the probability matrix P Formula for calculating the element at: ； in, Indicates when the label When it appears, the tag The conditional probability of occurrence Indicates label and tags The probability of them occurring simultaneously Indicates label The probability of it appearing in the training set. M ij Indicates label and The number of co-occurrences in the training set, where m is the number of training samples in the training set. Indicates label Number of annotations in the training set samples Represents the position in probability matrix P The element at position , whose value corresponds to .

3. The multi-label classification method that integrates higher-order label correlations according to claim 1, characterized in that, Labeled hypergraphs use the conditional probability of label co-occurrence as a metric for searching nearest neighbor vertices. That is, each hyperedge selects a vertex as its centroid and connects the K nearest neighbor vertices with the conditional probability value. Hypergraph Independence Matrix The element definition formula: ； in, Represents vertices The set of K nearest neighbor vertices, , , Represents the hypergraph incidence matrix The element position; due to the use of the nearest neighbor search method, a corresponding hyperedge will be generated for each vertex, making ; When the vertex lie in In the nearest neighbor set, the hypergraph incidence matrix The value at the corresponding position is set to the conditional probability value of the labels corresponding to the two vertices; otherwise, it is set to 0.

4. The multi-label classification method that integrates higher-order label correlations according to claim 1, characterized in that, The propagation formula between the hypergraph convolutional layers is as follows: ； in, For the first Layer label embedding vectors, initial label embeddings It can be obtained through word embedding technology; H is the hypergraph association matrix; and Let be the diagonal matrices representing the vertex degree and hyperedge degree of the hypergraph, respectively. Let be the transpose of the hypergraph incidence matrix. express The weights of each hyperedge are set, and W is initially set to the identity matrix I. It is a non-linear activation function. For the first Layer weights The function representing the hypergraph convolution operation. For the first Layer label embedding vector.

5. The multi-label classification method that integrates higher-order label correlations according to claim 1, characterized in that, The backbone feature extraction network adopts the ResNet-101 network; connection points are set between Conv_2, Conv_3, Conv_4 and Conv_5 in the ResNet-101 network.

6. The multi-label classification method that integrates higher-order label correlations according to claim 1, characterized in that, The LC operation steps include: During the LC operation between the label embedding vector and the ResNet feature vector, the tensor shapes of the label embedding vector and the ResNet feature vector are adjusted to ensure that they remain consistent in dimension. The relationship between ResNet feature vectors and label embedding vectors with consistent dimensions is calculated by tensor multiplication, and then the relevant tensors are mapped to the hidden space by 1x1x1 convolution to encode the relationship between ResNet features and label embeddings. The relation tensor generated by the 1×1×1 convolution is added to the original ResNet feature tensor to form higher-order label features. .

7. The multi-label classification method that integrates higher-order label correlations according to claim 1, characterized in that, The formula for the LC operation is as follows: ； in, The ResNet feature at the l-th connection point, Here, q represents the fused features obtained after the LC operation, c represents the number of labels, h and w represent the height and width of the feature tensor. This is the label embedding vector in the convolutional layer. This represents a 1×1×1 convolution function. This represents the matrix multiplication operation. Indicates the transpose operation; This indicates a tensor dimension adjustment operation, used to adjust the input tensor according to the dimensions of the number of labels q, the feature tensor height h, and the feature tensor width w. It is a non-linear activation function. This represents the dimensions of the feature tensor height h, feature tensor width w, and number of channels c after the tensor dimension adjustment operation.

Images