Multi-view multi-label learning method based on deep feature map fusion

Abstract

This invention discloses a multi-view, multi-label learning method based on deep feature map fusion. Addressing the limitation of single shared subspace models in fully describing all semantic information of multi-view data, this method proposes a multi-view, multi-label classification method based on deep feature map fusion. By mining the complementary relationships of instance features across multiple views and the structured symbiotic relationships of label features, it constructs a more representative instance-label structured vector representation and classifies the data by averaging the "instance-label" affinity matching results of individual views. This method enhances the structured semantic representation capability of each individual view by constructing a unified feature map structure across multiple views, fusing nearest neighbor relationships within a single view and alignment relationships across views. It emphasizes the contribution of individual views to specific semantic representations while integrating the consensus and complementary relationships of multi-view data. This effectively improves the semantic representation capability of multi-view data and has strong application value for practical data analysis and decision-making.

Description

Technical Field

[0001] This invention relates to multi-view, multi-label information fusion and deep learning technology, specifically to a multi-view, multi-label learning method based on deep feature map fusion. Background Technology

[0002] With the rapid development of cloud computing, the Internet of Things, and especially artificial intelligence technologies and the widespread use of related methods, massive amounts of multi-view, multi-label data have emerged. How to rationally and efficiently apply this data is a crucial issue. Existing research suffers from incomplete semantic representation at the feature level when using this data; that is, the semantic information of the view data is difficult to fully characterize. In multi-view, multi-label learning, each instance is described by multiple heterogeneous feature representations and simultaneously associated with multiple valid labels. Past research in this field has largely focused on using shared subspaces to represent multi-view consensus information across different views. However, the effectiveness of using shared subspaces to solve this problem remains to be verified. In particular, while this method can fuse consensus information from multiple views, the view fusion process inevitably leads to the obscuring of some key individual view semantic information, thus affecting the representation of its complete semantics.

[0003] To address this issue, this invention proposes a multi-view, multi-label classification algorithm based on deep feature map fusion (L-VSM: Label DrivenView-Special Fusion for Multi-View Multi-Label Learning) to solve the problem of incomplete semantic representation in multi-view, multi-label learning. By constructing relevant graph structures to obtain structured representations of views and labels, and finally emphasizing the contribution of individual views to specific semantic representations based on consensus and complementary information, the algorithm improves the classification performance of multi-view, multi-label data. Summary of the Invention

[0004] The technical problem solved by this invention is to propose a multi-view, multi-label learning method based on deep feature map fusion, which solves the problems of difficulty in multi-view feature fusion and incomplete semantic representation in existing multi-view learning algorithms.

[0005] The technical solution of this invention is a multi-view multi-label learning method based on deep feature map fusion. This method addresses the issue that a single shared subspace model cannot fully describe all the semantic information of multi-view data. It proposes a multi-view multi-label classification method based on deep feature map fusion. By mining the complementary relationship between instance features and the structured symbiotic relationship between label features, it constructs a more representative instance label structured vector representation and classifies the data by averaging the "instance-label" affinity matching results of individual views. Specifically, the label-driven view-specific fusion MVML method (L-VSM) can bypass the search for the shared subspace representation and fuse the supplementary information of each individual view with other views through a deep graph neural network, directly contributing this useful information to the final discrimination model. This method consists of the following four parts: (1) construction of a unified feature map for multi-view based on label embedding; (2) structural instance feature representation; (3) structural label feature representation; and (4) multi-label classification.

[0006] The specific steps are as follows:

[0007] In this invention, matrices are represented by bold uppercase letters, such as X; vectors are represented by bold lowercase letters, such as x; furthermore, (XR) represents the matrix obtained by X·R, where · represents matrix multiplication. The inverse and transpose of matrix X are respectively represented as X -1 X T X ν Let X represent the feature matrix of the v-th view. ν The i-th column and j-th row are denoted as (X) ν ) :,i and (X) ν ) j,: (X) v ) i,j It is X v The (i,j) element, x i Let represent the i-th element of vector x. Additionally, use... It represents the real number field.

[0008] (1) Construction of a unified feature map for multiple views based on label embedding;

[0009] definition Let T be the feature space with T views, and As a tag space with q class tags, where d t (1≤t≤T) is the feature dimension of the t-th view. Given training data D={(X i ,y i For each instance, |1≤i≤n}. Represented by T eigenvectors, y i ∈[0,1]q×1 Let f represent the label vector of the i-th instance. In constructing the multi-view unified feature map, the feature representations of different instances under the same view are used as nodes. That is, each view corresponds to a feature map, and each node is described by a feature representation of an instance. Node feature similarity is used as edges to construct k-nearest neighbor feature representation maps for different views. The proposed L-VSM structure aims to integrate these different representations from different views to construct a robust multi-label classifier f: Furthermore, it predicts appropriate labels for invisible instances. Specifically, in this label-driven feature map construction strategy, the invention embeds label information into each feature map and adaptively selects reliable neighbors (rather than static neighbors) to construct the desired instance map. Then, by connecting the different feature representation nodes in each instance, the above graphs are integrated into a unified feature map. Finally, a graph attention mechanism is used to fuse intra-view correlation and inter-view alignment into each feature node to form a structural representation of each instance. Here, intra-view correlation reflects the instance relationships under each individual view, while inter-view alignment reflects the view connections between each instance. This strategy effectively avoids the minority class instance feature representations being overwhelmed by the majority class instance feature representations and naturally improves the performance of the final model. In different views, neighbors of the same instance are adaptively selected, which is more suitable for multi-view, multi-label tasks because different views often reflect different label information, and different labels correspond to different semantic relationships. In different views, view correlation is instance-specific, which brings richer view complementarity. The value of the number of neighbors k is to be determined by embedding label information, i.e.:

[0010]

[0011] Where · is the vector y i and y j The dot product operation, ║Δ║ is the L1 norm of the vector Δ, and θ represents the label confidence threshold. After that, the feature representation nodes under different views of the same instance are connected to each other to form a unified multi-view feature representation graph.

[0012] Construct different graphs G from all instances in different views. (t) =(V (t) E (t) ), where t∈{1,2,…,T}. Each node V in the graph (t) Let E represent the feature representation under the t-th view, and let E represent the edge. (t) This represents the similarity between two connected nodes. Specifically, in each graph G... (t) In this context, each instance node is described using a d-dimensional vector. Then for the edge e between each pair of nodes (t) ∈E(t) The following results can be produced:

[0013]

[0014] in Indicates about The k-nearest neighbors (measured by Euclidean distance), and It indicates that it comes from arrive Undirected edges, otherwise

[0015] After obtaining each individual feature representation graph, the different feature representation nodes in each instance are connected together, and the above individual feature graphs are integrated into a unified multi-view feature representation graph, where the edges between different types of feature nodes, i.e. different views, represent the view correlation between their connected views.

[0016] (2) Structural instance feature representation

[0017] In the process of structuring the representation of instance features, the original features under the t-th view are... An attention-based deep graph neural network architecture (R-GCN) is used to compute each hidden feature node. The representation uses a graph attention mechanism to aggregate and update sample attributes, intra-view nearest neighbor relationships, and inter-view alignment relationships, resulting in a more representative structured vector representation. R-GCN is typically used to process graph-structured data, where nodes represent instance features and edges represent relationships between nodes. The core idea is to update and aggregate node representations by learning interaction patterns between nodes. Specifically, for each hidden feature node, an attention mechanism is used to calculate importance weights to weight different pieces of information. This attention mechanism determines importance weights based on relationships and features between nodes, effectively aggregating different information. In this architecture, by utilizing the sample's own attribute information, the nearest neighbor relationships of nodes within a view, and the alignment relationships between views, R-GCN can comprehensively consider features from multiple views and weight them through the attention mechanism. This results in more representative, structured vectors for further task processing or analysis.

[0018]

[0019] here, Representing the instance features after unifying the dimensions, σ(·)=max(0,·) is the activation function. and It is a weight matrix, obtained by using the corresponding example features. Multiply by the weight matrix and Obtain the updated structure representation.

[0020] Under the t-th view yes The nearest neighbor, The encoding weight matrix, k and V represent the number of neighbors and views, respectively. According to formula (3), each feature representation is accompanied by three types of structural information: self-attribute information (first term), intra-view related information (second term), and inter-view alignment information. Here, intra-view relatedness integrates the contributions of the k nearest neighbors under the same view, while inter-view alignment integrates the complementary information between different views in the same instance, jointly enhancing the ability to identify the represented instance and further improving the robustness of the final model. Furthermore, to avoid the model falling into overfitting, the weights... Regularization was performed as a basis transformation with coefficients. A linear combination of .

[0021]

[0022] Furthermore, considering the contributions of other instances represented in different views, the ability to recognize structural features is enhanced. In the experiment, the output equation (3) is used as its input, and such propagation operations are repeated to fuse more complementary information between views into each feature node. This process can be repeated multiple times to gradually extract and integrate richer structural feature representations. Through iterative propagation operations, the model can better capture the correlation information between different views and represent instances more accurately. Then, the desired structural feature representation is obtained. Used for subsequent multi-label classification.

[0023] (3) Structural marker feature representation

[0024] To explore the widespread label relevance in multi-label learning, a Transforer architecture is introduced to construct a dynamic semantic-aware label graph and generate a structural semantic representation for each specific label accordingly. Specifically, considering the diversity of semantic relationships in different views, for each specific label c... i (1≤i≤q), generate T original labeled feature representations. Each of the labeling features represents By averaging its corresponding instance features get.

[0025]

[0026] Here, Represents and is marked ci The number of associated instances for each labeled feature representation We construct separate, fully connected, undirected labeled graphs to mine their unique internal semantic relationships. Constructing multiple labeled graphs with different semantic relationships is primarily because different views typically reflect different semantic information, and different semantic information often corresponds to different semantic relationships. After initializing these labeled feature representations, we then... Transform into higher-dimensional features To acquire sufficient expressive ability. It is a shared linear transformation matrix, and σ(·) = max(0,·) is an element-wise activation function. The standard Transformer encoder structure is applied as the Transformer unit to establish dynamic semantic relationships.

[0027]

[0028] For each specific class, a special semantic representation is generated accordingly:

[0029]

[0030]

[0031] here, This represents the label matrix. Represents the weight matrix. Represents the transformation matrix. and This represents the bias vector. It's important to note that the above operations are performed independently in each different view to maintain specific semantic relationships across different views.

[0032] (4) Multi-label classification

[0033] To overcome the limitations of traditional shared subspaces, the model is updated by optimizing Multi-Label Soft Margin Loss, and label prediction is performed by averaging the "instance-label" matching affinity results of the individual view.

[0034] In the proposed L-VSM, each individual structural feature obtained in steps (2) and (3) is represented by... and symbols represent Get the representation for each instance Marked confidence score Each instance X i The final labeled confidence score [p]i1 p i2 , ..., p iq The confidence level is calculated by extracting the marker confidence from different views.

[0035]

[0036] Then, each X is calculated by averaging the label confidence from different views. i The final labeled confidence score of the instance [p] i1 ,p i2 ,…,p iq ]:

[0037]

[0038] Widely used Multi-Label Soft Margin Loss:

[0039]

[0040] in, S represents the matching affinity, and S(·) represents the sigmoid function.

[0041] Compared to existing technologies, the innovation of this method lies in addressing the problem that traditional shared subspace methods cannot comprehensively represent all semantics of samples. By constructing a unified feature map structure across multiple views, it integrates the nearest neighbor relationships within a single view and the alignment relationships across views, thereby enhancing the structured semantic representation capability of each individual view. Its distinctive feature is emphasizing the contribution of individual views to specific semantic representations while integrating the consensus and complementary relationships of multi-view data. This method effectively improves the semantic representation capability of multi-view data, overcoming the incomplete semantic representation problem of shared subspace methods, and has strong application value for practical data analysis and decision-making. Attached Figure Description

[0042] Figure 1 The training process of the L-VSM model is shown.

[0043] Specific implementation methods

[0044] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0045] 1. Multi-view feature map construction:

[0046] V feature maps G(t) are constructed under different views, where each edge is defined by equation (2). After obtaining each individual feature map, the nodes representing different features in each instance are connected together, and the above individual feature maps are integrated into a unified multi-view feature map, where the edges between different types of feature nodes (i.e. different views) encode the view correlation between their connected views.

[0047] 2. Regarding the representation of structural instance features:

[0048] Specifically, firstly, each original input feature vector Transform into higher-level features To achieve sufficient expressiveness, among which It is a shared linear transformation matrix, and σ(·)=max(0,·) is the element activation function. Then, each feature representation node in the unified multi-view graph can be updated using formula (3). In addition, to avoid the model from getting overfitted, the weights in the formula are adjusted. Regularization was performed as a basis transformation with coefficients. A linear combination of the two. Further considering the contributions of other instances represented in different views, the ability to recognize structural features is enhanced. In the experiment, the output equation (3) was also used as its input, and this propagation operation was repeated to fuse more complementary information between views into each feature node, obtaining the desired structural feature representation. Used for subsequent multi-label classification.

[0049] 3. Regarding the representation of structural marker features:

[0050] Construct a fully connected labeled semantic graph and calculate the original labeled representation using equation (5). For each type of label feature representation An independent, fully connected, undirected labeled graph was constructed to mine their unique internal semantic relationships. After initializing these labeled feature representations, each original labeled feature representation... Transform into higher-level features To obtain sufficient expressive power. In the process of constructing the structured feature relationship of the marker, it is proposed to introduce the Transformer structure to construct a dynamic semantic-aware marker graph, and generate a structured semantic representation for each specific marker accordingly. Considering that different views reflect different semantic information and thus correspond to different semantic associations, the semantic association relationships under different views are constructed as shown in formula (6). According to the different semantic association relationships, a corresponding class structured representation is generated for each marker. As shown in formulas (7) and (8). Formula (8) is used to update... As input to the continuous attention unit, record Used for subsequent multi-label classification.

[0051] 4. Multi-label classification:

[0052] The structure is represented by each individual structural feature obtained through continuous updates above. and To obtain for each case Marked confidence score Each instance X i The final labeled confidence score [p] i1 p i2 , ..., p iq The final label confidence score is obtained by calculating the label confidence score using formula (10). The model parameters are then updated by minimizing the Multi-Label Soft Margin Loss in the equation, i.e., formula (11).

[0053] Experimental dataset description:

[0054] To evaluate the performance of the proposed L-VSM, comprehensive experiments were conducted on 10 benchmark datasets. Emotions contains 593 music tracks described by two perspectives: 8 rhythmic attributes and 64 timbre attributes. Scene consists of 2407 images, with 294 features from the two views reflecting the brightness and chromaticity of colors, respectively. Yeast is a biological gene dataset where the phylogenetic spectrum (24 attributes) and genetic expression connectivity (79 attributes) of a gene correspond to its two different feature perspectives. Plant and Human are two multiple protein localization classification datasets, composed of two features (amino acids and dipeptides) extracted from 978 and 3106 sequences from plants and humans, respectively. Corel5k and Espgame contain 4999 and 20770 images, respectively, both represented by four different features: GIST, HSV, HUE, and DIFT. Pascal and Mirflflickr, in addition to the four views mentioned above, also include a text view to describe their labeled properties. Table 1 summarizes the features of the above datasets.

[0055] Table 1: Characteristics of the dataset used

[0056]

[0057] -D min-max The minimum and maximum dimensions of the feature.

[0058] Experimental Design:

[0059] A comparative study was conducted using six state-of-the-art methods in two categories: the first category consisted of multi-label learning methods such as ML-KNN, RakeLD, and LSPC, which used all view features as input to the learning model; the second category consisted of multi-view, multi-label methods such as LrMMC, SIMM, D-VSM, FIMAN, GRADIS, iMVML, and NAIM3L, which combined the complementarity of different views to summarize the classification model.

[0060] Among these methods, the multi-label learning method ML-KNN was published in PR 2007, a top journal in computer vision; RakeLD was published in TKDE 2011, a top journal in data mining; LSPC was published in Entropy, a journal under MDPI, in 2016; the multi-view multi-label method LrMMC was published at the top conference AAAI in 2015; SIMM was published at the International Joint Conference on Artificial Intelligence (IJCAI) 2019; FIMAN was published at ACM SIGKDD 2020, an international conference on knowledge discovery and data mining; D-VSM was published at AAAI 2022; GRADIS was published at AAAI 2020; iMVML was published at IJCAI 2018; and NAIM3L was published in the computer science journal TPAMI 2021. The configuration parameters for all the above methods were set according to the recommendations in the relevant literature.

[0061] In addition, six popular multi-label metrics for evaluating each comparison method were used: Hamming Loss, Ranking Loss, One-Error, Coverage, Average Precision, and Micro-F1.

[0062] Experimental results:

[0063] Table 2-9 illustrates the experimental comparisons of the proposed L-VSM with seven other methods across all evaluation metrics, recording the mean and standard deviation of each metric. In the statistical comparisons of 420 (10 datasets × 7 methods × 6 evaluation metrics), the following observations can be made: From the perspective of the comparison methods, the proposed L-VSM significantly outperforms the two multi-label learning methods and the multi-view multi-label learning method. Specifically, L-VSM outperforms ML-KNN, RakeLD, LSPC, LrMMC, and FIMAN in 100% of cases. Correspondingly, L-VSM outperforms SIMM and D-VSM in 96.25% and 88.75% of cases, respectively. These results demonstrate that the proposed view-specific strategy can effectively improve the learning performance from multi-view multi-label data. For datasets with a large number of classes (such as Corel5k and Iaprtc12), L-VSM also outperforms the other comparison methods in 97.02% of cases. Furthermore, L-VSM still achieves good performance for datasets with high-dimensional features (such as Pascal). These results demonstrate the effectiveness of the proposed L-VSM in learning from complex multi-view, multi-label data. From an evaluation metric perspective, the proposed L-VSM still shows significant improvements across almost all metrics. In particular, for the Macro-F1 metric, which reflects learning performance on imbalanced multi-label data, L-VSM outperforms other comparable methods in 97.14% of cases, with its advantage being even more pronounced on certain imbalanced datasets. These results empirically demonstrate the effectiveness of the proposed label-driven instance graph construction strategy in handling imbalanced multi-view, multi-label data.

[0064] Table 2: Hamming Loss(the lower the better)

[0065]

[0066]

[0067] Table 3: Ranking Loss(the lower the better)

[0068]

[0069] Table 4: One Error(the lower the better)

[0070]

[0071]

[0072] Table 5: Coverage(the lower the better)

[0073]

[0074]

[0075] The proposed L-VSM method was experimentally compared with other comparative methods on Hamming Loss, Ranking Loss, One Error, and Coverage metrics. The best performance is shown in bold and "-", indicating that FIMAN requires more than 256G of RAM on the Mirflflick dataset.

[0076] Table 6: Average Precision (the higher the better)

[0077]

[0078]

[0079] Table 7: Micro-F1(the higher the better)

[0080]

[0081]

[0082] Table 8: Subset Accuracy(the higher the better)

[0083]

[0084]

[0085] Table 9: Macro-F1(the higher the better)

[0086]

[0087] The proposed L-VSM was experimentally compared with other comparative methods on the metrics of Average Precision, Micro-F1, SubsetAccuracy, and Macro-F1. The best performance is indicated by bold and "-", indicating that FIMAN requires more than 256G of RAM on the Mirflflick dataset.

[0088] This paper proposes a label-driven view-specific fusion model, L-VSM, for multi-view, multi-label tasks. This model integrates the complementarity of different views into each individual view and directly uses these individual views to induce the final model. Compared with previous methods, L-VSM transcends the limitation of shared subspaces and improves model performance by simultaneously utilizing complementary information between different views and view-specific information within individual views. The label-driven feature map construction strategy and the transformer-based dynamic labeling relationship also jointly improve the effectiveness and robustness of the learned model. Extensive experimental results on classic multi-view, multi-label tasks and weakly supervised multi-view, multi-label tasks demonstrate the significant advantages of the proposed L-VSM over existing methods.

Claims

1. A multi-view multi-label learning method based on deep feature map fusion, characterized in that: This method is based on multi-view multi-label classification using deep feature map fusion. By mining the complementary relationship between instance features and the structured symbiotic relationship between label features, it constructs a more representative instance-label structured vector representation and classifies the data by averaging the "instance-label" affinity matching results of individual views. This method bypasses the search for shared subspace representations and fuses the supplementary information of each individual view with other views through a deep graph neural network, directly contributing this useful information to the final discrimination model. This method consists of the following four parts: (1) construction of a unified feature map for multi-view based on label embedding; (2) structural instance feature representation; (3) conclusion. (4) Multi-label classification; The dataset used is Scene, which consists of 2407 images, of which 294 features from two views reflect the brightness and chromaticity of the color respectively; or the dataset used is Corel5k and Espgame, which contain 4999 and 20770 images respectively, both represented by 4 different features: GIST, HSV, HUE, DIFT; or the dataset used is Pascal and Mirflflickr, which, in addition to the above four views of GIST, HSV, HUE, DIFT, also add a text view to describe their label characteristics; The process of constructing a multi-view unified feature map based on labeled embedding is as follows; definition For a feature space with T views, Represent the field of real numbers, and As a tag space with q class tags, where It is the feature dimension of the t-th view. Given training data Each instance Represented by T eigenvectors, Let represent the label vector of the i-th instance. During the construction of the unified feature map for multiple views, the feature representations of different instances under the same view are used as nodes. That is, each view corresponds to a feature map, and each node is described by a feature representation of an instance. The node feature similarity is used as an edge. This is used to construct the feature map for different views. Nearest neighbor feature representation graph; Embed the tag information into each In the nearest neighbor feature representation graph, the nearest neighbors are adaptively selected to construct the desired instance graph; then, by connecting the different feature representation nodes in each instance, the above instance graphs are integrated into a unified feature graph; finally, a graph attention mechanism is used to fuse intra-view correlation and inter-view alignment into each feature node to form a structural representation of each instance; Intra-view correlation reflects the instance relationships within each individual view; inter-view alignment reflects the view connections between each instance; nearest neighbor count. The value is to be determined by embedding tag information, that is: (1)； Where · is a vector and The dot product operation, ║Δ║ is the L1 norm of the vector Δ. After marking the confidence threshold, feature representation nodes under different views of the same instance are connected to form a unified multi-view feature representation graph; Construct different graphs from all instances in different views. ,in Nodes in each graph Let represent the feature representation under the t-th view, and represent the edge. Represents the similarity between two connected nodes; in each graph In the middle, use A dimensional vector is used to describe each instance node. Then for each pair of nodes, the edge is... The following results were produced: (2) in Indicates about of Neighbor, and It indicates that it comes from arrive Undirected edges, otherwise ; After obtaining each individual feature representation map, the different feature representation nodes in each instance are connected together, and the above individual feature representation maps are integrated into a unified multi-view feature representation map, where the edges between different types of feature nodes, i.e., different views, represent the view correlation between their connected views; the structurally labeled feature representation is as follows: The Transformer architecture is introduced to construct a dynamic semantic-aware label graph, for each specific label. Generate T original labeled feature representations. Each of the labeling features represents By averaging its corresponding instance features get; （5）； Representation and Marking The number of associated instances for each labeled feature representation Each graph is constructed as an independent, fully connected, undirected labeled graph to mine their unique internal semantic relationships; after initializing the labeled feature representations, each original labeled feature representation is... Transform into higher-dimensional features To acquire sufficient expressive ability; It is a shared linear transformation matrix. It is an element-wise activation function; it uses the standard Transformer encoder structure as Transformer units to establish dynamic semantic relationships. ; （6）； For each specific class, a special semantic representation is generated accordingly: ; （7）； （8）； Represents the label matrix; , , Represents the weight matrix. , , Represents the transformation matrix. and This represents the bias vector.

2. The multi-view, multi-label learning method based on deep feature map fusion according to claim 1, characterized in that: The process of representing structural instance features is as follows: Original features under the t-th view It employs an attention-based deep graph neural network architecture, R-GCN, to compute each hidden feature node. The representation uses a graph attention mechanism to aggregate and update sample attributes, intra-view nearest neighbor relationships, and inter-view alignment relationships, resulting in a more representative structured vector representation. R-GCN is used to process graph-structured data, where nodes represent instance features and edges represent relationships between nodes. It updates and aggregates node representations by learning interaction patterns between nodes. For each hidden feature node, an attention mechanism is used to calculate importance weights to weight different pieces of information. (3)； This represents the instance features after unifying the dimensions. It is an activation function. , and It is a weight matrix, obtained by using the corresponding example features. Multiply by the weight matrix , and Obtain the updated structure representation; Under the t-th view yes The nearest neighbor, Encoding weight matrix, and These represent the number of neighbors and the number of views, respectively. To avoid the model from overfitting, the weights are adjusted. Regularization was performed as a basis transformation with coefficients. A linear combination of; ； Using the output equation (3) as its input, and repeating the propagation operation to fuse more complementary information between views to each feature node; thus obtaining the desired structural feature representation. Used for subsequent multi-label classification .

3. The multi-view, multi-label learning method based on deep feature map fusion according to claim 2, characterized in that: Multi-label classification includes the following: The model is updated by optimizing the Multi-Label Soft Margin Loss, and label prediction is performed by averaging the "instance-label" matching affinity results of the individual view. By employing each individual structural feature obtained in steps (2) and (3) to represent and symbols represent Get the representation for each instance Marked confidence score Each instance Final label confidence score It is calculated by extracting the confidence scores of the markers from different views; （9）； Then, by averaging the confidence scores of the tags from different views, each... Final label confidence score of the instance : （10）； Use Multi-Label Soft Margin Loss: （11）； in, Indicates matching affinity. This represents the sigmoid function.

Images