Incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and double-graph driving
Through pseudo-label uncertainty perception and dual-graph driving methods, the problem of insufficient utilization of supervision information in incomplete multi-view multi-label data is solved. Features are generated through cross-view and category encoder, which improves classification performance and solves the problem of view or Multi-view multi-label classification problem with missing tags.
Patent Information
- Application Number
- CN202411159232.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-08-22
- Publication Date
- 2024-11-15
- Estimated Expiration
- Not applicable · inactive patent
AI Technical Summary
Existing multi-view multi-label classification methods are difficult to make full use of supervision information in incomplete data, especially in the absence of views or labels, it is difficult to extract discriminant features.
Using an incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-graph drive, embedding features are generated through cross-view encoder and cross-category encoder, and average view marks and label-related graphs are introduced to construct the loss function to Integrate the inherent relationship between missing information and retaining data.
The classification performance of the model on incomplete multi-view multi-label data is improved, uncertainty perception and label semantic information are fully utilized, and feature discrimination ability and classification accuracy are improved.
Smart Images

Figure CN118968184A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of pattern recognition, and in particular relates to an incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving. Background Art
[0002] With the development of data collection technology, many researchers have found that multi-view data integrated from different sources can provide more nuanced and diverse object representations. For example, images can be described by different feature descriptors such as SIFT, Gist, and HSV. Therefore, multi-view learning has become an important method in the field of data analysis. On the other hand, as a classic classification problem, multi-label classification has also long occupied an important position in the field of pattern recognition. Unlike single-label data that requires mutual exclusivity between labels, multi-label data contains various category labels and naturally maintains complex label correlations. For example, an image may be described by multiple elements such as "neon lights", "sidewalks", and "passing vehicles", each of which contributes to a more comprehensive understanding of the scene. However, traditional multi-label classification methods often have a major problem because they mainly rely on features extracted from a single view, which limits their performance. By integrating multi-view learning into multi-label classification, this limitation can be effectively alleviated. Therefore, multi-view multi-label classification came into being and attracted more and more attention from researchers.
[0003] Multi-view multi-label classification synergistically combines the advantages of multi-view learning and multi-label classification, providing a more powerful and comprehensive framework for understanding complex multi-view multi-label data. A large number of related works have been proposed in recent years. For example, some methods use latent conditional Bernoulli mixture methods to integrate features from multiple views and use shared latent subspaces to construct label dependencies. Other methods propose to use matrix decomposition in kernel space and Hilbert-Schmidt independence criteria to align latent semantic components across multiple views.
[0004] However, a major challenge in multi-view multi-label classification is the frequent presence of incomplete data, both in terms of views and labels. This challenge directly leads to the necessity of incomplete multi-view multi-label classification, which focuses on scenarios where some views or labels are missing. Although some work has made progress, due to the incompleteness problem, existing methods still have difficulty in fully utilizing the supervised information to help the model extract more discriminative features. Summary of the invention
[0005] In order to solve the above technical problems, the present invention proposes an incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving to solve the problems existing in the above-mentioned prior art.
[0006] To achieve the above object, the present invention provides an incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving, comprising:
[0007] Obtain and integrate multi-view training data, incomplete label sets, missing view indication matrices, and missing label indication matrices to obtain an incomplete multi-view incomplete multi-label dataset;
[0008] Constructing a cross-view encoder, inputting the incomplete multi-view incomplete multi-label dataset into the cross-view encoder, and generating an embedding feature for each view;
[0009] Constructing a dynamic weighted fusion module, inputting the embedded features of each view into the dynamic weighted fusion module to generate a common representation of multiple views;
[0010] Constructing a cross-category encoder, and obtaining the embedded features of the view-category interaction of the multi-view common representation through the cross-category encoder;
[0011] A final classification representation is obtained based on the embedded features after the view-category interaction, and a classification prediction result is calculated for the final classification representation through a classifier.
[0012] Preferably, the process of constructing a cross-view encoder further comprises:
[0013] Introduce the same average view token as an additional input to the cross-view encoder;
[0014] The expression of the average view mark of the i-th sample is:
[0015]
[0016] Among them, Λ i represents the average view label of the i-th sample, m is the number of views, v indicates the v-th view, W is the missing view indicator matrix, and W i,v is the element in the i-th row and v-th column of the matrix W.
[0017] Preferably, for the cross-view encoder input of the i-th sample, the expressions for view correlation and output are:
[0018]
[0019]
[0020] in, is the view dependency, is the output, d h =d e / h represents the dimension of each attention head, is the output of the tth attention head, is the projection matrix, d e Represents the mapping dimension of multi-view data.
[0021] Preferably, the expression for generating a common representation of multiple views is:
[0022]
[0023] in, Represents Z i The embedding feature of the vth view / row in a v represents the learnable scalar weight of the v-th view, f i is the fusion feature of sample i.
[0024] Preferably, the cross-view encoder combines the structure of the latent representation with the supervised label graph based on the cross-entropy loss of the graph. Target structure alignment;
[0025] The expression of the cross entropy loss of the graph is:
[0026]
[0027] in, represents the number of available sample pairs in all views, is the similarity between the embedded features of two samples in the vth view, Q i,j is the element in the i-th row and j-th column of the label-based correlation graph Q.
[0028] Preferably, the loss function of the cross-category encoder is expressed as:
[0029]
[0030] in, is the label correlation graph, represents the probability that a sample has both the i-th and j-th labels, K i,j is the similarity between the category vector of the i-th category and the category vector of the j-th category based on the cosine distance.
[0031] Preferably, the method further comprises: quantifying the uncertainty of the predicted label by a Monte Carlo method;
[0032] The expression for the uncertainty of the quantitative prediction label is:
[0033]
[0034] Among them, U i,j represents the uncertainty of the jth output of the ith sample, is the variance of the model output.
[0035] Preferably, the loss function of the classifier is expressed as:
[0036]
[0037] Among them, O is the final output, G i,j is the element in the i-th row and j-th column of the missing label indicator matrix G, The weight matrix representing the soft pseudo labels selected from the previous round of training The element in the i-th row and j-th column of ; Pseudo label
[0038] The final output expression is:
[0039]
[0040] Among them, Ψ is the final classifier, It is the embedded feature after view-category interaction for final classification.
[0041] Compared with the prior art, the present invention has the following advantages and technical effects:
[0042] This paper proposes an incomplete multi-view multi-label classification method based on uncertainty-aware pseudo-labeling and dual-graph driving, which integrates the information contained in the missing labels through a novel uncertainty-aware pseudo-labeling strategy. This paper proposes a label-guided dual-constraint method that exploits the untapped potential of multi-label semantics and preserves the intrinsic relationship of the original data in the embedding space by constructing similarity graph constraints. Considering the sensitivity of Transformer to different tokens, the average view tagging (AVT) is introduced into the Transformer architecture of this paper to better learn and synthesize consistent information across multiple views. BRIEF DESCRIPTION OF THE DRAWINGS
[0043] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:
[0044] Figure 1 A schematic diagram of an incomplete multi-view incomplete multi-label classification model according to an embodiment of the present invention;
[0045] Figure 2 This is a flowchart of pseudo-label selection based on uncertainty perception according to an embodiment of the present invention. DETAILED DESCRIPTION
[0046] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.
[0047] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.
[0048] Embodiment 1
[0049] This embodiment provides an incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving, including:
[0050] The classification method uses Transformer as the basic network framework. Through the dual graph constraint learning method at the feature level and the label level, the extracted features can maintain the topological structure of the original data, and a strategy based on pseudo-label uncertainty perception is designed to make full use of the semantic information in incomplete multi-label data, thereby improving the classification performance of the model on incomplete multi-view incomplete multi-label data.
[0051] The incomplete multi-view incomplete multi-label classification problem is defined as follows: given data Incomplete multi-label set Y∈{0,1} n×c , missing view indicator matrix W∈{0,1} n×m and missing label indicator matrix G∈{0,1} n×c . Where m is the number of views, n is the number of samples, and d v is the feature dimension of view v, and c is the number of label categories. The matrix element W i,j = 0 means that the jth view of the i-th sample is missing, W i,j =1 indicates that the corresponding view is available. Similarly, G i,j =1 indicates that the jth label of the ith sample is available, otherwise the label information is unknown. The deep neural network model proposed in this embodiment trains model parameters based on incomplete multi-view incomplete multi-label data and infers and predicts labels for test data.
[0052] The success of multi-view learning lies in utilizing the consistency and complementary information of multi-view data. To achieve this, this embodiment designs a new cross-view encoder to promote cross-view information interaction and extract high-quality cross-view features. In order to ensure the uniformity of the feature dimensions of each view, this embodiment first uses a multi-layer perceptron (MLP) to map the original multi-view data into a shared embedding space, represented as where d eRepresents the mapping dimension of multi-view data. In addition, in order to ensure the consistency of multi-view features, this embodiment introduces some identical average view tags (called AVT) as additional inputs to the cross-view encoder. These tags are calculated by the average of all available views for each sample. The calculation formula of AVT corresponding to the i-th sample is as follows:
[0053]
[0054] in represents the average view label of the i-th sample. In formula (1), the missing view indicator matrix is introduced to ignore the impact of missing views in each sample.
[0055] Since the self-attention mechanism dynamically adjusts the influence of the mark according to the distribution of the attention score, introducing multiple identical marks can amplify their influence in the sequence. In the method of this embodiment, this embodiment adopts the method of each sample having the same value. Average view tags.
[0056] Considering these labels, the cross-view encoder input for the i-th sample can be expressed as: in and In order to alleviate the negative impact of missing views when calculating attention scores in the cross-view encoder, this embodiment also utilizes the missing view indication matrix W to mask the missing views when calculating the multi-head self-attention scores.
[0057] The process of masking the multi-head self-attention layer in this embodiment is described in detail as follows: Figure 1 As shown, each input is linearly projected to obtain query, key, and value, using h sets of projection matrices Where h represents the number of attention heads.
[0058] Due to the expansion The average view tag AVT is taken as input and the mask matrix Expand, where the dimension of the expanded mask matrix should be Its newly added All elements in the column have a value of 1.
[0059] Then this embodiment constructs the mask matrix of sample i: in yes Then for the i-th sample input V i , view dependencies and output The calculation of is as follows:
[0060]
[0061]
[0062] where d h =d e / h represents the dimension of each attention head, is the output of the tth attention head. is the projection matrix. In addition, this embodiment introduces a filling function fill(A,B) before the softmax function in the cross-view encoder. If B i,j =0, then set A i,j =-10 9 . Through such a padding operation, softmax can ignore the negative impact of missing views when calculating the attention score. Finally, this embodiment connects the outputs of h attention heads to obtain the cross-view encoder output of the sample: In general, the cross-view encoder of this embodiment can be simply described as: And the first m rows of the cross-view encoder output is considered as the embedded feature of each sample.
[0063] After using the cross-view encoder to obtain the embedded features of each view, the goal of this embodiment is to create a unified and consistent representation to fully characterize multi-view data. However, different views may have different discriminative information. In order to balance the information between views, this embodiment introduces an adaptive weighted fusion strategy and combines the fused features f of sample i into i The definition is as follows:
[0064]
[0065] in Represents Z i The embedding feature of the vth view / row in a vRepresents the learnable scalar weight of the vth view, and is adaptively updated according to the network training process. It is worth noting that in order to mitigate the negative impact of missing views in the multi-view fusion process, this embodiment introduces a missing view indication matrix W in the fusion module. This equation captures an adaptive weighting mechanism for each view, resulting in a comprehensive feature representation that contains all available view information. While the present embodiment obtains a consistent common representation from incomplete multi-view data, another key issue is how to leverage the relationship between multiple labels to enhance the discriminative ability of the multi-view representation. To achieve this goal, the present embodiment introduces label embedding, maps each category label directly into the feature space, and uses a self-attention mechanism to capture category correlations. More specifically, the present embodiment randomly initializes c label embedding features as marking features for c categories They are introduced into the cross-category transformation encoder (also called cross-category encoder) together with the fusion features of the samples. Therefore, the input of the cross-category encoder is Among them C i =Concat(f i ;cls1;...;cls c ;). The cross-category encoder is similar to the cross-view encoder in that it can be used to facilitate information exchange between fused features and class labels. This process ensures that the view fusion features capture sub-category information based on similarity and align them closely with class labels. The output of the cross-category encoder is and The first line in is regarded as the final discriminant feature of classification. In short, the cross-category encoder of this embodiment can be simply expressed as:
[0066] In this embodiment, this embodiment attempts to introduce two label-guided constraints to improve the quality of latent representation and initialization class labeling.
[0067] Label-guided potential representation constraints:
[0068] In general, raw data usually exhibits a natural topological structure, in which some samples (such as nearest neighbor samples) or categories are similar while other samples are different. For example, some samples may be annotated with two or more identical labels at the same time. Unfortunately, most deep learning methods ignore this similarity structure relationship between samples when converting data into feature space. In order to solve the above problems and obtain more reasonable representation and good classification performance, this embodiment introduces the latent representation The label-guided constraint of , utilizes the rich semantic information of multiple categories to guide network training. First, based on the smoothness assumption, that is, if two samples have more common labels, they should also have higher similarity, such as Figure 2 As shown, this embodiment constructs a label-based correlation graph based on available label data. As shown below:
[0069]
[0070] Where ⊙ represents the Hadamard product. T Used to calculate the label-based correlation matrix The equation quantifies the similarity between samples according to the number of common positive labels they share. Simply put, if two samples share a large number of common positive labels, they are considered to be more similar. In addition, this embodiment can use cosine similarity to measure the similarity between the embedded features of two samples in each view:
[0071]
[0072] in and is the embedding feature of the vth view of the i-th and j-th samples output by the cross-view encoder. Then, this embodiment introduces the following graph-based cross entropy loss to combine the structure of the potential representation with the supervision label graph The target structure is aligned:
[0073]
[0074] in Represents the number of available sample pairs in all views.
[0075] Label-guided category labeling constraints
[0076] In the method of this embodiment, this embodiment introduces labels of c categories as the input of the cross-category encoder to improve the interaction between multi-view features and labels. Accordingly, this embodiment obtains c-category label embedding features, which can be regarded as discriminative representations of c categories. For these label embeddings, this embodiment also expects them to retain label correlations similar to the original labels. To this end, this embodiment introduces label guidance constraints on these label embeddings. First, this embodiment constructs a new label correlation graph Each element represents the probability that a sample has both the i-th and j-th labels, as shown below:
[0077]
[0078] in Represents the number of co-occurrences of category i and category j. Represents the sum of the number of occurrences of category i and category j. Similarly, for the label embedding features corresponding to the category labels output by the cross-category encoder, this embodiment can calculate their similarity based on the cosine distance as follows:
[0079]
[0080] in Represents the label embedding of the i-th category. In order to retain a structure similar to the label, this embodiment introduces the following loss on the label embedding:
[0081]
[0082] The pseudo-label strategy has been proven to be effective in semi-supervised learning and unsupervised learning tasks. Figure 1 The incomplete label learning network shown in the figure, during the training phase, the output of its classifier can actually be regarded as the "pseudo-label" of the input data. If these pseudo-labels can be used to restore the original missing label information, the amount of information of the label can be increased, which is expected to further improve the performance. To this end, this embodiment designs a pseudo-label uncertainty perception strategy to explore more reliable and high-quality supervisory information for incomplete label data. When designing the pseudo-label perception strategy of this embodiment, the key challenge is to evaluate the uncertainty of the predicted labels and select reliable pseudo-labels to guide model training. The method of this embodiment applies the Monte Carlo Dropout (MC dropout) method to quantify the uncertainty of the predicted labels. Dropout is achieved by using a Bernoulli distribution to probabilistically deactivate neurons, so that the network's output y* for the input x* follows a predicted distribution rather than a definite value. Consider a deep neural network with L layers, each with N in the i-th layer. i neurons, and the network parameters are expressed as in According to the standard dropout following the Bernoulli distribution, this embodiment performs subsampling to approximate the predictive distribution and obtain A set of binary vectors, each of which represents a unique dropout operation across layers, is represented as in is the binary vector of the i-th layer in the t-th dropout operation. The dropout of each neuron From the Bernoulli distribution Where p is the dropout ratio used during training. Then this embodiment can obtain Group network parameter set, expressed as follows:
[0083]
[0084] in represents the Hadamard product. Figure 2 As shown, given a data set X with n samples, this embodiment can use Perform a random forward pass times and obtained Group Output In this embodiment, the average value of the output can be estimated as follows:
[0085]
[0086] in Indicates passing The approximate predicted distribution after sampling. Then this embodiment estimates:
[0087]
[0088] Finally, this embodiment can calculate the variance of the model output:
[0089]
[0090] Then, this embodiment can normalize the variance to obtain the uncertainty of the output prediction label (ie, pseudo label):
[0091]
[0092] Among them U i,j represents the uncertainty of the jth output of the ith sample, Afterwards, this embodiment can The predicted average of the output Consider them as candidate pseudo labels, and select reliable pseudo labels from them according to the uncertainty. This embodiment adopts a self-paced learning selection strategy to select these reliable pseudo labels. First, this embodiment introduces an adjustment parameter λ and initializes its value to 0, and then gradually increases its value as shown below:
[0093]
[0094] Where e represents the number of training iterations in the eth round, Represents the total number of training iterations. Specifically, this embodiment introduces the temperature coefficient To control the growth rate of λ. Gradually increasing λ can select more credible candidate pseudo labels as the number of training iterations increases. Then this embodiment constructs the weight matrix as follows:
[0095]
[0096] Where λ is a gradually increasing dynamic threshold (calculated as (16)). In this embodiment, the output position with uncertainty less than this threshold is marked as 1 in D for subsequent selection. In order to select pseudo labels for missing labels, this embodiment needs to adjust the weight matrix according to the missing label index G in Its element value can be used to indicate the position of the missing label in the label matrix. Then this embodiment can select the soft pseudo label of the next period from the candidate pseudo labels:
[0097]
[0098] Finally, this embodiment can obtain the final output Where Ψ is the final classifier. This embodiment then uses the masked binary cross entropy function to calculate the following multi-label classification loss, which combines the known labels and pseudo labels as follows:
[0099]
[0100] Among them, the missing label indication matrix G is introduced in the first term to calculate the loss of the observed label, and the second term Represents the weight matrix used to select soft pseudo labels from the previous round of training.
[0101] In general, the total model loss function of this embodiment can be expressed as:
[0102]
[0103] Where α and β represent the penalty coefficients.
[0104] The entire model is divided into two parts: training and testing (also known as the application phase):
[0105] Model training:
[0106] 1) Training preparation stage:
[0107] 1. Prepare multi-view training data Incomplete label set Y∈{0,1} n×c , missing view indicator matrix W∈{0,1} n×m , missing label indicator matrix G∈{0,1} n×c .
[0108] 2. Set the hyperparameter β, Learning rate, training iteration parameters
[0109] 3. Fill all missing views and missing labels with '0'.
[0110] 4. Initialize network model parameters.
[0111] 2) Model training phase
[0112] 1. Input training data.
[0113] 2. Through the multi-layer perceptron corresponding to each view Mapping the original multi-view data into the shared feature space
[0114] 3. Calculate the average view tag AVT using formula (1).
[0115] 4. Extract the embedded features of each training sample after cross-view interaction through the cross-view encoder Γ
[0116] 5. From Extract the first m rows as the embedded view feature Z of each sample i .
[0117] 6. Calculate the feature-level graph constraint loss using formula (7)
[0118] 7. Calculate the multi-view joint representation f through formula (4).
[0119] 8. Extract the embedding features after view-category interaction through the cross-category encoder Υ
[0120] 9. Calculate the label-level graph constraint loss using formula (10)
[0121] 10. From Extract the final classification representation
[0122] 11. Calculate the classification prediction result O through the classifier Ψ.
[0123] 12. Calculate the uncertainty U using formula (5).
[0124] 13. Calculate the pseudo label using formula (17)
[0125] 14. Calculate the classification loss using formula (18)
[0126] 15. Calculate the total loss using formula (19) And update the model parameters.
[0127] 16. Determine whether the maximum number of training rounds has been reached If it is reached, go to step 17, otherwise return to step 2.
[0128] 17. Save the model parameters and end the training.
[0129] Model testing (application):
[0130] 1) Test (application) preparation stage:
[0131] 1. Prepare multi-view training data Incomplete label set Y∈{0,1} n×c , missing view indicator matrix W∈{0,1} n×n , missing label indicator matrix G∈{0,1} n×c .
[0132] 2. Fill all missing views with '0'.
[0133] 3. Load the trained network model parameters.
[0134] 2) Model testing phase
[0135] 1. Enter test data.
[0136] 2. Through the multi-layer perceptron corresponding to each view Mapping the original multi-view data into the shared feature space
[0137] 3. Calculate the average view tag AVT using formula (1).
[0138] 4. Extract the embedded features of cross-view interactions of each test sample (sample to be classified) through the cross-view encoder Γ
[0139] 5. From Extract the first m rows as the embedded view features Z of the input test (to be classified) sample i .
[0140] 6. Calculate the multi-view joint representation f through formula (4).
[0141] 7. Extracting embedding features after view-category interaction through cross-category encoder Υ
[0142] 8. From Extract the final classification representation
[0143] 9. Calculate the classification prediction result O through the classifier Ψ.
[0144] 10. Output the prediction results.
[0145] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.
Claims
1. An incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image drive, characterized in that: The following steps are involved: Obtain and integrate multi-view training data, incomplete label sets, missing view indication matrices, and missing label indication matrices to obtain an incomplete multi-view incomplete multi-label dataset; Constructing a cross-view encoder, inputting the incomplete multi-view incomplete multi-label dataset into the cross-view encoder, and generating an embedding feature for each view; Constructing a dynamic weighted fusion module, inputting the embedded features of each view into the dynamic weighted fusion module to generate a common representation of multiple views; Constructing a cross-category encoder, and obtaining the embedded features of the view-category interaction of the multi-view common representation through the cross-category encoder; A final classification representation is obtained based on the embedded features after the view-category interaction, and a classification prediction result is calculated for the final classification representation through a classifier.
2. The incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving according to claim 1 is characterized in that: The process of constructing a cross-view encoder further includes: Introduce the same average view token as an additional input to the cross-view encoder; The expression of the average view mark of the i-th sample is: Among them, Λ i represents the average view label of the i-th sample, m is the number of views, v indicates the v-th view, W is the missing view indication matrix, W i,v is the element in the i-th row and v-th column of the matrix W.
3. The incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving according to claim 1 is characterized in that: For the cross-view encoder input of the i-th sample, the expressions for view correlation and output are: in, is the view dependency, is the output, d h =d e / h represents the dimension of each attention head, is the output of the tth attention head, is the projection matrix, d e Represents the mapping dimension of multi-view data.
4. The incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving according to claim 1 is characterized in that: The expression for generating a common representation of multiple views is: in, Represents Z i The embedding feature of the vth view / row in a v represents the learnable scalar weight of the v-th view, f i is the fusion feature of sample i.
5. The incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving according to claim 1 is characterized in that: The cross-view encoder combines the structure of the latent representation with the supervision label graph based on the cross-entropy loss of the graph. Target structure alignment; The expression of the cross entropy loss of the graph is: in, represents the number of available sample pairs in all views, is the similarity between the embedded features of two samples in each view, Q i,j is the element in the i-th row and j-th column of the label-based correlation graph Q.
6. The incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving according to claim 1 is characterized in that: The loss function of the cross-category encoder is expressed as: in, is a label correlation graph, each element represents the probability that a sample has both the i-th and j-th labels, K i,j is the similarity between the category vector of the i-th category and the category vector of the j-th category based on the cosine distance.
7. The incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving according to claim 1 is characterized in that: Also includes: Quantify the uncertainty of predicted labels through Monte Carlo methods; The expression for the uncertainty of the quantitative prediction label is: Among them, U i,j represents the uncertainty of the jth output of the ith sample, is the variance of the model output.
8. The incomplete multi-view multi-label classification method based on pseudo-label uncertainty perception and dual-image driving according to claim 1 is characterized in that: The loss function of the classifier is expressed as: Among them, O is the final output, G i,j is the element in the i-th row and j-th column of the missing label indicator matrix G, The weight matrix representing the soft pseudo labels selected from the previous round of training The element in the i-th row and j-th column of ; is a pseudo label; The final output expression is: Among them, Ψ is the final classifier, It is the embedded feature after view-category interaction for final classification.
Citation Information
Patent Citations
Label guidance-based incomplete multi-view multi-label classification method and system
CN116108363A
Multi-view data classification method and related equipment
CN116975711A
Incomplete multi-view incomplete multi-label classification method and device
CN117556349A
Cited By
Incomplete multi-view multi-label classification method and system based on consistency representation extraction
CN121188630A
