Incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation

Abstract

The invention discloses an incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation. The method comprises the following steps: acquiring a plurality of incomplete multi-view incomplete multi-label data; constructing an incomplete multi-view incomplete multi-label learning model, inputting the incomplete multi-view incomplete multi-label data into the incomplete multi-view incomplete multi-label learning model for training, and obtaining a trained incomplete multi-view incomplete multi-label learning model; and obtaining complete or incomplete multi-view data to be detected, and inputting the complete or incomplete multi-view data to be detected into the trained incomplete multi-view incomplete multi-label learning model to complete multi-label classification of the multi-view data. According to the method, the complete label of each input sample is accurately predicted according to the partial or complete multi-view features of each input sample.

Description

Technical Field

[0001] The present invention belongs to the technical field of pattern recognition, and in particular relates to an incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation. Background Art

[0002] In recent years, the study of multi-view learning has attracted the attention of many researchers. Multi-view data refers to data obtained by using different data collection methods for an object. For example, an image can be described by its color features (RGB), scale-invariant feature transform (SIFT), etc. Compared with single-view data, multi-view data can provide a more comprehensive description of the research object. Multi-view representation learning is widely used in various applications, such as recommendation systems and bioinformatics analysis. The purpose of multi-view representation learning is to enable the model to learn a comprehensive feature representation of an object from multi-view data so that it can be effectively used for classification tasks. In classification tasks, an object can be annotated with multiple labels instead of a single label. Multiple labels can more comprehensively describe the sample. For example, a photo can be labeled as "person", "car", "road", "tree", etc. The goal of multi-view multi-label classification is to learn a representation from multi-view data to accurately capture the discriminative features of the sample, and then assign different labels to the sample. In recent years, some methods for multi-view multi-label classification have been proposed. Zhang et al. proposed a latent semantic perception model based on matrix decomposition, which exploits the complementarity between views to extract consistent representations of multiple views. Sun and Zong proposed a latent conditional Bernoulli mixture (LCBM) model, which uses the Bernoulli mixture model to construct label dependencies. LCBM uses the mean field variational inference framework to approximate posterior inference. Zhao et al. developed a method to maximize the dependency of view-specific labels on label features, enhancing the complementarity and consistency of label space. However, these methods assume that all views and labels are complete. In practical applications, incomplete views and missing labels are ubiquitous. Therefore, the classification of incomplete multi-view and incomplete multi-label is more practical. So Tan et al. proposed incomplete multi-view weak label learning (iMvWLL), which establishes the connection between sample feature space and label space through two-level matrix decomposition. iMvWLL first projects the sample features of multiple views into a common subspace through the first matrix decomposition, and then uses the second matrix decomposition to establish the connection between the feature space and the label space, and predicts the category of the sample. However, iMvWLL cannot predict the labels of new samples, but can only predict the labels of samples used for training, which is obviously not practical and scalable. Secondly, iMvWLL is implemented based on traditional methods, which limits the model's ability to extract high-order semantic information. Therefore, Wen et al. proposed a deep dual incomplete multi-view multi-label learning network (DD-ImvMLC-net) based on a neural network framework. DD-ImvMLC-net solves the above problems well. It can not only extract high-order semantic information of samples, but also make end-to-end multi-label predictions for new samples.However, the above methods all use an average weighting strategy when aggregating multi-view information, which means that they all assign equal weights to each view by default. But in practical applications, the information quality of each view is different, which means that the fusion weight of each view is also different, which leads to the problem that the fusion weight does not match the information quality of the view. In addition, since each view contains information about the same object, the information in multiple views should maintain a high degree of consistency. The above methods all ignore these problems. Summary of the invention

[0003] To solve the above technical problems, the present invention proposes an incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation, which accurately predicts the complete label of each input sample according to the partial or complete multi-view features of each input sample.

[0004] To achieve the above object, the present invention provides an incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation, comprising:

[0005] Obtain some incomplete multi-view incomplete multi-label data;

[0006] Constructing an incomplete multi-view incomplete multi-label learning model, inputting the incomplete multi-view incomplete multi-label data into the incomplete multi-view incomplete multi-label learning model for training, and obtaining a trained incomplete multi-view incomplete multi-label learning model;

[0007] Acquire complete or incomplete multi-view data to be tested, input the multi-view data to be tested into the trained incomplete multi-view incomplete multi-label learning model, and complete multi-label classification of the input data to be tested.

[0008] Optionally, the incomplete multi-view incomplete multi-label learning model includes: a view feature extraction module, a quality perception module based on view information, a dynamic weighted fusion module and a missing multi-label classification module;

[0009] The view feature extraction module is used to extract high-order semantic features of the incomplete multi-view incomplete multi-label data;

[0010] The quality perception module based on view information is used to capture the quality of high-order semantic features of the incomplete multi-view incomplete multi-label data, assign view fusion weights and obtain optimal fusion weights;

[0011] The dynamic weighted fusion module is used to dynamically weight the fusion by combining the allocated optimal fusion weight with the prior missing information of the view to obtain a fusion result;

[0012] The missing multi-label classification module is used to perform classification according to the fusion result to obtain a classification result.

[0013] Optionally, extracting high-order semantic features of the incomplete multi-view incomplete multi-label data includes:

[0014] X′ v =f v (X v )

[0015] Among them, X′ v and f v (.) are the discriminant features extracted from the v-th view and the encoder of the v-th view respectively; X v is the original feature of the data in the vth view.

[0016] Optionally, capturing the quality of the high-order semantic features of the incomplete multi-view incomplete multi-label data, assigning view fusion weights and obtaining optimal fusion weights includes:

[0017] Predict the class probability P of the sample in the vth view based on the extracted view features v , where the classifier consists of a fully connected layer and a sigmoid activation function. v Convert to label matrix

[0018] The classification accuracy θ of each view is calculated using the predicted label matrix and the true label matrix v and the index matrix of valid labels

[0019]

[0020] Among them, G and W are the indicator matrix of missing labels and the indicator matrix of missing views respectively; Indicates that the jth label of the i-th sample in the v-th view is valid;

[0021] By introducing the indicator matrix of valid labels, the classification accuracy θ of view v is calculated v :

[0022]

[0023] Among them, ⊙ represents the dot product operation, Indicates the same or same operation;

[0024] Based on the classification accuracy as an indicator of the quality of view feature information, the view fusion weight α is learned v :

[0025] Optionally, the optimal fusion weights are assigned and combined with the prior missing information of the view to perform dynamic weighted fusion, and the fusion results obtained include:

[0026]

[0027] Among them, Z i is the fusion representation of the i-th sample; is the new feature vector of the i-th sample extracted from the v-th view; a v is the fusion weight of the vth view;

[0028] In the classification process according to the fusion result, binary cross entropy is used to train the objective function, and the weighted loss is obtained as follows:

[0029]

[0030] in, is the classification loss of the fusion feature, n represents the number of samples, L represents the number of labels, P i,j is the final predicted probability of the i-th sample for the j-th label, Y i,j represents the jth true label of the i-th sample, G i,j Indicates whether the jth true label of the i-th sample in the training dataset is known.

[0031] Optionally, the view feature extraction module also introduces contrastive learning to capture the consistency information between multiple views, and the contrast loss is:

[0032]

[0033] in, It is to calculate two vectors and The cosine similarity function between neg is the sum of the cosine similarities of the negative sample pairs; W is the view missing indicator matrix; W i,v =1 means that the i-th sample is not missing in the v-th view, otherwise the sample is missing; represents the vth view of the i-th sample after encoder f v (.) Output features after.

[0034] Optionally, the quality perception module based on view information further introduces a classification loss related to the view:

[0035]

[0036] in, is the classification loss of the view, V represents the number of views, is the predicted probability of the jth category for the ith sample in the vth view, is the indicator matrix of valid labels in the vth view.

[0037] Optionally, the dynamic weighted fusion module further designs a consistency loss between the instance structure in the fusion feature space and the instance structure in the original space as follows:

[0038]

[0039] Among them, diag(W :,v ) is a diagonal matrix to avoid the negative impact of missing samples on the consistency structure; is the structural consistency loss; V is the number of views; n is the number of samples; Represents the difference in sample structure between fused features and original features.

[0040] Optionally, the classification loss of the missing multi-label classification module is:

[0041]

[0042] Optionally, the training loss function of the incomplete multi-view incomplete multi-label learning model is:

[0043]

[0044] Technical effect of the present invention: The present invention discloses an incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation, which provides reliable guidance for the fusion of multiple views by dynamically evaluating the importance of each view and assigning appropriate weights to the views, thereby obtaining reliable fusion features; fully explores the consistency information between multi-view data; introduces contrastive learning to enhance the consistency between features of the same sample in different views, and proposes consistency loss to increase the structural consistency between samples in the fused feature space and the original multi-view space; accurately predicts the complete label of each input sample based on partial or complete multi-view features of each input sample. BRIEF DESCRIPTION OF THE DRAWINGS

[0045] The drawings constituting a part of the present application are used to provide a further understanding of the present application. The illustrative embodiments and descriptions of the present application are used to explain the present application and do not constitute an improper limitation on the present application. In the drawings:

[0046] Figure 1 A flowchart of an incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation according to an embodiment of the present invention;

[0047] Figure 2 A schematic diagram of the process of model training according to an embodiment of the present invention;

[0048] Figure 3 Schematic diagram of the process of sample label prediction according to an embodiment of the present invention. DETAILED DESCRIPTION

[0049] It should be noted that, in the absence of conflict, the embodiments and features in the embodiments of the present application can be combined with each other. The present application will be described in detail below with reference to the accompanying drawings and in combination with the embodiments.

[0050] It should be noted that the steps shown in the flowcharts of the accompanying drawings can be executed in a computer system such as a set of computer executable instructions, and that, although a logical order is shown in the flowcharts, in some cases, the steps shown or described can be executed in an order different from that shown here.

[0051] like Figure 1 As shown, this embodiment provides an incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation, including:

[0052] Obtain some incomplete multi-view incomplete multi-label data;

[0053] An incomplete multi-view incomplete multi-label learning model is constructed, and incomplete multi-view incomplete multi-label data is input into the incomplete multi-view incomplete multi-label learning model for training to obtain a trained incomplete multi-view incomplete multi-label learning model;

[0054] Obtain complete or incomplete multi-view data to be tested, input the data to be tested into the trained incomplete multi-view incomplete multi-label learning model, and complete the multi-label classification of the multi-view data.

[0055] Optionally, the incomplete multi-view incomplete multi-label learning model includes: a view feature extraction module, a quality perception module based on view information, a dynamic weighted fusion module and a missing multi-label classification module;

[0056] View feature extraction module, used to extract high-order semantic features of incomplete multi-view incomplete multi-label data;

[0057] The quality-aware module based on view information is used to capture the quality of high-order semantic features of incomplete multi-view and incomplete multi-label data, assign view fusion weights and obtain the optimal fusion weights;

[0058] The dynamic weighted fusion module is used to dynamically weight the fusion by combining the optimal fusion weight with the prior missing information of the view to obtain the fusion result;

[0059] The missing multi-label classification module is used to classify according to the fusion results to obtain the classification results.

[0060] Optionally, extracting high-order semantic features of incomplete multi-view incomplete multi-label data includes:

[0061] X′ v =f v (X v )(1)

[0062] Among them, X′ v and f v (.) are the discriminant features extracted from the v-th view and the encoder of the v-th view respectively; X v is the original data in the vth view.

[0063] Optionally, the view feature extraction module also introduces contrastive learning to capture the consistency information between multiple views, and the contrast loss is:

[0064]

[0065] in, It is to calculate two vectors and The cosine similarity function between neg is the sum of the cosine similarities of the negative sample pairs; W is the view missing indicator matrix; W i,v =1 means that the i-th sample is not missing in the v-th view, otherwise the sample is missing; represents the vth view of the i-th sample after encoder f v (.) Output features after.

[0066] Optionally, capturing the quality of high-order semantic features of incomplete multi-view incomplete multi-label data, assigning view fusion weights and obtaining optimal fusion weights includes:

[0067] Predict the class probability P of the sample in the vth view based on the extracted view features v , where the classifier consists of a fully connected layer and a sigmoid activation function. v Convert to label matrix

[0068] The classification accuracy θ of each view is calculated using the predicted label matrix and the true label matrix v and the index matrix of valid labels

[0069]

[0070] Among them, G and W are the indicator matrix of missing labels and the indicator matrix of missing views respectively; Indicates that the jth label of the i-th sample in the v-th view is valid, otherwise it means that the label is invalid;

[0071] By introducing the indicator matrix of valid labels, the classification accuracy θ of view v is calculated v :

[0072]

[0073] Among them, n represents the number of samples, L represents the number of labels, ⊙ represents the dot product operation, Indicates the same or same operation; is the predicted probability label P v The converted label matrix.

[0074] Based on the classification accuracy as an indicator of the quality of view feature information, we learn the view fusion weight a v :

[0075] Optionally, the quality-aware module based on view information also introduces a view-related classification loss as:

[0076]

[0077] in, is the classification loss of the view, V represents the number of views, is the predicted probability of the jth category for the ith sample in the vth view, is the indicator matrix of valid labels in the vth view.

[0078] Optionally, the optimal fusion weights are assigned and combined with the prior missing information of the view to perform dynamic weighted fusion, and the fusion results obtained include:

[0079]

[0080] Among them, Z i is the fusion representation of the i-th sample; is the new feature vector of the i-th sample extracted from the v-th view; a v is the fusion weight of the vth view, and its value is adaptively learned by the view quality-aware module. In addition, to avoid the negative impact of missing views, the view weights are redistributed together with the view missing indicator matrix W.

[0081] Optionally, the dynamic weighted fusion module also designs a consistency loss between the instance structure in the fusion feature space and the instance structure in the original space:

[0082]

[0083] Among them, diag(W :,v ) is a diagonal matrix to avoid the negative impact of missing samples on the consistency structure; is the structural consistency loss; V is the number of views; n is the number of samples; Represents the difference in sample structure between fused features and original features.

[0084] The present invention uses a simple classifier to connect the fused feature space with the label space to obtain an end-to-end multi-label prediction P for any input multi-view data. The classifier consists of a fully connected layer and a sigmoid activation function. The fully connected layer scores each category of each sample, and uses the activation function to normalize the evaluation score of each category to the range of [0,1]. Binary cross entropy (BCE) is used as the objective function to guide model training. In order to eliminate the negative impact of missing labels, the present invention adopts the following weighted BCE loss

[0085]

[0086] in, is the classification loss of the fusion feature, n represents the number of samples, L represents the number of labels, P i,j is the final predicted probability of the i-th sample for the j-th label, Y i,j represents the jth true label of the i-th sample, G i,j Indicates whether the jth true label of the i-th sample in the training dataset is known.

[0087] Optionally, the classification loss for the missing multi-label classification module is:

[0088]

[0089] Optionally, the training loss function for the incomplete multi-view incomplete multi-label learning model is:

[0090]

[0091] Among them, α and β represent the penalty coefficients of contrast loss and consistency loss, respectively.

[0092] like Figure 2 As shown in the figure, the model training process includes:

[0093] First, we give the problem description and symbolic description of incomplete multi-view incomplete multi-label learning and classification: the original multi-view data is represents, Y represents the corresponding label. V is the number of views, n is the number of samples, d v is the number of sample feature dimensions in the vth view. Y∈{0,1} n×L ,L represents the number of labels, Y i,j=1 indicates that sample i has the jth label, otherwise it does not have the label. In addition, the view missing indication matrix W∈{0,1} is introduced n×V and label missing indicator matrix G∈{0,1} n×L , W i,j = 1 means that the i-th sample exists in the j-th view, W i,j = 0 means that sample i is missing in view j; G i,j =1 indicates that sample i is certain about whether it contains the jth label, otherwise the label information is uncertain (missing). Given incomplete multi-view training data with incomplete multi-labels, the goal of the present invention is to train a classification model to accurately predict the complete label of each input sample based on partial or complete multi-view features of each input sample.

[0094] The specific training steps of the present invention are described as follows:

[0095] Step 1: Data preprocessing: The present invention first fills the missing sample features and missing labels with 0, and defines the initial iteration number of model training as t=1.

[0096] Step 2: Data division. The preprocessed feature matrix and label matrix are randomly divided into non-repeating batches, of which Indicates rounding up.

[0097] Step 3: Set the current batch i=0.

[0098] Step 4: Deep feature extraction. The i-th batch of data ( Y, W and G) are input into the specific view feature extraction network to obtain the discriminative representation of each view, denoted as

[0099] Step 5: Compute contrast loss. Calculate the contrast loss of multiple views according to formula (2):

[0100] Step 6: Calculate the view fusion weight. The features extracted from each view are input into the quality perception module based on view-specific information. The fusion weight of each view is calculated using formula (5). The classification loss of the module is calculated using formula (6).

[0101] Step 7: Weighted fusion of multi-view features. The features extracted from each view in the fourth step are input into the dynamic weighted fusion module, and the fusion representation Z of the input multi-view data is obtained by formula (7). Since the true label is required to calculate the fusion weight, and the true label is not used as the input of the classification model, the calculation of this step cannot be performed directly in the network. Therefore, the present invention uses an average weighted strategy in the first batch of the first iteration, and then corrects the fusion weight of the view during the gradient update optimization. After the last gradient update, the fusion weight of each view will be fixed and used in the test phase.

[0102] Step 8: Multi-label classification. The fusion representation of the sample obtained in the previous step is input into the multi-label classification module, and the label prediction probability value P of the input sample is obtained through the missing multi-label classification module. The classification loss of the fusion feature is calculated according to formula (9):

[0103] Step 9: Calculate network loss. Calculate the total loss of the model for the i-th batch,

[0104] Step 10: Gradient feedback optimization. Based on the network loss obtained in the previous step Use the gradient descent optimization algorithm to perform parameter optimization. If i < m, let i = i + 1, then update the network parameters and continue to execute the fourth step; if i = m, execute the eleventh step.

[0105] Step 11: Convergence judgment. When the number of iterations t>20 and When , stop the training of the network model and output the network model parameters; otherwise t=t+1 and jump to the second step to continue execution. and denote the total losses at the t-th step and the t-1-th step respectively.

[0106] like Figure 3 As shown in Figure 2, the test sample label prediction process includes:

[0107] The complete or incomplete multi-view test data to be tested is represented as An index matrix indicating whether its view is missing. The specific test process is described as follows:

[0108] Step 1: Data preprocessing: fill missing view features with 0 values.

[0109] Step 2: Deep feature extraction. Input a specific view feature extraction network to obtain high-level semantic features extracted from each view The specific view feature extraction network parameters in this step are the trained model parameters.

[0110] Step 3: Weighted feature fusion. The discriminant features and the view missing indicator matrix of the previous step are combined. Input weighted feature fusion module, according to the formula Get the fusion representation Z of the test data. v is the view fusion weight obtained through training.

[0111] Step 4: Multi-label classification. Input the fused feature Z into the multi-label classification module to obtain the label prediction result P of the input incomplete multi-view data. In this step, the parameters of the classifier are the model parameters trained in Section 3.2.1.

[0112] Step 5: Output the recognition results. Define rules based on multi-label classification labels Get the final 0-1 binary label matrix of the input data

[0113] The technical effects of the present invention are compared with those of the prior art DD-ImvMLC-net. As shown in Table 1, the average precision (AP) of the present invention and DD-ImvMLC-net on five data sets when the view missing rate is 50%, the label missing rate is 50%, and the training sample rate is 70%.

[0114] Table 1

[0115]

[0116]

[0117] A specific application embodiment of the present invention is:

[0118] In an intelligent monitoring system, multiple cameras monitor the same area from different angles to ensure safety. However, due to various reasons (such as camera failure, occlusion, network problems), not all cameras can always provide complete video data, resulting in incomplete data. At the same time, the monitoring system needs to identify multiple events or behaviors, that is, multi-label classification problems, such as intrusion, fighting, fire, fall, etc. However, in practical applications, due to negligence or fatigue of the labeling personnel, lack of experience, unclear events, data loss and other reasons, some video labels may be missing. That is, incomplete multi-view incomplete multi-label training data is common in the field of intelligent monitoring, and the traditional multi-view multi-label learning model cannot use this type of data for model training, and is therefore not suitable for processing the event classification task of incomplete multi-view detection data in the field of intelligent monitoring, that is, the existing multi-view multi-label classification model has limited adaptability in similar intelligent monitoring applications. The model designed by the present invention can better handle incomplete multi-view incomplete multi-label model learning and incomplete or complete multi-view multi-label classification tasks similar to the above-mentioned intelligent monitoring.

[0119] The present invention discloses an incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation. By dynamically evaluating the importance of each view and assigning appropriate weights to the views, reliable guidance is provided for the fusion of multiple views, thereby obtaining credible fusion features. The consistency information between multi-view data is fully explored. Contrastive learning is introduced to enhance the consistency between features of the same sample in different views, and consistency loss is proposed to increase the structural consistency between samples in the fused feature space and the original multi-view space. The complete label of each input sample is accurately predicted based on partial or complete multi-view features of each input sample.

[0120] The above are only preferred specific implementations of the present application, but the protection scope of the present application is not limited thereto. Any changes or substitutions that can be easily thought of by a person skilled in the art within the technical scope disclosed in the present application should be included in the protection scope of the present application. Therefore, the protection scope of the present application should be based on the protection scope of the claims.

Claims

1. An incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation, characterized in that: include: Obtain some incomplete multi-view incomplete multi-label data; Constructing an incomplete multi-view incomplete multi-label learning model, inputting the incomplete multi-view incomplete multi-label data into the incomplete multi-view incomplete multi-label learning model for training, and obtaining a trained incomplete multi-view incomplete multi-label learning model; Acquire complete or incomplete multi-view data to be tested, input the multi-view data to be tested into the trained incomplete multi-view incomplete multi-label learning model, and complete multi-label classification of the input data to be tested.

2. The incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation according to claim 1, characterized in that: The incomplete multi-view incomplete multi-label learning model includes: a view feature extraction module, a quality perception module based on view information, a dynamic weighted fusion module and a missing multi-label classification module; The view feature extraction module is used to extract high-order semantic features of the incomplete multi-view incomplete multi-label data; The quality perception module based on view information is used to capture the quality of high-order semantic features of the incomplete multi-view incomplete multi-label data, assign view fusion weights and obtain optimal fusion weights; The dynamic weighted fusion module is used to dynamically weight the fusion by combining the allocated optimal fusion weight with the prior missing information of the view to obtain a fusion result; The missing multi-label classification module is used to perform classification according to the fusion result to obtain a classification result.

3. The incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation according to claim 2, characterized in that: Extracting high-order semantic features of the incomplete multi-view incomplete multi-label data includes: X′ v =f v (X v ) Among them, X′ v and f v (.) are the discriminant features extracted from the v-th view and the encoder of the v-th view respectively; X v is the original feature of the data in the vth view.

4. The incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation according to claim 2, characterized in that: Capturing the quality of the high-order semantic features of the incomplete multi-view incomplete multi-label data, allocating view fusion weights and obtaining optimal fusion weights includes: Predict the class probability P of the sample in the vth view based on the extracted view features v , where the classifier consists of a fully connected layer and a sigmoid activation function. v Convert to label matrix The classification accuracy θ of each view is calculated using the predicted label matrix and the true label matrix v and the index matrix of valid labels Among them, G and W are the indicator matrix of missing labels and the indicator matrix of missing views respectively; Indicates that the jth label of the i-th sample in the v-th view is valid; By introducing the indicator matrix of valid labels, the classification accuracy θ of view v is calculated v : Among them, ⊙ represents the dot product operation, Indicates the same or same operation; Based on the classification accuracy as an indicator of the quality of view feature information, the view fusion weight a is learned v :

5. The incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation according to claim 2, characterized in that: The optimal fusion weights are assigned and combined with the prior missing information of the view for dynamic weighted fusion. The fusion results include: Among them, Z i is the fusion representation of the i-th sample; is the new feature vector of the i-th sample extracted from the v-th view; a v is the fusion weight of the vth view; In the classification process according to the fusion result, binary cross entropy is used to train the objective function, and the weighted loss is obtained as follows: in, is the classification loss of the fusion feature, n represents the number of samples, L represents the number of labels, P i，j is the final predicted probability of the i-th sample for the j-th label, Y i，j represents the jth true label of the i-th sample, G i，j Indicates whether the jth true label of the i-th sample in the training dataset is known.

6. The incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation according to claim 5, characterized in that: The view feature extraction module also introduces contrastive learning to capture the consistency information between multiple views, and the contrast loss is: in, It is to calculate two vectors and The cosine similarity function between neg is the sum of the cosine similarities of the negative sample pairs; W is the view missing indicator matrix; W i，v =1 means that the i-th sample is not missing in the v-th view, otherwise the sample is missing; represents the vth view of the i-th sample after encoder f v (.) Output features after.

7. The incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation according to claim 6, characterized in that: The quality-aware module based on view information also introduces a view-related classification loss: in, is the classification loss of the view, V represents the number of views, is the predicted probability of the jth category for the ith sample in the vth view, is the indicator matrix of valid labels in the vth view.

8. The incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation according to claim 7, characterized in that: The dynamic weighted fusion module also designs the consistency loss between the instance structure in the fusion feature space and the instance structure in the original space as follows: Among them, diag(W ：，v ) is a diagonal matrix to avoid the negative impact of missing samples on the consistency structure; is the structural consistency loss; V is the number of views; n is the number of samples; Represents the difference in sample structure between fused features and original features.

9. The incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation as claimed in claim 8, characterized in that: The classification loss of the missing multi-label classification module is:

10. The incomplete multi-view incomplete multi-label learning and classification method based on view information quality estimation according to claim 9, characterized in that: The training loss function of the incomplete multi-view incomplete multi-label learning model is:

Images