A method for learning and classifying incomplete multi-view and incomplete multi-label systems based on view information quality estimation

Abstract

This invention discloses a method for learning and classifying incomplete multi-view, incomplete multi-label data based on view information quality estimation. The method includes: acquiring several incomplete multi-view, incomplete multi-label data sets; constructing an incomplete multi-view, incomplete multi-label learning model; inputting the incomplete multi-view, incomplete multi-label data into the incomplete multi-view, incomplete multi-label learning model for training; and acquiring test complete or incomplete multi-view data; inputting the test complete or incomplete multi-view data into the trained incomplete multi-view, incomplete multi-label learning model to complete the multi-label classification of the multi-view data. This invention accurately predicts the complete label of each input sample based on the partial or complete multi-view features of each input sample.

Description

Technical Field

[0001] This invention belongs to the field of pattern recognition technology, and in particular relates to a method for learning and classifying incomplete multi-view and incomplete multi-label based on view information quality estimation. Background Technology

[0002] In recent years, research on multi-view learning has attracted much attention from researchers. Multi-view data refers to data obtained by using different data acquisition methods on an object. For example, an image can be described by its color features (RGB), scaling-invariant feature transform (SIFT), etc. Compared with single-view data, multi-view data can provide a more comprehensive description of the research object. Multi-view representation learning is widely used in various applications, such as recommender systems and bioinformatics analysis. The goal of multi-view representation learning is to enable models to learn a comprehensive feature representation of an object from multi-view data, thus making it effective for classification tasks. In classification tasks, an object can be annotated with multiple labels instead of a single label. Multiple labels can more comprehensively describe the sample; for example, a photo can be labeled as "person," "car," "road," "tree," etc. The goal of multi-view multi-label classification is to learn a representation from multi-view data to accurately capture the discriminative features of the sample and then assign different labels to the sample. In recent years, several methods for multi-view multi-label classification have been proposed. Zhang et al. proposed a latent semantic awareness model based on matrix factorization, which utilizes the complementarity between views to extract a consistent representation of multiple views. Sun and Zong proposed a Latent Conditional Bernoulli Mixture (LCBM) model, which uses a Bernoulli mixture model to construct label dependencies. LCBM utilizes a mean-field variational inference framework to approximate posterior inference. Zhao et al. developed a method to maximize the dependence of view-specific labels on label features, enhancing the complementarity and consistency of the label space. However, these methods all assume that all views and labels are complete. In practical applications, incomplete views and missing labels are common. Therefore, classification of incomplete multi-view and incomplete multi-label data is more practically meaningful. Tan et al. proposed Incomplete Multi-View Weak Label Learning (iMvWLL), which establishes a connection between the sample feature space and the label space through two-level matrix factorization. iMvWLL first projects the sample features of multiple views into a common subspace through the first matrix factorization, and then uses the second matrix factorization to establish a connection between the feature space and the label space, predicting the sample category. However, iMvWLL cannot predict the label of new samples; it can only predict the labels of samples used for training, which is clearly impractical and lacks scalability. Secondly, iMvWLL is based on traditional methods, which limits the model's ability to extract high-order semantic information. Therefore, Wen et al. proposed the Deep Dual Incomplete Multi-View Multi-Label Learning Network (DD-ImvMLC-net) based on a neural network framework. DD-ImvMLC-net effectively solves the above problems, not only extracting high-order semantic information of samples, but also making end-to-end multi-label predictions for new samples.However, the methods described above all use an average weighting strategy when aggregating information from multiple views. This means they all assume that each view is assigned an equal weight, but in practice, the information quality of each view is different, which means that the fusion weights for each view are also different. This leads to a mismatch between the fusion weights and the information quality of the views. Furthermore, since each view contains information about the same object, the information across multiple views should maintain a high degree of consistency. The methods described above ignore these issues. Summary of the Invention

[0003] To address the aforementioned technical problems, this invention proposes an incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation. This method accurately predicts the complete label of each input sample based on the partial or complete multi-view features of each input sample.

[0004] To achieve the above objectives, this invention provides a method for learning and classifying incomplete multi-view, incomplete multi-label systems based on view information quality estimation, comprising:

[0005] Retrieve several incomplete multi-view, incomplete multi-label data;

[0006] Construct an incomplete multi-view, incomplete multi-label learning model, input the incomplete multi-view, incomplete multi-label data into the incomplete multi-view, incomplete multi-label learning model for training, and obtain a trained incomplete multi-view, incomplete multi-label learning model.

[0007] Obtain complete or incomplete multi-view data to be tested, input the multi-view data to be tested into the trained incomplete multi-view incomplete multi-label learning model, and complete the multi-label classification of the input data to be tested.

[0008] Optionally, the incomplete multi-view incomplete multi-label learning model includes: a view feature extraction module, a quality perception module based on view information, a dynamic weighted fusion module, and a missing multi-label classification module;

[0009] The view feature extraction module is used to extract high-order semantic features from the incomplete multi-view and incomplete multi-label data;

[0010] The quality perception module based on view information is used to capture the quality of the high-order semantic features of the incomplete multi-view and incomplete multi-label data, assign view fusion weights, and obtain the optimal fusion weights.

[0011] The dynamic weighted fusion module is used to dynamically weight and fuse the allocated optimal fusion weights with the prior missing information of the view to obtain the fusion result.

[0012] The missing multi-label classification module is used to classify based on the fusion result and obtain the classification result.

[0013] Optionally, extracting high-order semantic features from the incomplete multi-view, incomplete multi-label data includes:

[0014] X′ v =f v (X v )

[0015] Where, X′ v and f v (.) represent the discriminative features extracted from the v-th view and the encoder of the v-th view, respectively; X v These are the original features of the data in the v-th view.

[0016] Optionally, capturing the quality of high-order semantic features of the incomplete multi-view, incomplete multi-label data, assigning view fusion weights, and obtaining the optimal fusion weights includes:

[0017] Predict the class probability P of the sample in the v-th view based on the extracted view features. v The classifier consists of a fully connected layer and a sigmoid activation function, which will convert P... v Convert to label matrix

[0018] The classification accuracy θ for each view is calculated using the predicted label matrix and the true label matrix. v and the index matrix of valid tags

[0019]

[0020] Where G and W are the indicator matrix for missing labels and the indicator matrix for missing views, respectively; This indicates that the j-th label of the i-th sample in the v-th view is valid;

[0021] By introducing an indicator matrix with effective labels, the classification accuracy θ of view v is calculated. v :

[0022]

[0023] Here, ⊙ represents the dot product operation. Indicates the XOR operation;

[0024] Based on the classification accuracy as an indicator of the quality of view feature information, the view fusion weight α is learned. v :

[0025] Optionally, the optimal fusion weights are combined with prior missing information from the view for dynamic weighted fusion, resulting in fusion results including:

[0026]

[0027] Among them, Z i Let be the fused representation of the i-th sample; Let a be the new feature vector of the i-th sample extracted from the v-th view; v Let the fusion weight be the value of the v-th view.

[0028] In the classification process based on the fusion result, binary cross-entropy is used to train the objective function, and the weighted loss is obtained as follows:

[0029]

[0030] in, It is a classification loss based on fused features, where n represents the number of samples, L represents the number of labels, and P i,j Y is the final predicted probability of the i-th sample with respect to the j-th label. i,j Let G represent the j-th true label of the i-th sample. i,j This indicates whether the true label of the j-th sample given in the training dataset is known.

[0031] Optionally, the view feature extraction module also introduces contrastive learning to capture consistency information among multiple views, with the contrastive loss being:

[0032]

[0033] in, It calculates two vectors. and The cosine similarity function between them; S neg W is the sum of the cosine similarities of the negative sample pairs; W is the view missing indicator matrix; W i,v =1 indicates that the i-th sample is not missing in the v-th view; otherwise, the sample is missing. This indicates that the v-th view of the i-th sample has been processed by encoder f. v Output features following (.).

[0034] Optionally, the quality perception module based on view information also introduces a view-related classification loss as follows:

[0035]

[0036] in, It is the classification loss of the views, where V represents the number of views. It is the predicted probability of the j-th class for the i-th sample in the v-th view. It is the indicator matrix of valid labels in the v-th view.

[0037] Optionally, the dynamic weighted fusion module also designs a consistency loss between the instance structure in the fused feature space and the instance structure in the original space as follows:

[0038]

[0039] Among them, diag(W) :,v () is a diagonal matrix to avoid the negative impact of missing samples on the consistency structure; The structural consistency loss is represented by V; the number of views is represented by n; and the number of samples is represented by n. This indicates the difference in sample structure between the fused features and the original features.

[0040] Optionally, the classification loss of the missing multi-label classification module is:

[0041]

[0042] Optionally, the training loss function of the incomplete multi-view, incomplete multi-label learning model is:

[0043]

[0044] Technical effects of this invention: This invention discloses an incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation. By dynamically evaluating the importance of each view and assigning appropriate weights to the views, it provides reliable guidance for the fusion of multiple views, thereby obtaining credible fusion features. It fully explores the consistency information between multi-view data; introduces contrastive learning to enhance the consistency of features of the same sample across different views; and proposes a consistency loss to increase the structural consistency between samples in the fusion feature space and the original multi-view space. Based on the partial or complete multi-view features of each input sample, it accurately predicts the complete label of each input sample. Attached Figure Description

[0045] The accompanying drawings, which form part of this application, are used to provide a further understanding of this application. The illustrative embodiments and descriptions of this application are used to explain this application and do not constitute an undue limitation of this application. In the drawings:

[0046] Figure 1 This is a flowchart illustrating an embodiment of the present invention of an incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation;

[0047] Figure 2 This is a schematic diagram of the model training process in an embodiment of the present invention;

[0048] Figure 3 This is a schematic diagram of the sample label prediction process in an embodiment of the present invention. Detailed Implementation

[0049] It should be noted that, unless otherwise specified, the embodiments and features described in this application can be combined with each other. This application will now be described in detail with reference to the accompanying drawings and embodiments.

[0050] It should be noted that the steps shown in the flowchart in the accompanying drawings can be executed in a computer system such as a set of computer-executable instructions, and although a logical order is shown in the flowchart, in some cases the steps shown or described may be executed in a different order than that shown here.

[0051] like Figure 1 As shown, this embodiment provides a method for learning and classifying incomplete multi-view, incomplete multi-label learning based on view information quality estimation, including:

[0052] Retrieve several incomplete multi-view, incomplete multi-label data;

[0053] Construct an incomplete multi-view, incomplete multi-label learning model, input incomplete multi-view, incomplete multi-label data into the incomplete multi-view, incomplete multi-label learning model for training, and obtain a trained incomplete multi-view, incomplete multi-label learning model.

[0054] Obtain complete or incomplete multi-view data to be tested, input the data to be tested into a trained incomplete multi-view incomplete multi-label learning model, and complete the multi-label classification of multi-view data.

[0055] Optionally, the incomplete multi-view and incomplete multi-label learning model includes: a view feature extraction module, a quality perception module based on view information, a dynamic weighted fusion module, and a missing multi-label classification module;

[0056] The view feature extraction module is used to extract high-order semantic features from incomplete multi-view and incomplete multi-label data;

[0057] The quality perception module based on view information is used to capture the quality of high-order semantic features of incomplete multi-view and incomplete multi-label data, assign view fusion weights, and obtain the optimal fusion weights.

[0058] The dynamic weighted fusion module is used to dynamically weight and fuse the optimal fusion weights with the prior missing information of the view to obtain the fusion result.

[0059] The multi-label classification module is missing, which is used to classify based on the fusion result and obtain the classification result.

[0060] Optionally, extracting higher-order semantic features from incomplete multi-view and incomplete multi-label data includes:

[0061] X′ v =f v (X v (1)

[0062] Where, X′ v and f v (.) represent the discriminative features extracted from the v-th view and the encoder of the v-th view, respectively; X v This represents the original data in the v-th view.

[0063] Optionally, the view feature extraction module also introduces contrastive learning to capture consistency information among multiple views, with the contrastive loss being:

[0064]

[0065] in, It calculates two vectors. and The cosine similarity function between them; S neg W is the sum of the cosine similarities of the negative sample pairs; W is the view missing indicator matrix; W i,v =1 indicates that the i-th sample is not missing in the v-th view; otherwise, the sample is missing. This indicates that the v-th view of the i-th sample has been processed by encoder f. v Output features following (.).

[0066] Optionally, the quality of capturing high-order semantic features of incomplete multi-view and incomplete multi-label data, and the allocation of view fusion weights to obtain the optimal fusion weights include:

[0067] Predict the class probability P of the sample in the v-th view based on the extracted view features. v The classifier consists of a fully connected layer and a sigmoid activation function, which will convert P... v Convert to label matrix

[0068] The classification accuracy θ for each view is calculated using the predicted label matrix and the true label matrix. v and the index matrix of valid tags

[0069]

[0070] Where G and W are the indicator matrix for missing labels and the indicator matrix for missing views, respectively; This indicates that the j-th label of the i-th sample in the v-th view is valid; otherwise, the label is invalid.

[0071] By introducing an indicator matrix with effective labels, the classification accuracy θ of view v is calculated. v :

[0072]

[0073] Where n represents the number of samples, L represents the number of labels, and ⊙ represents the dot product operation. Indicates the XOR operation; It is determined by the predicted probability label P v The resulting tag matrix.

[0074] Based on classification accuracy as an indicator of the quality of view feature information, the view fusion weight 'a' is learned. v :

[0075] Optionally, the quality perception module based on view information also introduces a view-related classification loss:

[0076]

[0077] in, It is the classification loss of the views, where V represents the number of views. It is the predicted probability of the j-th class for the i-th sample in the v-th view. It is the indicator matrix of valid labels in the v-th view.

[0078] Optionally, the optimal fusion weights are combined with prior missing information from the view for dynamic weighted fusion, resulting in fusion results including:

[0079]

[0080] Among them, Z i Let be the fused representation of the i-th sample; Let a be the new feature vector of the i-th sample extracted from the v-th view; v Let be the fusion weight for the v-th view, whose value is adaptively learned by the view quality awareness module. Furthermore, to avoid the negative impact of missing views, the view weights are redistributed along with the view missing indicator matrix W.

[0081] Optionally, the dynamic weighted fusion module also designs a consistency loss between the instance structure in the fused feature space and the instance structure in the original space as follows:

[0082]

[0083] Among them, diag(W) :,v () is a diagonal matrix to avoid the negative impact of missing samples on the consistency structure; The structural consistency loss is represented by V; the number of views is represented by n; and the number of samples is represented by n. This indicates the difference in sample structure between the fused features and the original features.

[0084] This invention employs a simple classifier that connects the fused feature space and label space to obtain an end-to-end multi-label prediction P for any input multi-view data. The classifier consists of a fully connected layer and a sigmoid activation function. The fully connected layer scores each class for each sample, and the activation function normalizes the score for each class to the range [0,1]. Binary cross-entropy (BCE) is used as the objective function to guide model training. To eliminate the negative impact of missing labels, this invention uses the following weighted BCE loss.

[0085]

[0086] in, It is a classification loss based on fused features, where n represents the number of samples, L represents the number of labels, and P i,j Y is the final predicted probability of the i-th sample with respect to the j-th label. i,j Let G represent the j-th true label of the i-th sample. i,j This indicates whether the true label of the j-th sample given in the training dataset is known.

[0087] Optionally, the classification loss for the missing multi-label classification module is:

[0088]

[0089] Optionally, the training loss function for the incomplete multi-view, incomplete multi-label learning model is:

[0090]

[0091] Where α and β represent the penalty coefficients for contrast loss and consistency loss, respectively.

[0092] like Figure 2 As shown, the model training process includes:

[0093] First, a problem description and symbol explanation for incomplete multi-view and incomplete multi-label learning and classification are given: the original multi-view data is used... In this diagram, Y represents the corresponding label. V is the number of views. n is the number of samples, d v Y represents the number of feature dimensions of the samples in the v-th view. Y∈{0,1} n×L L represents the number of tags, Y i,j=1 indicates that sample i has the j-th label; otherwise, it does not have that label. Additionally, a view missing indicator matrix W∈{0,1} is introduced. n×V The missing label indicator matrix G∈{0,1} n×L W i,j =1 indicates that the i-th sample exists in the j-th view, W i,j =0 indicates that sample i is missing in view j; G i,j =1 indicates that sample i is certain whether it contains the j-th label; otherwise, the label information is uncertain (missing). For a given incomplete multi-view training data with incomplete multi-labels, the goal of this invention is to train a classification model that accurately predicts the complete label of each input sample based on the partial or complete multi-view features of each input sample.

[0094] The specific training steps of this invention are described below:

[0095] Step 1: Data Preprocessing. This invention first fills in missing sample features and missing labels with 0, and defines the initial number of iterations for model training as t = 1.

[0096] Step 2: Data Partitioning. The preprocessed feature matrix and label matrix are randomly partitioned into... Non-repeating batches, among which This indicates rounding up to the nearest integer.

[0097] Step 3: Set the current batch i = 0.

[0098] Step 4: Deep Feature Extraction. This involves extracting the data from the i-th batch mentioned above ( Y, W, and G are input into a specific view feature extraction network to obtain discriminative representations of each view, denoted as .

[0099] Step 5: Calculate the contrast loss. Calculate the contrast loss of the multiple views according to equation (2).

[0100] Step 6: Calculate the view fusion weights. Input the features extracted from each view into the quality perception module based on view-specific information, calculate the fusion weights of each view using formula (5), and calculate the classification loss of the module according to formula (6).

[0101] Step 7: Multi-view feature weighted fusion. The features extracted from each view in Step 4 are input into the dynamic weighted fusion module, and the fusion representation Z of the input multi-view data is obtained through Equation (7). Since the real labels are required when calculating the fusion weights, and the real labels are not used as input to the classification model, this step cannot be calculated directly in the network. Therefore, this invention uses an average weighting strategy in the first batch of the first iteration, and then corrects the fusion weights of the views during gradient update optimization. After the last gradient update, the fusion weights of each view will be fixed and used in the testing phase.

[0102] Step 8: Multi-label classification. Input the fused representation of the samples obtained in the previous step into the multi-label classification module, and obtain the label prediction probability value P of the input sample through the missing multi-label classification module. Then calculate the classification loss of the fused features according to equation (9).

[0103] Step 9: Network Loss Calculation. (Using formula...) Calculate the total loss of the i-th batch in the model.

[0104] Step 10: Gradient Feedback Optimization. Based on the network loss obtained in the previous step... Perform parameter optimization using the gradient descent algorithm. If i < m, let i = i + 1, then update the network parameters and continue to step four; if i = m, then proceed to step eleven.

[0105] Step 11: Convergence Check. When the number of iterations t > 20 and If the network model training stops and its parameters are output, then t = t + 1 and the process jumps to the second step. and Let represent the total loss at step t and step t-1, respectively.

[0106] like Figure 3 As shown, the test sample label prediction process includes:

[0107] The complete or incomplete multi-view test data to be tested is represented as follows: An index matrix indicating whether a view is missing. The specific testing process is described below:

[0108] Step 1: Data preprocessing. Fill in missing view features with 0 values.

[0109] Step 2: Deep Feature Extraction. This involves extracting deep features from the preprocessed data. Input a specific view feature extraction network to obtain high-order semantic features extracted from each view. The specific view feature extraction network parameters in this step are the trained model parameters.

[0110] Step 3: Weighted Feature Fusion. This involves combining the discriminative features from the previous step with the view missing indicator matrix. Input weighted feature fusion module, according to formula The fusion representation Z of the test data is obtained. Where a v These are the view fusion weights obtained during training.

[0111] Step 4: Multi-label classification. The fused feature Z is input into the multi-label classification module to obtain the label prediction result P of the incomplete multi-view data. In this step, the classifier parameters are the model parameters trained as described in Section 3.2.1.

[0112] Step 5: Output the recognition results. Based on the multi-label classification label definition rules. The final 0-1 binary label matrix of the input data is obtained.

[0113] The technical performance of this invention is compared with that of the existing technology DD-ImvMLC-net. As shown in Table 1, the average precision (AP) of this invention and DD-ImvMLC-net on five datasets with a view missing rate of 50%, a label missing rate of 50%, and a training sample rate of 70%.

[0114] Table 1

[0115]

[0116]

[0117] A specific application embodiment of the present invention is as follows:

[0118] In intelligent surveillance systems, multiple cameras monitor the same area from different angles to ensure security. However, due to various reasons (such as camera malfunction, obstruction, and network problems), not all cameras can always provide complete video data, leading to data incompleteness. Simultaneously, surveillance systems need to identify multiple events or behaviors, i.e., multi-label classification problems, such as intrusion, fighting, fire, and falls. However, in practical applications, due to negligence or fatigue of labelers, lack of experience, unclear events, and data loss, some video labels may be missing. In other words, incomplete multi-view and incomplete multi-label training data are prevalent in the field of intelligent surveillance, and traditional multi-view and multi-label learning models cannot be trained on this type of data. Therefore, they are unsuitable for handling event classification tasks with incomplete multi-view detection data in intelligent surveillance, meaning existing multi-view and multi-label classification models have limited adaptability in similar intelligent surveillance applications. The model designed in this invention can better handle incomplete multi-view and incomplete multi-label model learning and multi-label classification tasks with incomplete or complete multi-views in intelligent surveillance, as described above.

[0119] This invention discloses an incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation. By dynamically evaluating the importance of each view and assigning appropriate weights to the views, it provides reliable guidance for the fusion of multiple views, thereby obtaining credible fusion features. It fully explores the consistency information between multi-view data; introduces contrastive learning to enhance the consistency of features of the same sample across different views; and proposes a consistency loss to increase the structural consistency between samples in the fusion feature space and the original multi-view space. Based on the partial or complete multi-view features of each input sample, it accurately predicts the complete label of each input sample.

[0120] The above are merely preferred embodiments of this application, but the scope of protection of this application is not limited thereto. Any variations or substitutions that can be easily conceived by those skilled in the art within the scope of the technology disclosed in this application should be included within the scope of protection of this application. Therefore, the scope of protection of this application should be determined by the scope of the claims.

Claims

1. A method for learning and classifying incomplete multi-view, incomplete multi-label data based on view information quality estimation, characterized in that, include: Acquire several incomplete multi-view, incomplete multi-label data, wherein the incomplete multi-view, incomplete multi-label data is video data; Construct an incomplete multi-view, incomplete multi-label learning model, input the incomplete multi-view, incomplete multi-label data into the incomplete multi-view, incomplete multi-label learning model for training, and obtain a trained incomplete multi-view, incomplete multi-label learning model. Acquire complete or incomplete multi-view data to be tested, and input the complete or incomplete multi-view data to be tested into the trained incomplete multi-view incomplete multi-label learning model to complete the multi-label classification of the input data to be tested. The incomplete multi-view and incomplete multi-label learning model includes: a view feature extraction module, a quality perception module based on view information, a dynamic weighted fusion module, and a missing multi-label classification module; The view feature extraction module is used to extract high-order semantic features from the incomplete multi-view and incomplete multi-label data; The quality perception module based on view information is used to capture the quality of the high-order semantic features of the incomplete multi-view and incomplete multi-label data, assign view fusion weights, and obtain the optimal fusion weights. The dynamic weighted fusion module is used to dynamically weight and fuse the allocated optimal fusion weights with the prior missing information of the view to obtain the fusion result. The missing multi-label classification module is used to classify based on the fusion result and obtain the classification result; The process of capturing the quality of high-order semantic features of the incomplete multi-view, incomplete multi-label data, assigning view fusion weights, and obtaining the optimal fusion weights includes: Predict the class probability P of the sample in the v-th view based on the extracted view features. v The classifier consists of a fully connected layer and a sigmoid activation function, which will convert P... v Convert to label matrix The classification accuracy θ for each view is calculated using the predicted label matrix and the true label matrix. v and the index matrix of valid tags Where G and W are the indicator matrix for missing labels and the indicator matrix for missing views, respectively; This indicates that the j-th label of the i-th sample in the v-th view is valid; By introducing an indicator matrix with effective labels, the classification accuracy θ of view v is calculated. v : Where ⊙ represents the dot product operation. Indicates the XOR operation; Based on the classification accuracy as an indicator of the quality of view feature information, the view fusion weight 'a' is learned. v :

2. The incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation as described in claim 1, characterized in that, Extracting high-order semantic features from the incomplete multi-view and incomplete multi-label data includes: X′ v =f v (X v ) Where, X′ v and f v (.) represent the discriminative features extracted from the v-th view and the encoder of the v-th view, respectively; X v These are the original features of the data in the v-th view.

3. The incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation as described in claim 1, characterized in that, The optimal fusion weights are combined with prior missing information from the view for dynamic weighted fusion, resulting in the following fusion results: Among them, Z i Let be the fused representation of the i-th sample; Let a be the new feature vector of the i-th sample extracted from the v-th view; v Let the fusion weight be the value of the v-th view. In the classification process based on the fusion result, binary cross-entropy is used to train the objective function, and the weighted loss is obtained as follows: in, It is a classification loss based on fused features, where n represents the number of samples, L represents the number of labels, and P i，j Y is the final predicted probability of the i-th sample with respect to the j-th label. i，j Let G represent the j-th true label of the i-th sample. i，j This indicates whether the true label of the j-th sample given in the training dataset is known.

4. The incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation as described in claim 3, characterized in that, The view feature extraction module also introduces contrastive learning to capture consistency information among multiple views, with the contrastive loss being: in, It calculates two vectors. and The cosine similarity function between them; S neg W is the sum of the cosine similarities of the negative sample pairs; W is the view missing indicator matrix; W i，v =1 indicates that the i-th sample is not missing in the v-th view; otherwise, the sample is missing. This indicates that the v-th view of the i-th sample has been processed by encoder f. v Output features following (.).

5. The incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation as described in claim 4, characterized in that, The quality perception module based on view information also introduces a view-related classification loss as follows: in, It is the classification loss of the views, where V represents the number of views. It is the predicted probability of the j-th class for the i-th sample in the v-th view. It is the indicator matrix of valid labels in the v-th view.

6. The incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation as described in claim 5, characterized in that, The dynamic weighted fusion module also designs a consistency loss between the instance structure in the fused feature space and the instance structure in the original space as follows: Among them, diag(W) ：，v () is a diagonal matrix to avoid the negative impact of missing samples on the consistency structure; The structural consistency loss is represented by V; the number of views is represented by n; and the number of samples is represented by n. This indicates the difference in sample structure between the fused features and the original features.

7. The incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation as described in claim 6, characterized in that, The classification loss of the missing multi-label classification module is:

8. The incomplete multi-view, incomplete multi-label learning and classification method based on view information quality estimation as described in claim 7, characterized in that, The training loss function of the incomplete multi-view, incomplete multi-label learning model is:

Images