Multi-view data classification method and system based on dynamic self-attention capsule network
Through the dynamic self-attention capsule network method, dynamically selecting and weighting the features in multi-view data, the problem of insufficient information integration in the prior art is solved, and the accuracy and robustness of multi-view data classification is significantly improved.
Patent Information
- Application Number
- CN202411375132.2
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-09-30
- Publication Date
- 2024-11-01
- Estimated Expiration
- 2044-09-30
Smart Images

Figure CN118887482A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of multi-view data classification, in particular to effectively integrating and utilizing information from multiple different perspectives, and to a multi-view data classification method and system based on a dynamic self-attention capsule network. Background Art
[0002] With the development of data collection technology, the diversity and complexity of data are increasing. Data of different views come from different data sources or multiple forms of expression. These views often contain rich and complementary information. How to integrate and utilize these multi-view data is an important research topic. Multi-view learning is an important technology for processing data with multiple different views. The application of multi-view not only improves the effect of data analysis, but also provides new perspectives and methods for understanding complex systems.
[0003] The focus of multi-view data classification is how to effectively integrate and utilize information from multiple views to improve the accuracy and robustness of classification. Each view represents a different feature or aspect of the data, and there may be a complementary relationship between these views. Multi-view clustering considers the differences and complementarities between different views and simultaneously analyzes data from multiple views to discover the potential structure and patterns in the data, thereby effectively processing multi-view data. Based on the research on multi-view clustering, multi-view data classification technology has gradually developed. Unlike clustering, the goal of multi-view data classification is to use existing label information to accurately classify new data by training models.
[0004] Multilayer Perceptron (MLP) is a nonlinear feature extractor that maps raw view data into a new feature space. By training MLP independently on each view or jointly training MLP on multiple views, more abstract and representative features can be obtained. Capsule Network is a new architecture based on neural network. It realizes feature extraction through dynamic routing mechanism and can effectively capture the spatial hierarchical relationship of input data. The early capsule network had high computational complexity, which limited its application on large-scale datasets. In recent years, various improved methods, such as Efficient Capsule Network, have been proposed to solve these problems.
[0005] Existing methods mainly rely on simple feature fusion strategies. Most of the current methods do not consider the dynamic selection of information between different feature information and views and the dynamic weighting of useful information, resulting in some useful information being ignored or redundant information being introduced, affecting the final performance of the model.
[0006] To this end, the present invention proposes a new multi-view data classification method based on a dynamic self-attention capsule network for multi-view data integration. Summary of the invention
[0007] In view of the shortcomings of the prior art, the present invention provides a multi-view data classification method and system based on a dynamic self-attention capsule network. First, in the process of multi-view data integration, the feature information of different views is dynamically selected to make full use of the useful feature information of each view and remove the influence of redundant and noisy data. Secondly, for the information of different perspectives provided by the multi-view data, the multi-view information is dynamically weighted according to the task requirements. Therefore, the present invention aims to improve the classification accuracy and model robustness of multi-view learning by dynamically weighting and fusing different information in multi-view data.
[0008] The method of the present invention mainly dynamically selects the features with the largest amount of information, eliminates the influence of redundant and noisy feature data, and retains the most useful features. At the same time, different views are weighted to maximize the use of the specific information of each view, thereby improving the quality and interpretation ability of the final data fusion. Finally, the fused data is accurately classified.
[0009] In order to solve the above technical problems, the technical solution adopted by the present invention is: The multi-view data classification method based on dynamic self-attention capsule network includes the following steps: S1. Multi-view feature self-attention learning: For each view, calculate the feature self-attention weight and view self-attention weight, perform self-attention weighting, and obtain the data features after feature weighting and view weighting; S2. Dynamic learning of view self-attention weights: The weighted final data of each view is classified, and the view self-attention weight is dynamically learned through the predicted probability value corresponding to the sample label category and the single weight value of the view, and the difference between the sample predicted probability and the label, and finally the feature matrix is obtained; S3, data classification: The concatenated feature matrix M is input into the capsule network composed of a fully connected layer, a main capsule layer, and a phenotype capsule layer to complete the task of accurately classifying the sample data.
[0010] Further, step S1 includes the following steps: S11, multi-view data input: the data samples and their features corresponding to each view are taken as input; S12. Learn feature self-attention weights through the encoder: assign a self-attention weight to each feature of each view in all samples, multiply the obtained self-attention weight by the initial feature value element by element, perform attention weighting on the feature, and obtain feature data after feature self-attention weighting; S13, firstly, use the encoder to reduce the dimension of each view to obtain a low-dimensional representation of the view feature data, and realize the dimension alignment of the view data; then, for the data after dimension reduction, assign a single weight value based on the overall feature of the view to each view, and directly multiply the weight value with the feature matrix to realize the view self-attention weighting of the data after feature self-attention weighting;
[0011] Furthermore, S12 is as follows: S1201: inputting the data corresponding to each view into the feature self-attention encoder implemented by the multi-layer perceptron MLP separately; the dimension of the weight matrix in the MLP is the same as the dimension of the sample feature of each view, indicating the self-attention weight of the feature under each sample; S1202: performing element-level multiplication operation on the obtained weight matrix and the initial feature matrix to weight the original eigenvalues; S1203: Perform L1 norm constraint on the obtained weight matrix.
[0012] Furthermore, S13 is as follows: S1301: For each view, input the feature-weighted data into an independent view encoder for dimensionality reduction processing, that is, reduce the feature dimension of each view, extract a low-dimensional representation of the feature, and achieve dimension alignment of view feature data; S1302: inputting the obtained low-dimensional feature representation into an MLP for view weight calculation to generate a single weight value of the view, indicating the different importance of each view in the overall feature fusion process; S1303: Multiply the weight value obtained for each view by the corresponding low-dimensional feature representation to perform overall weighted processing.
[0013] Further, step S2 is specifically as follows: S21. For each view, the data features after feature and view weighting are classified by using an auxiliary classifier implemented by MLP to obtain an initial classification prediction result of each sample in a single view; S22. According to the initial classification prediction results of the view, dynamically learn and adjust the view self-attention weight: compare the predicted probability value corresponding to the sample label category of each view with the single weight value of the view, and the sample prediction probability with the sample label, and calculate the loss function; use the loss value to perform back-propagation training on the model to optimize the parameters of the model; S23, multi-view data splicing and integration: The splicing operation is used to splice the feature representations from different views together according to the samples to obtain the final feature matrix M.
[0014] Furthermore, step S3 is specifically as follows: S31, data is sequentially input into the fully connected layer, the main capsule layer, and the phenotype capsule layer; S32, Dynamic routing dynamically adjusts the parameters in the network structure: The main capsule layer and the phenotype capsule layer dynamically adjust the information transmission path between the two layers based on the similarity and correlation between capsules through the dynamic routing mechanism, and update the weight coefficient and coupling coefficient in the routing process; S33. Obtain the final category prediction output probability and complete the classification task.
[0015] Furthermore, step S31 is specifically as follows: S3101: The processed multi-view data is first passed through a fully connected layer with bias and activation function using Relu() to map the concatenated high-dimensional data dimension to a low dimension; S3102: In the main capsule layer, convolution operation is performed on the features using the convolution kernel to convert the features into capsule vector representation. The length of the capsule vector in the main capsule layer is L, and the number of main capsules is N. S3103: Each capsule in the main capsule layer generates a feature vector for the input data, and the length is L, the vector length of the main capsule layer; S3104: The feature vector in each primary capsule and the weight matrix W in the phenotype capsule ij Perform matrix multiplication to generate the prediction vector of each phenotypic capsule j for the main capsule i .
[0016] Furthermore, in step S32, the feature information between the main capsule layer and the phenotype capsule layer is dynamically routed n times. The detailed steps of the dynamic routing process are as follows: S3201: For each pair of primary capsule and phenotype capsule, calculate its routing weight coefficient , Initial value setting: When the data enters the capsule for calculation for the first time, it does not go through dynamic routing calculation, and the initial weight coefficient Set to 0; S3202: Calculate the coupling coefficient of the phenotypic capsule : Use the softmax function to calculate the routing coefficients of all primary capsules to each phenotype capsule Normalize them so that their sum is 1; The specific calculation is: ; because The initial value is set to 0, so Initially, it is calculated as a series of uniformly distributed probability values; in the dynamic routing process, the coupling coefficient is calculated in each round of routing according to Update the value of S3203: For each phenotype capsule j, calculate its activation value ; S3204: Get the output vector of each phenotype capsule , use the Squash function to ensure that the length of the output vector is between 0 and 1: ; S3205: Update of routing parameters: for each final phenotype capsule output, the weight coefficient is iteratively updated through the prediction vector between capsule pairs, and the number of updates is the same as the number of routings; The update strategy can be summarized as: ; S3206: After the data is calculated for the first time to obtain the output vector, the newly updated weight coefficient is obtained After the value of , dynamic routing starts; Repeat S3202 to S3205 until a predetermined number of iterations is reached; In S33, the final phenotypic capsule output vector is converted into the final sample category prediction probability output through the softmax function. The number of capsules in the last phenotypic layer is the same as the sample category in the dataset, and the capsule vector lengths in the phenotypic capsule layer are both .
[0017] The present invention also provides a multi-view data classification system based on a dynamic self-attention capsule network, which is used to implement the multi-view data classification method based on a dynamic self-attention capsule network as described above, including a dynamic self-attention capsule network, the dynamic self-attention capsule network including a multi-view feature self-attention learning module, a view self-attention dynamic learning module, and a capsule network module. The multi-view feature self-attention learning module calculates the feature self-attention weight and the view self-attention weight for each view, performs self-attention weighting, and obtains feature-weighted and view-weighted data features; The view self-attention dynamic learning module is used to classify the weighted data of each view, dynamically learn the view self-attention weight through the predicted probability corresponding to the sample label category and the single weight value of the view, and the difference between the sample predicted probability and the label, and finally obtain the feature matrix M; The capsule network module is used to input the concatenated feature matrix M into the capsule network composed of a fully connected layer, a main capsule layer, and a phenotype capsule layer to complete the task of accurately classifying the sample data.
[0018] Compared with the prior art, the advantages of the present invention are: (1) For data from different views, by dynamically selecting and weighting the feature information of each view, the useful information of each view can be fully utilized, while redundant and noisy data can be removed, thereby improving the quality of multi-view data fusion.
[0019] (2) For multi-classification tasks, this method can significantly improve the classification accuracy and enhance the robustness of classification.
[0020] (3) Applicable to various scenarios of multi-view data classification, including cases with high data dimensionality and feature diversity. By comprehensively considering the information and characteristics of different views, the model can perform well in complex data environments.
[0021] (4) The dynamic self-attention mechanism optimizes the information fusion between views, which can effectively capture the complementarity and correlation between views, and improve the adaptability and performance stability of the model when processing heterogeneous data. BRIEF DESCRIPTION OF THE DRAWINGS
[0022] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other accompanying drawings can be obtained based on these accompanying drawings without paying creative work.
[0023] Figure 1 is a flow chart of the method of the present invention; Figure 2 is a system structure diagram of the present invention; Figure 3 This is a specific implementation diagram of the dynamic routing mechanism of the present invention. DETAILED DESCRIPTION
[0024] The present invention will be further described below in conjunction with the accompanying drawings and specific embodiments.
[0025] like Figure 1 As shown, this embodiment provides a multi-view data classification method based on a dynamic self-attention capsule network, which mainly dynamically selects the features with the largest amount of information, eliminates the influence of redundant and noisy feature data, and retains the most useful features. At the same time, different views are weighted to maximize the use of specific information of each view, and improve the quality and interpretation ability of the final data fusion. Finally, the fused data is accurately classified. In order to verify the effect of the data classification task proposed in the present invention under the multi-view background, the breast cancer patient data set provided by the TCGA (The cancer genome atlas) database is used to perform multi-classification tasks. Specifically, breast cancer patients are divided into 5 molecular subtypes: normal-like, basal-like, human epidermal growth factor receptor 2 (HER2) enriched, luminal a and luminal b. The three view data represent the mRNA expression data, DNA methylation data, and miRNA expression data of the sample respectively.
[0026] The specific steps include: S1. Multi-view feature self-attention learning: For each view, the feature self-attention weight and the view self-attention weight are calculated, and self-attention weighting is performed to obtain the feature-weighted and view-weighted data features.
[0027] S11. Multi-view data input: The data samples and features corresponding to each view are taken as input.
[0028] Select the multi-view feature data of the breast cancer dataset downloaded from the TCGA database; input the data samples and features corresponding to each view. The feature dimensions of the three view inputs are 1000, 1000, and 503 respectively. The sample dimension used for model training is 612 samples, and the number of test samples is 263.
[0029] S12. Learn feature self-attention weights through the encoder: Assign a self-attention weight to each feature of each view in all samples, multiply the obtained self-attention weight by the initial feature value element by element, perform attention weighting on the feature, and obtain feature data after feature self-attention weighting.
[0030] The details are as follows: S1201: inputting the data corresponding to each view into the feature self-attention encoder implemented by the multi-layer perceptron MLP separately; the dimension of the weight matrix in the MLP is the same as the dimension of the sample feature of each view, indicating the self-attention weight of the feature under each sample; S1202: performing an element-wise multiplication operation on the obtained weight matrix and the initial feature matrix, represented by the symbol e, to weight the original feature values; S1203: Perform L1 norm constraint on the obtained weight matrix to improve sparsity.
[0031] S13. First, use the encoder to reduce the dimension of each view to obtain a low-dimensional representation of the view feature data and align the view data dimensions. Then, for the reduced-dimensional data, assign a single weight value based on the overall features of the view to each view, and directly multiply the weight value with the feature matrix, which is represented by the symbol , perform view self-attention weighting on the data after feature self-attention weighting.
[0032] The details are as follows: S1301: For each view, the feature-weighted data is input into an independent view encoder for dimensionality reduction processing, and the hidden layer feature dimension is selected to be 500, that is, the feature dimension of each view is reduced to 500, and a low-dimensional representation of the feature is extracted to achieve dimension alignment of view feature data; S1302: Input the obtained low-dimensional feature representation into an MLP for view weight calculation, where the linear layer changes from 500 to 1, generating a single weight value for the view, indicating the different importance of each view in the overall feature fusion process; S1303: Multiply the weight value obtained for each view by the corresponding low-dimensional feature representation to perform overall weighted processing.
[0033] S2. Dynamic learning of view self-attention weights: The weighted final data of each view is classified, and the view self-attention weight is dynamically learned through the predicted probability value corresponding to the sample label category and the single weight value of the view, and the difference between the sample predicted probability and the label, and finally the feature matrix is obtained.
[0034] Step S2 is specifically as follows: S21. For each view, the data features after feature and view weighting are classified by using an auxiliary classifier implemented by MLP to obtain the initial classification prediction result of each sample in a single view; the number of neurons in the last layer of the classifier is the same as the sample category of the dataset, that is, the number of neurons in the last layer of the auxiliary MLP classifier of the breast cancer dataset is 5.
[0035] S22. According to the initial classification prediction results of the view, dynamically learn and adjust the view self-attention weight: compare the predicted probability value corresponding to the sample label category of each view with the single weight value of the view, and the sample prediction probability with the sample label, and calculate the loss function; use the loss value to perform back-propagation training on the model to optimize the parameters of the model; S23, multi-view data splicing and integration: The splicing operation is used to splice the feature representations from different views together according to the samples to obtain the final feature matrix M.
[0036] S3, data classification: The concatenated feature matrix M is input into the capsule network composed of a fully connected layer, a main capsule layer, and a phenotype capsule layer to complete the task of accurately classifying the sample data.
[0037] Step S3 is as follows: S31. The data is sequentially input into the fully connected layer, the main capsule layer, and the phenotype capsule layer.
[0038] Step S31 is specifically as follows: S3101: The processed multi-view data is first passed through a fully connected layer with a bias and activation function using Relu() to map the concatenated high-dimensional data dimension to a low dimension. In this embodiment, the fused data dimension 1500 is mapped to 150 dimensions.
[0039] S3102: In the main capsule layer, a 5×1 convolution kernel is used to perform a convolution operation on the features to convert the features into capsule vector representations. The length of the capsule vector in the main capsule layer is L, and the number of main capsules is N. In this embodiment, the length of the capsule vector is 4, and the number of main capsules is 16. For each capsule, its feature representation is (1, 73, 4).
[0040] S3103: Each capsule in the main capsule layer generates a feature vector for the input data, and the length is the vector length L of the main capsule layer; in this embodiment, the vector length is 4. The common representation of all 16 capsules in the main capsule layer can be expressed as: (73×16, 4), such as Figure 3 The main capsule layer structure represented in .
[0041] S3104: The feature vector in each primary capsule and the weight matrix W in the phenotype capsule ij Perform matrix multiplication to generate the prediction vector of each phenotypic capsule j for the main capsule i .
[0042] S32. Dynamic routing dynamically adjusts parameters in the network structure: The main capsule layer and the phenotype capsule layer use a dynamic routing mechanism to dynamically adjust the path of information transmission between the two layers based on the similarity and correlation between capsules, and update the weight coefficient and coupling coefficient in the routing process, thereby effectively capturing and transmitting useful feature information.
[0043] Step S32: The feature information between the main capsule layer and the phenotype capsule layer is dynamically routed n times (for example, 3 times), and the information transmission path between the two layers is dynamically adjusted based on the similarity and correlation between the capsules, and the relevant parameters in the routing are updated. Figure 3 As shown, the detailed steps of the dynamic routing process are as follows: S3201: For each pair of primary capsule and phenotype capsule, calculate its routing weight coefficient , Initial value setting: When the data enters the capsule for calculation for the first time, it does not go through dynamic routing calculation, and the initial weight coefficient Set to 0; S3202: Calculate the coupling coefficient of the phenotypic capsule : Use the softmax function to calculate the routing coefficients of all primary capsules to each phenotype capsule Normalize them so that their sum is 1; The specific calculation is: ; because The initial value is set to 0, so Initially, it is calculated as a series of uniformly distributed probability values; in the dynamic routing process, the coupling coefficient is calculated in each round of routing according to Update the value of S3203: For each phenotype capsule j, calculate its activation value ; S3204: Get the output vector of each phenotype capsule , use the Squash function to ensure that the length of the output vector is between 0 and 1: ; S3205: Update of routing parameters: for each final phenotype capsule output, the weight coefficient is iteratively updated through the prediction vector between capsule pairs, and the number of updates is the same as the number of routings; The update strategy can be summarized as: ; S3206: After the data is calculated for the first time to obtain the output vector, the newly updated weight coefficient is obtained After the value of , dynamic routing starts; Repeat S3202 to S3205 until the predetermined number of iterations (3 times) is reached. Through multiple iterations, the model gradually optimizes the coupling coefficient and makes the information transmission path more accurate.
[0044] in Figure 3 In the example, one main capsule layer and two phenotypic capsule layers are used. Figure 3 u1, u2, ..., u, etc. in the above formula represent all capsules in the main capsule layer. 11 , W 12 Represents the weight matrix multiplied by the feature vector of the first primary capsule, W i1 , W i2 Represents the weight matrix multiplied by the feature vector of the i-th primary capsule to generate the prediction vector, and Represents the corresponding prediction vector generated, r 11 , r i1 , r 12 , r i2 represents the coupling coefficient, I1 and I2 represent the activation values calculated by the two phenotypic capsules, and O1 and O2 represent the output vectors of the two phenotypic capsules.
[0045] S33. Obtain the final category prediction output probability and complete the classification task.
[0046] The final phenotypic capsule output vector is converted into the final sample category prediction probability output through the softmax function. The number of capsules in the last phenotypic layer is the same as the sample category in the dataset, and the capsule vector lengths in the phenotypic capsule layer are both That is, for the breast cancer dataset, the number of phenotypic capsules is 5. Finally, the final sample category prediction probability is calculated based on the capsule vector value calculated by the capsule network. Among them, the capsule vector length of the phenotypic capsule is 16.
[0047] like Figure 2 As shown, this embodiment also provides a multi-view data classification system based on a dynamic self-attention capsule network, which is used to implement a multi-view data classification method based on a dynamic self-attention capsule network, including a dynamic self-attention capsule network, which includes a multi-view feature self-attention learning module, a view self-attention dynamic learning module, and a capsule network module.
[0048] The multi-view feature self-attention learning module calculates the feature self-attention weight and the view self-attention weight for each view, performs self-attention weighting, and obtains data features after feature weighting and view weighting.
[0049] The view self-attention dynamic learning module is used to classify the weighted data of each view, dynamically learn the view self-attention weight through the predicted probability corresponding to the sample label category and the single weight value of the view, and the difference between the sample predicted probability and the label, and finally obtain the feature matrix M.
[0050] The capsule network module is used to input the concatenated feature matrix M into the capsule network composed of a fully connected layer, a main capsule layer, and a phenotype capsule layer to complete the task of accurately classifying the sample data.
[0051] The functional implementation and data processing process of each module can be found in the detailed steps of the multi-view data classification method based on dynamic self-attention capsule network recorded above, which will not be repeated here.
[0052] It should be noted that the present invention uses the evaluation indicators of classification tasks, accuracy (Acc), F1 score (F1-score), and area under the ROC curve (AUC) to evaluate the model performance, and conducts comparative tests with other multi-view data classification methods to evaluate the model performance and verify that the model can more effectively complete the multi-view classification task and achieve a better prediction effect than the comparison model.
[0053] The comparison models are described in detail as follows: K-nearest neighbor algorithm (KNN), support vector machine (SVM), fully connected neural network (NN), GMU: The Gated Multimodal Unit (GMU) constructs intermediate multi-view representations through data combination to promote information fusion; CF: Final multimodal representation cascade (CF), which integrates multimodal data by concatenating subsequent feature representations; TMC: Trusted Multi-View Classification (TMC), considering the different confidence levels of each view, performs decision fusion based on the confidence levels; MOGONET: Multi-omics graph convolutional network, builds a graph for each view, uses GCN to obtain information, and studies the correlation between multimodal data through correlation discovery network; Dynamic: Dynamic fusion of credible multi-view classification, modeling feature and modality information to distinguish the specificity of different features and views; MLCLNet: Multi-level Confidence Learning for Trusted Multi-view Classification, which uses feature and label confidence learning methods and uses GCN to obtain features for cross-modal label fusion.
[0054] The evaluation indicators are described in detail as follows: ACC (Accuracy): Definition: Accuracy refers to the ratio of the number of samples correctly predicted by the model to the total number of samples. That is: Accuracy (ACC) = Number of samples correctly predicted Total number of samples\Total number of samples. ACC is the most common classification performance indicator, which is used to measure the accuracy of the overall classification of the model.
[0055] F1_macro (macro F1 score): Definition: The macro F1 score is for multi-class classification problems. It is calculated by calculating the F1 score of each class (i.e. the harmonic mean of precision and recall) and then averaging the F1 scores of each class. It is a simple average of the F1 scores of all classes, regardless of the number of samples in the class. F1_macro focuses more on the performance of a few classes and is suitable for situations where the distribution between classes is unbalanced because it gives equal weight to each class.
[0056] F1_weighted (weighted F1 score): Definition: Weighted F1 score is similar to macro F1 score, but it performs a weighted average of the F1 scores of each class according to the proportion of the class in the dataset. The weight is determined by the proportion of samples in each class in the dataset. F1_weighted is particularly important on class-imbalanced datasets because it can reflect the actual contribution of different classes to the overall model performance. Compared with simple average, it can more accurately measure the performance of the model in the case of imbalanced class distribution and avoid the excessive influence of minority classes on the overall evaluation.
[0057] AUC (Area Under the Curve): Definition: AUC is an important indicator for measuring the performance of classification models, especially widely used in binary classification tasks to evaluate the model's ability to distinguish between positive and negative samples in binary classification tasks. The ROC curve depicts the relationship between the false positive rate (FPR) and the true positive rate (TPR) of the model under different thresholds. It quantifies the overall performance of the model's ability to distinguish between positive and negative samples.
[0058] This example uses a breast cancer patient dataset to evaluate the performance of the model in multi-view multi-classification tasks. As shown in Table 1, the model performs well in multiple indicators such as ACC (accuracy), F1_macro (macro F1 score) and F1_weighted (weighted F1 score). First, in terms of the ACC indicator, the model accuracy value is 0.875, which is better than most of the methods in the comparison method, indicating that the model can accurately classify most samples and demonstrates its strong overall classification performance. In addition, the F1_macro score is significantly higher than all other models, indicating that the model has significant advantages in processing class imbalanced data. In terms of the F1_weighted score, the model optimizes the overall classification task by comprehensively considering the imbalance of the number of samples in each category. These excellent performances fully demonstrate that the model has strong adaptability and generalization capabilities in multi-view data multi-classification tasks.
[0059] Table 1 Comparison of the effects of different methods on the multi-classification dataset of breast cancer patients Of course, the above description is not a limitation of the present invention, and the present invention is not limited to the above examples. Any changes, modifications, additions or substitutions made by ordinary technicians in this technical field within the essential scope of the present invention should fall within the protection scope of the present invention.
Claims
1. A multi-view data classification method based on a dynamic self-attention capsule network, characterized in that: The following steps are involved: S1. Multi-view feature self-attention learning: For each view, calculate the feature self-attention weight and view self-attention weight, perform self-attention weighting, and obtain the data features after feature weighting and view weighting; S2. Dynamic learning of view self-attention weights: The weighted final data of each view is classified, and the view self-attention weight is dynamically learned through the predicted probability value corresponding to the sample label category and the single weight value of the view, and the difference between the sample predicted probability and the label, and finally the feature matrix is obtained; S3, data classification: The concatenated feature matrix M is input into the capsule network composed of a fully connected layer, a main capsule layer, and a phenotype capsule layer to complete the task of accurately classifying the sample data.
2. The multi-view data classification method based on dynamic self-attention capsule network according to claim 1 is characterized in that: Specifically as follows: Step S1 includes the following steps: S11, multi-view data input: the data samples and their features corresponding to each view are taken as input; S12. Learn feature self-attention weights through the encoder: assign a self-attention weight to each feature of each view in all samples, multiply the obtained self-attention weight by the initial feature value element by element, perform attention weighting on the feature, and obtain feature data after feature self-attention weighting; S13. First, use the encoder to reduce the dimension of each view to obtain a low-dimensional representation of the view feature data and realize the dimension alignment of the view data; then, for the data after dimensionality reduction, assign a single weight value based on the overall characteristics of the view to each view, and directly multiply the weight value with the feature matrix to realize view self-attention weighting of the data after feature self-attention weighting.
3. The multi-view data classification method based on dynamic self-attention capsule network according to claim 2 is characterized in that: S12 is as follows: S1201: inputting the data corresponding to each view into the feature self-attention encoder implemented by the multi-layer perceptron MLP separately; the dimension of the weight matrix in the MLP is the same as the dimension of the sample feature of each view, indicating the self-attention weight of the feature under each sample; S1202: performing element-level multiplication operation on the obtained weight matrix and the initial feature matrix to weight the original eigenvalues; S1203: Perform L1 norm constraint on the obtained weight matrix.
4. The multi-view data classification method based on dynamic self-attention capsule network according to claim 2 is characterized in that: S13 is as follows: S1301: For each view, input the feature-weighted data into an independent view encoder for dimensionality reduction processing, that is, reduce the feature dimension of each view, extract a low-dimensional representation of the feature, and achieve dimension alignment of view feature data; S1302: inputting the obtained low-dimensional feature representation into an MLP for view weight calculation to generate a single weight value of the view, indicating the different importance of each view in the overall feature fusion process; S1303: Multiply the weight value obtained for each view by the corresponding low-dimensional feature representation to perform overall weighted processing.
5. The multi-view data classification method based on dynamic self-attention capsule network according to claim 1, characterized in that: Step S2 is specifically as follows: S21. For each view, the data features after feature and view weighting are classified by using an auxiliary classifier implemented by MLP to obtain an initial classification prediction result of each sample in a single view; S22. According to the initial classification prediction results of the view, dynamically learn and adjust the view self-attention weight: compare the predicted probability value corresponding to the sample label category of each view with the single weight value of the view, and the sample prediction probability with the sample label, and calculate the loss function; use the loss value to perform back-propagation training on the model to optimize the parameters of the model; S23. Multi-view data splicing and integration: The splicing operation is used to splice the feature representations from different views together according to the samples to obtain the final feature matrix M.
6. The multi-view data classification method based on dynamic self-attention capsule network according to claim 1, characterized in that: Step S3 is as follows: S31, data is sequentially input into the fully connected layer, the main capsule layer, and the phenotype capsule layer; S32, Dynamic routing dynamically adjusts the parameters in the network structure: The main capsule layer and the phenotype capsule layer use a dynamic routing mechanism to dynamically adjust the path of information transmission between the two layers based on the similarity and correlation between capsules, and update the weight coefficient and coupling coefficient in the routing process; S33. Obtain the final category prediction output probability and complete the classification task.
7. The multi-view data classification method based on dynamic self-attention capsule network according to claim 6, characterized in that: Step S31 is specifically as follows: S3101: The processed multi-view data is first passed through a fully connected layer with bias and activation function using Relu() to map the concatenated high-dimensional data dimension to a low dimension; S3102: In the main capsule layer, convolution operation is performed on the features using the convolution kernel to convert the features into capsule vector representation. The length of the capsule vector in the main capsule layer is L, and the number of main capsules is N. S3103: Each capsule in the main capsule layer generates a feature vector for the input data, and the length is L, the vector length of the main capsule layer; S3104: The feature vector in each primary capsule and the weight matrix W in the phenotype capsule ij Perform matrix multiplication to generate the prediction vector of each phenotypic capsule j for the main capsule i .
8. The multi-view data classification method based on dynamic self-attention capsule network according to claim 6, characterized in that: Step S32: The feature information between the main capsule layer and the phenotype capsule layer is dynamically routed n times. The detailed steps of the dynamic routing process are as follows: S3201: For each pair of primary capsule and phenotype capsule, calculate its routing weight coefficient , Initial value setting: When the data enters the capsule for calculation for the first time, it does not go through dynamic routing calculation, and the initial weight coefficient Set to 0; S3202: Calculate the coupling coefficient of the phenotypic capsule : Use the softmax function to calculate the routing coefficients of all primary capsules to each phenotype capsule Normalize them so that their sum is 1; The specific calculation is: ; because The initial value is set to 0, so Initially, it is calculated as a series of uniformly distributed probability values; in the dynamic routing process, the coupling coefficient is calculated in each round of routing according to Update the value of S3203: For each phenotype capsule j, calculate its activation value ; S3204: Get the output vector of each phenotype capsule , use the Squash function to ensure that the length of the output vector is between 0 and 1: ; S3205: Update of routing parameters: for each final phenotype capsule output, the weight coefficient is iteratively updated through the prediction vector between capsule pairs, and the number of updates is the same as the number of routings; The update strategy can be summarized as: ; S3206: After the data is calculated for the first time to obtain the output vector, the newly updated weight coefficient is obtained After the value of , dynamic routing starts; Repeat S3202 to S3205 until a predetermined number of iterations is reached; In S33, the final phenotypic capsule output vector is converted into the final sample category prediction probability output through the softmax function. The number of capsules in the last phenotypic layer is the same as the sample category in the dataset, and the capsule vector lengths in the phenotypic capsule layer are both .
9. A multi-view data classification system based on dynamic self-attention capsule network, characterized in that The method for implementing the multi-view data classification method based on the dynamic self-attention capsule network as claimed in any one of claims 1 to 8 comprises a dynamic self-attention capsule network, wherein the dynamic self-attention capsule network comprises a multi-view feature self-attention learning module, a view self-attention dynamic learning module, and a capsule network module. The multi-view feature self-attention learning module calculates the feature self-attention weight and the view self-attention weight for each view, performs self-attention weighting, and obtains feature-weighted and view-weighted data features; The view self-attention dynamic learning module is used to classify the weighted data of each view, dynamically learn the view self-attention weight through the predicted probability corresponding to the sample label category and the single weight value of the view, and the difference between the sample predicted probability and the label, and finally obtain the feature matrix M; The capsule network module is used to input the concatenated feature matrix M into the capsule network composed of a fully connected layer, a main capsule layer, and a phenotype capsule layer to complete the task of accurately classifying the sample data.
Citation Information
Patent Citations
Rectal cancer pathology image classification method based on multi-channel collaborative capsule network
CN111191660A
Low-illumination image classification method based on attention mechanism and capsule network
CN111950649A
Pulmonary nodule auxiliary diagnosis method based on three-dimensional multi-resolution attention capsule network
CN113208641A
Image classification method based on improved capsule network
CN116246110A
Multi-view multi-label classification method based on view position information
CN116452893A
Cited By
Underwater image depth estimation method oriented to real-time underwater robot and based on adaptive physical information and significance guidance
CN121640258A
An underwater image depth estimation method based on adaptive physical information and saliency guidance for real-time underwater robots
CN121640258B
Graph learning classification method based on granular balls under multi-view data and related device
CN121937809A
A method for learning classification based on particle ball under multi-view data and related device
CN121937809B
