Multi-view semi-supervised classification method based on graph Laplacian learning network

Abstract

The invention belongs to the technical field of multi-view classification, and particularly relates to a multi-view semi-supervised classification method based on a graph Laplacian learning network. According to the method, multi-view data is taken as input, firstly, a graph structure is constructed for each view, a graph Laplacian matrix corresponding to each view is obtained, and the graph Laplacian matrixes are fused through an attention mechanism. In order to more effectively utilize the topological relation of data and alleviate the problem of information loss generated in the graph construction process, the scheme combines a spectrogram theory, and introduces a fused Laplacian matrix as a regular term into a graph convolution network, so that node representation keeps smooth change in the geodesic line direction of a data manifold in the multilayer graph convolution process, and the problem of information loss generated in the graph construction process is solved. Therefore, the robustness of the model to noise and abnormal values is enhanced, and the classification performance is improved. Furthermore, sparsity constraint is introduced to the fused multi-view graph structure, and structural redundancy is effectively reduced.

Description

Technical Field

[0001] This invention belongs to the field of multi-view classification technology, specifically relating to a multi-view semi-supervised classification method based on a graph Laplacian learning network. Background Technology

[0002] Classification is a core task in many machine learning applications, enabling systems to make informed decisions based on data, thus driving advancements across various fields. However, most existing classification algorithms heavily rely on large-scale labeled datasets during training, which are often scarce and costly to acquire in practical applications. Furthermore, with varying data collection methods and diverse feature extraction techniques, data often exhibits multi-view characteristics, meaning the same sample has multiple feature representations. Each feature representation (or view) captures specific and complementary information about the target sample. Effectively fusing these multi-view feature representations helps to better utilize their complementarity, thereby improving classification discriminative power. To address these challenges, multi-view semi-supervised classification methods have gradually gained attention as a promising research direction. These methods accurately learn the latent label distribution of the data by fusing multi-view data features from labeled and unlabeled data.

[0003] In existing literature, graph construction methods are an important category in multi-view semi-supervised classification. These methods typically construct an independent similarity graph for each view based on the similarity of sample pairs, and comprehensively utilize multi-view information through well-designed fusion strategies. Despite some achievements, these methods still have limitations that affect their performance and applicability. First, they usually use Euclidean distance as the similarity metric to construct the similarity graph, which is susceptible to noise and outliers, weakening the graph's expressive power and thus affecting the subsequent multi-graph fusion effect. Second, the graph construction process transforms sample features into a graph structure, inevitably ignoring some original feature information, resulting in the fused graph not accurately depicting the true relationships between samples, thus affecting the final classification performance. Finally, most existing methods assume that view weights have a certain prior distribution, neglecting the characteristics of the data structure. Summary of the Invention

[0004] To address the aforementioned problems, this invention proposes a multi-view semi-supervised classification method based on a learnable graph Laplacian learning network. Considering multi-view data as input, a graph structure is first constructed for each view, and the corresponding graph Laplacian matrix is ​​obtained. These matrices are then fused using an attention mechanism. To more effectively utilize the topological relationships of the data and mitigate information loss during graph construction, this scheme incorporates spectral graph theory, introducing the fused Laplacian matrix as a regularization term into the graph convolutional network. This ensures that the node representations maintain a smooth change along the geodesic direction of the data manifold during multi-layer graph convolution, thereby enhancing the model's robustness to noise and outliers and improving classification performance. Furthermore, sparsity constraints are introduced into the fused multi-view graph structure, effectively reducing structural redundancy.

[0005] The technical solution of this invention is as follows:

[0006] A multi-view semi-supervised classification method based on a graph Laplacian learning network includes the following steps:

[0007] S1. Acquire data, including labeled and unlabeled data; use the feature extraction module to extract data features using different feature extraction methods to construct multi-view data. Define the first... The characteristics of the view data are ,in Indicates the first The first sample Individual view data characteristics, , Represents the real number field. The number of data samples. For the first The feature dimensions of each view Indicates matrix transpose; transposes all data. After concatenating the features of each view, a multi-view feature matrix is ​​obtained. ;

[0008] S2. Construct a multi-view semi-supervised classification model based on a graph Laplacian learning network. The model includes a graph structure construction module, a graph Laplacian matrix generation module, and a graph Laplacian learning network.

[0009] The graph structure construction module builds a graph structure based on the acquired feature data. Specifically, it constructs an adjacency matrix based on the features extracted from each view data. For each sample feature in the data features of each view, retain its relationship with the previous view. The similarity of the nearest neighbor samples is used to construct the nearest neighbor similarity matrix. :

[0010] ,

[0011] in, Indicates sample of The nearest neighbor samples, similarity matrix That is, the first A graph structure of view data;

[0012] The graph Laplacian matrix generation module generates a graph Laplacian matrix based on the acquired graph structure. Specifically, it generates a corresponding graph Laplacian matrix for the graph structure of each view data, defining the first... The regularized Laplacian matrix generated by each view is ,in for identity matrix , , It is the first The degree matrix of each view;

[0013] The input to the Graph Laplacian learning network is the output of the S1 feature extraction module and the output of the Graph Laplacian matrix generation module. The output is defined as follows: The method to obtain the output from the input is to solve the following optimization objective function:

[0014] ,

[0015] in, Indicates the first One convolutional layer, This represents the total number of convolutional layers in the network. It is the first The input of each convolutional layer The multi-view feature matrix is ​​the output of the feature extraction module. yes Through learnable weight matrix transformation, For the number of categories, For learnable multi-view shared graph Laplacian matrix, and They represent -norm and -norm, Represents the weight vector. This indicates that the weight vector is fused with the input graph Laplacian matrix. For non-negative hyperparameters, and Each element is equal to 1. and of Dimensional column vector.

[0016] To optimize the objective function, maintain , and Fixed, the solution is obtained about Closed form solution:

[0017] ,

[0018] Through Taylor's demonstration and further consideration Given the constraints, we can approximate the following:

[0019] ,

[0020] in, The activation function is the approximation obtained. As the information propagation mechanism of the Graph Laplace learning network, the final result is... As the output of the network, it is used to indicate the final category label;

[0021] S3. A multi-view semi-supervised classification model is constructed by training with given label information. Specifically, the graph structure construction module and the graph Laplacian matrix generation module in the model are used to process the labeled and unlabeled multi-view data features obtained by the feature extraction module in S1. The process is performed to obtain the multi-view diagram structure. and the Laplace matrix of multi-view graphs After random initialization, the Graph Laplace learning network is based on the input data. and The message transmission mechanism described above uses the following loss function:

[0022] ,

[0023] in, and It is a balancing factor for weighing the three losses; For structural consistency loss, A graph Laplacian matrix for multi-view data sharing; For sparsity loss; Cross-entropy loss is used to measure the difference between the given label information and the network output. Differences between them:

[0024] ,

[0025] in For the set of label indicators Induced label matrix, ;

[0026] Update weights using attention mechanism Optimization through backpropagation and The training of a multi-view semi-supervised classification model is obtained after convergence or reaching the set number of iterations.

[0027] S4. Using the trained multi-view semi-supervised classification model, through:

[0028] ,

[0029] Identify unlabeled samples Category labels, where, This represents the set of indicators for all samples.

[0030] The beneficial effects of this invention are as follows: This invention primarily targets the classification and prediction task of multi-view data, enhancing the accuracy of the classification task based on both an adaptive multi-view data fusion network and a semi-supervised algorithm. Compared with existing methods, adaptive fusion overcomes the limitation of existing graph neural network-based methods that rely on static graphs; the proposed learning network elucidates the optimization objective behind the propagation mechanism, making the model interpretable. This interpretability helps the network reveal potential patterns in the data and improves its ability to generalize from finite-labeled data to unlabeled data; simultaneously, by introducing Laplace regularization and sparsity constraints, the model's discriminative ability is enhanced, facilitating classification. Attached Figure Description

[0031] Figure 1 This is a frame diagram of the device according to an embodiment of the present invention.

[0032] Figure 2 This is a schematic diagram comparing the performance of the present invention with that of existing technologies on the multi-view dataset Handwritten numerals (HW) and YouTube.

[0033] Figure 3 This is a schematic diagram comparing the performance of the present invention with that of existing technologies on the multi-view datasets NUS-WIDE and COIL.

[0034] Figure 4 This is a schematic diagram comparing the performance of the present invention with that of existing technologies on the multi-view datasets Scene15 and Caltech10.

[0035] Figure 5 This diagram illustrates a comparison of the performance of the present invention with existing technologies on the multi-view datasets Attributes (AwA) and Reuters. Detailed Implementation

[0036] The present invention will now be described in detail with reference to the accompanying drawings and embodiments.

[0037] Example:

[0038] like Figure 1 As shown, this example describes in detail the classification of images with 10 categories of labels: logo, flower, airplane, face, car, bird, butterfly, apple, horse, and ball. The steps include:

[0039] S1: Data is retrieved from the internet using web scraping technology. In the entire dataset, each sample's category is uniquely represented by a label; some samples have label information, called labeled data, while the remaining unlabeled samples are called unlabeled data. In this example, the labels correspond to 10 categories: logo, flower, airplane, face, car, bird, butterfly, apple, horse, and ball. The feature extraction module uses different feature extraction methods to extract data features and construct multi-view data. All extracted data The multi-view feature matrix is ​​obtained by concatenating the features of each view. ,in, Indicates the first Individual view data characteristics, The number of data samples. For the first The feature dimensions of each view This indicates the matrix transpose.

[0040] S2: Construct a multi-view semi-supervised classification model based on a graph Laplacian learning network. The model includes a graph structure construction module, a graph Laplacian matrix generation module, and a graph Laplacian learning network.

[0041] The graph structure construction module builds a graph structure based on the acquired feature data. Specifically, it constructs an adjacency matrix based on the features extracted from each view data. For each sample feature in the view data, the nearest neighbor number is... of Method: Construct the nearest neighbor similarity matrix :

[0042] ,

[0043] here, express of The nearest neighbor samples, similarity matrix That is, the first A graph structure of view data;

[0044] The Graph Laplacian Matrix Generation Module is based on the acquired graph structure. Generating the Graph Laplacian Matrix Specifically ,in for identity matrix , , It is the first The degree matrix of each view;

[0045] The input to the Graph Laplacian learning network is the output of the feature extraction module in S1 and the output of the Graph Laplacian matrix generation module. The output is defined as follows: The method of obtaining output from input is through network propagation mechanism.

[0046]

[0047] Proceed. Here, For embedding factors, Indicates the first One convolutional layer, This represents the total number of convolutional layers in the network. The Laplacian matrix of the learnable multi-view shared graph is obtained. Finally, we get... ( The number of categories is used as the output of the network to indicate the final category label;

[0048] S3: A multi-view semi-supervised classification model trained using given label information. This embodiment uses an embedding factor of... A two-layer graph Laplacian learning network with 128 hidden dimensions and a 0 dropout rate (i.e., , To learn about the Laplace matrix of a multi-view shared graph , respectively and Used as the activation function for both the hidden and output layers. Initialization. and weight ( (Number of views), and simultaneously initialize the parameters of the graph Laplacian learning network randomly based on the input data. and The information is transmitted through the above propagation mechanism, and the loss function used is:

[0049] ,

[0050] in, and It is a balancing factor for the trade-off between the three losses; in this embodiment, it is set to... . This represents a loss of structural consistency. For sparsity loss; Cross-entropy loss is used to measure the difference between the given label information and the network output. Differences between them:

[0051] ,

[0052] in, For the set of label indicators Induced label matrix, ;here,

[0053] Update weights using attention mechanism The algorithm was optimized using the Adam optimizer with a default learning rate of 0.01 and a weight decay of 0.0005. and The training continues until convergence or at most 1000 iterations are performed to obtain a well-trained multi-view semi-supervised classification model.

[0054] S4. Using the trained multi-view semi-supervised classification model, through:

[0055] ,

[0056] Determine unlabeled images The category labels are ultimately assigned to all unlabeled data, resulting in 10 categories: logo, flower, airplane, face, car, bird, butterfly, apple, horse, or ball. This represents the set of indicators for all samples.

[0057] like Figure 2-5 As shown, the classification prediction performance of this invention is compared with other existing methods on eight multi-view datasets (Handwritten numerals (HW), YouTube, NUS-WIDE, COIL, Scene15, Caltech101, Animals with Attributes (AwA), and Reuters). The proportion of correctly classified samples, i.e., ACC, is used as the evaluation metric for classification performance. We divide the percentage of labeled samples in each dataset from 10% to 80% for training, and the obtained performance reflects the ability of different semi-supervised learning methods to extract information from the data. To mitigate the influence of randomness, we repeat each algorithm 10 times at different label proportions and then report the average classification accuracy and standard deviation on each dataset. The table shows that the proposed solution of this invention has a certain performance improvement compared to classic machine learning algorithms and the latest deep learning algorithms, which verifies the effectiveness of this invention in classification prediction methods.

Claims

1. A multi-view semi-supervised classification method based on a graph Laplacian learning network, characterized in that, Includes the following steps: S1. Acquire data, including labeled and unlabeled data; extract data features using different feature extraction methods through the feature extraction module to construct multi-view data; define the first... The characteristics of the view data are: ,in Indicates the first The first sample Individual view data characteristics, , Represents the real number field. The number of data samples. For the first The feature dimensions of each view Indicates matrix transpose; transposes all data. After concatenating the features of each view, a multi-view feature matrix is ​​obtained. ; S2. Construct a multi-view semi-supervised classification model based on a graph Laplacian learning network. The model includes a graph structure construction module, a graph Laplacian matrix generation module, and a graph Laplacian learning network. The graph structure construction module builds a graph structure based on the acquired feature data. Specifically, it constructs an adjacency matrix based on the features extracted from each view data. For each sample feature in the data features of each view, retain its relationship with the previous view. The similarity of the nearest neighbor samples is used to construct the nearest neighbor similarity matrix. : ， in, Indicates sample of The nearest neighbor samples, similarity matrix That is, the first A graph structure of view data; The graph Laplacian matrix generation module generates a graph Laplacian matrix based on the acquired graph structure. Specifically, it generates a corresponding graph Laplacian matrix for the graph structure of each view data, defining the first... The regularized Laplacian matrix generated by each view is ,in for identity matrix , , It is the first The degree matrix of each view; The input to the Graph Laplacian learning network is the output of the S1 feature extraction module and the output of the Graph Laplacian matrix generation module. The output is defined as follows: The method to obtain the output from the input is to solve the following optimization objective function: ， in, Indicates the first One convolutional layer, This represents the total number of convolutional layers in the network. It is the first The input of each convolutional layer The multi-view feature matrix is ​​the output of the feature extraction module. yes Through learnable weight matrix transformation, For the number of categories, For learnable multi-view shared graph Laplacian matrix, and They represent -norm and -norm, Represents the weight vector. This indicates that the weight vector is fused with the input graph Laplacian matrix. For non-negative hyperparameters, and Each element is equal to 1. and of 3D column vector; To optimize the objective function, maintain , and Fixed, the solution is obtained about Closed form solution: ， Through Taylor's demonstration and further consideration Given the constraints, we can approximate the following: ， in, For the activation function; the approximate result is... As the information propagation mechanism of the Graph Laplace learning network, the final result is... As the output of the network, it is used to indicate the final category label; S3. A multi-view semi-supervised classification model is constructed by training with given label information. Specifically, the graph structure construction module and the graph Laplacian matrix generation module in the model are used to process the labeled and unlabeled multi-view data features obtained by the feature extraction module in S1. The process is performed to obtain the multi-view diagram structure. and the Laplace matrix of multi-view graphs After random initialization, the Graph Laplace learning network is based on the input data. and Message transmission is performed through an information propagation mechanism, and the loss function used is: ， in, and It is a balancing factor for weighing the three losses; For structural consistency loss, A graph Laplacian matrix for multi-view data sharing; For sparsity loss; Cross-entropy loss is used to measure the difference between the given label information and the network output. Differences between them: ， in For the set of label indicators Induced label matrix, ; Update weights using attention mechanism Optimization through backpropagation and The training of a multi-view semi-supervised classification model is obtained after convergence or reaching the set number of iterations. S4. Using the trained multi-view semi-supervised classification model, through: ， Identify unlabeled samples Category labels, where, This represents the set of indicators for all samples.

Images