Multi-view image fusion representation learning method, device and equipment based on double-layer optimization

Abstract

The invention discloses a multi-view image fusion representation learning method, device and equipment based on double-layer optimization, and relates to the technical field of machine learning. The method comprises the following steps: splicing feature matrixes of a multi-view data set to obtain a multi-view data matrix; calculating an initial affinity matrix of each view, and initializing a fusion affinity matrix, a view weight set, potential feature representation and a shared mapping matrix; a bilayer optimization model is constructed to learn potential feature representations. The model comprises an objective function of an upper-layer optimization task for updating potential feature representation and sharing a mapping matrix under the constraint of a fixed fusion affinity matrix, and an objective function of a lower-layer optimization task for updating the fusion affinity matrix and a view weight set under the constraint of the fixed potential feature representation. And executing alternative optimization iteration: executing upper-layer optimization task update and first, and then executing lower-layer optimization task update and until ending. And outputting the final potential feature representation for executing the task.

Description

Technical Field

[0001] This invention relates to the field of machine learning technology, and more specifically, to a multi-view graph fusion representation learning method, apparatus, and device based on two-layer optimization. Background Technology

[0002] Multi-view data refers to a collection of observations describing the same underlying object or phenomenon from different perspectives. For example, it captures the same event through modalities such as text, video, and audio. Its low-level representations are heterogeneous but share consistent high-level semantics. This data paradigm, by integrating complementary information from multiple dimensions and modalities, transcends the limitations of single-view data, providing a robust foundation for revealing the underlying structure and intrinsic properties of data. This drives the need to learn unified and robust latent representations to improve the performance of downstream tasks such as clustering and classification.

[0003] To achieve multi-view representation learning, existing methods mainly include canonical correlation analysis and its variants, such as sparse CCA and kernel CCA, which aim to project different views onto a shared low-dimensional subspace and maximize relevance. Deep learning-based methods use deep neural networks instead of linear projections to learn more expressive nonlinear latent representations. Graph-based methods explicitly model pairwise similarities between views; for example, graph convolutional networks discover consensus graphs by jointly integrating structural and feature information. Mainstream methods cover predefined graph fusion, learnable graph optimization, and subspace-based learning paradigms.

[0004] Despite the progress made by existing methods, they suffer from significant drawbacks. Existing methods typically neglect the coupled learning of graph topological consistency and feature space invariance, failing to adequately meet the need for coherent alignment of topology and feature distributions at multiple granularities. Mainstream methods employ simple strategies such as weighted averaging or final-layer attention mechanisms, which cannot jointly model topological consistency and feature invariance across multiple layers. This limits the completeness and discriminativeness of the resulting cross-view representations, creating an urgent need for new learning paradigms. Summary of the Invention

[0005] The present invention provides a multi-view graph fusion representation learning method, apparatus, and device based on two-layer optimization to improve at least one of the above-mentioned technical problems.

[0006] In a first aspect, the present invention provides a multi-view graph fusion representation learning method based on two-layer optimization, which includes steps S1 to S5.

[0007] S1. Obtain the multi-view dataset and concatenate the feature matrices of each view to obtain the multi-view data matrix. .

[0008] S2. Based on the multi-view dataset, calculate the initial affinity matrix for each view using a kernel function, and initialize the fusion affinity matrix based on the initial affinity matrix. and view weight set Simultaneously, the latent feature representation of multiple views. and shared mapping matrix Perform random initialization.

[0009] S3. Construct a two-layer optimization model based on the data matrix. and fusion affinity matrix Learning latent feature representations The two-layer optimization model comprises an upper-layer optimization task and a lower-layer optimization task. The objective function of the upper-layer optimization task is used to optimize the model with a fixed fusion affinity matrix. Update the latent feature representation under constraints. and shared mapping matrix The objective function of the lower-level optimization task is used to define a fixed latent feature representation. Under the constraints, update the fusion affinity matrix. and view weight set .

[0010] S4. Perform alternating optimization iterations: In each iteration, the upper-level optimization task is updated first. and Based on the updated Execute lower-level optimization task updates and The iteration continues until the preset termination condition is met.

[0011] S5. Output the final latent feature representation. With fusion affinity matrix Among them, latent features represent Used to perform clustering or classification tasks.

[0012] As a further aspect of the present invention, the two-layer optimization model is constructed as follows.

[0013] .

[0014] In the formula This represents the minimum value. Represents the real number field. This represents the number of samples. The dimension is the dimension of the low-dimensional space. It is the upper-level optimization objective function. This indicates a constraint condition. This represents the parameter that minimizes the objective function. It is the lower-level optimization objective function. To indicate the first Affinity matrix for each view. Indicates the number of views.

[0015] As a further aspect of the present invention, the lower-level optimization task is simplified to the following constraint optimization objective.

[0016] .

[0017] The constraints are: .

[0018] In the formula Indicates the first The weight of each view. These are pre-defined hyperparameters. for and The similarity. This represents the minimum value. It is the lower-level optimization objective function. Indicates the number of views. To indicate the first Affinity matrix for each view. This represents the Frobenius norm. The weighting coefficients for consistency loss. This represents the number of samples. and These are the indexes of the samples. and They are respectively The row and number row vector Each row of vectors represents the encoding of a sample. This represents the L2 norm.

[0019] As a further aspect of the present invention, S4 performs lower-level optimization task updates. and Includes: fixed fusion affinity matrix Update the view weight set through closed-form solution Among them, weight The solution model is as follows.

[0020] when hour: .

[0021] when hour: .

[0022] In the formula for The optimal solution. This represents the parameter that minimizes the objective function.

[0023] As a further aspect of the present invention, S4 performs lower-level optimization task updates. and Also includes: Fixed view weight set The fusion affinity matrix is ​​updated using the projective gradient descent method. .

[0024] .

[0025] .

[0026] .

[0027] .

[0028] .

[0029] In the formula For the updated This is the second stage of the projection operation. This is the first stage of the projection operation. For the first The fusion affinity matrix at the next iteration. For the first The learning rate at the next iteration. It is the upper-level optimization objective function At point The gradient at that point. Based on the current feature representation matrix The calculated squared distance matrix between samples. for The Row vectors. This indicates taking the larger value. This indicates transpose.

[0030] As a further aspect of the present invention, the objective function of the upper-level optimization task... Specifically, it is defined as a simplified loss applicable to both unsupervised and semi-supervised learning.

[0031] .

[0032] .

[0033] In the formula and These are the indexes of the samples. It is a symmetric normalized affinity matrix. for medium sample and samples The similarity values ​​between them after symmetry adjustment. and They are respectively The row and number row vector Each row of vectors represents the encoding of a sample. Degree matrix The One diagonal element. The degree matrix of the first degree One diagonal element. These are weighted parameters. For the first A labeled sample with one-hot encoding. yes The quantity. This represents the L2 norm. This represents the Frobenius norm.

[0034] As a further aspect of the present invention, the upper-level optimization task update is performed in S4. and The steps are as follows.

[0035] Fixed shared mapping matrix By solving the Sylvester equation, we can obtain... The solution. The form of the Sylvester equation is as follows.

[0036] .

[0037] When labels for all data are unavailable, the Sylvester equation takes the following form.

[0038] .

[0039] In the formula It is an identity matrix. This is a transpose. This represents the true label matrix of the labeled samples.

[0040] Fixed latent feature representation Update the shared mapping matrix .

[0041] As a further aspect of the present invention, in step S2, the kernel function is a Gaussian kernel, a polynomial kernel, or a linear kernel.

[0042] Fusion Affinity Matrix Initialize to the average of the initial affinity matrices of all views.

[0043] View weight set Initialize to a uniform distribution, meaning each view has an equal weight, equal to one-tenth the number of views.

[0044] Random initialization specifically involves randomly generating a matrix of a preset shape from a standard normal distribution.

[0045] Secondly, the present invention provides a multi-view graph fusion representation learning device based on two-layer optimization, which includes a data module, an initialization module, a model module, an iteration module and an output module.

[0046] The data module is used to acquire a multi-view dataset and concatenate the feature matrices of each view to obtain the multi-view data matrix. .

[0047] The initialization module is used to calculate the initial affinity matrix for each view based on the multi-view dataset using a kernel function, and to initialize the fusion affinity matrix based on the initial affinity matrix. and view weight set Simultaneously, the latent feature representation of multiple views. and shared mapping matrix Perform random initialization.

[0048] The model module is used to build a two-level optimization model based on the data matrix. and fusion affinity matrix Learning latent feature representations The two-layer optimization model comprises an upper-layer optimization task and a lower-layer optimization task. The objective function of the upper-layer optimization task is used to optimize the model with a fixed fusion affinity matrix. Update the latent feature representation under constraints. and shared mapping matrix The objective function of the lower-level optimization task is used to define a fixed latent feature representation. Under the constraints, update the fusion affinity matrix. and view weight set .

[0049] The iteration module is used to perform alternating optimization iterations: in each iteration, the upper-level optimization task is updated first. and Based on the updated Execute lower-level optimization task updates and The iteration continues until the preset termination condition is met.

[0050] The output module is used to output the final latent feature representation. With fusion affinity matrix Among them, latent features represent Used to perform clustering or classification tasks.

[0051] Thirdly, the present invention provides a multi-view graph fusion representation learning device based on two-layer optimization, characterized in that it includes a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a multi-view graph fusion representation learning method based on two-layer optimization as described in any paragraph of the first aspect.

[0052] By adopting the above technical solution, the present invention can achieve the following technical effects: This invention introduces a two-layer optimization framework, formalizing the joint learning of graph topological consistency and feature space invariance into a rigorous mathematical problem. This effectively overcomes the limitations of existing multi-view fusion methods, which typically employ simple weighted averaging or final-layer attention mechanisms. The framework adaptively fuses the graph structures of each view and learns view weights through a lower-layer optimization task, while simultaneously learning discriminative latent feature representations under the constraints of the fused graph structure through an upper-layer optimization task. This achieves tight coupling and alternating optimization of graph construction and feature learning between the two layers. This mechanism enables the invention to coherently align the topology and feature distribution of multi-view data at multiple granularities, thereby learning a more complete and consistent cross-view representation.

[0053] Thanks to the aforementioned fundamental innovations, this invention achieves significant and stable performance improvements on various real-world datasets. Experiments show that in clustering tasks, this method consistently outperforms existing state-of-the-art methods on six benchmark datasets, achieving a maximum accuracy improvement of 9.92% and an average improvement of 6.31%, while also demonstrating good convergence. Furthermore, this method is insensitive to changes in key hyperparameters, maintaining excellent performance over a wide range, demonstrating strong robustness and practicality, and providing a more reliable unified representation for the application of multi-view data in downstream tasks such as clustering and classification. Attached Figure Description

[0054] To more clearly illustrate the technical solution of the present invention, the accompanying drawings used in the specific embodiments of the present invention will be briefly introduced below. It should be understood that the following drawings only show some specific embodiments of the present invention and should not be regarded as a limitation on the scope. For those skilled in the art, other related drawings can be obtained from these drawings without creative effort.

[0055] Figure 1 This is a logical block diagram of a two-layer optimization method (integrating affinity matrix construction and feature representation learning through hierarchical objectives).

[0056] Figure 2This is a convergence analysis graph (loss, accuracy, and F1 score) for the Animals dataset.

[0057] Figure 3 This is a convergence analysis graph (loss, accuracy, and F1 score) for the BDGP dataset.

[0058] Figure 4 This is a convergence analysis graph (loss, accuracy, and F1 score) for the COIL20 dataset.

[0059] Figure 5 This is a convergence analysis graph (loss, accuracy, and F1 score) for the Hdigit dataset.

[0060] Figure 6 This is a convergence analysis graph (loss, accuracy, and F1 score) for the WebKB dataset.

[0061] Figure 7 This is a convergence analysis graph (loss, accuracy, and F1 score) from the Wikipedia dataset. Detailed Implementation

[0062] The technical solutions of the present invention will now be clearly and completely described with reference to the accompanying drawings in the embodiments of the present invention.

[0063] Example 1: The framework of the two-layer optimization method proposed in this example is as follows: Figure 1 As shown, it explicitly models the coupling between graph topology and feature invariance. The framework refines view-specific representations by solving lower-level optimization tasks and determines the optimal fusion strategy through upper-level optimization tasks, together ensuring a complete and task-effective representation for both clustering and classification tasks.

[0064] Please see Figure 1 The first embodiment of the present invention provides a multi-view graph fusion representation learning method based on two-layer optimization, which can be executed by a multi-view graph fusion representation learning device based on two-layer optimization (hereinafter referred to as: multi-view graph fusion representation device). In particular, it is executed by one or more processors in the multi-view graph fusion representation device to implement steps S1 to S5.

[0065] S1. Obtain the multi-view dataset and concatenate the feature matrices of each view to obtain the multi-view data matrix. The multi-view dataset contains... Feature matrix of each view .

[0066] S2. Based on the multi-view dataset, use a kernel function to calculate the initial affinity matrix for each view. The fusion affinity matrix is ​​initialized based on the initial affinity matrix. and view weight set Simultaneously, the latent feature representation of multiple views. and shared mapping matrix Perform random initialization.

[0067] The kernel function used is a Gaussian kernel, a polynomial kernel, or a linear kernel. Among them, the Gaussian kernel (RBF kernel) is the most commonly used kernel function because it can effectively capture the nonlinear relationships in the data and performs well in multi-view learning.

[0068] Fusion Affinity Matrix Initialized to the average of the initial affinity matrices of all views. View weight set. The initialization is uniform, meaning each view has an equal weight, equal to one-tenth the number of views. This initialization ensures that the contribution of each view is equal at the start of optimization, providing a reasonable starting point for subsequent alternating optimization.

[0069] Random initialization specifically involves randomly generating a matrix of a pre-defined shape from a standard normal distribution. This random initialization method is standard practice in deep learning and optimization problems, ensuring that the optimization process starts from a reasonable random starting point, avoiding symmetry issues, and helping the model learn meaningful feature representations.

[0070] S3. Construct a two-layer optimization model based on the data matrix. and fusion affinity matrix Learning latent feature representations The two-layer optimization model consists of an upper-layer optimization task and a lower-layer optimization task.

[0071] The objective function of the upper-level optimization task is used to optimize the fusion affinity matrix. Update the latent feature representation under constraints. and shared mapping matrix .

[0072] The objective function of the lower-level optimization task is used to fix the latent feature representation. Under the constraints, update the fusion affinity matrix. and view weight set .

[0073] S4. Perform alternating optimization iterations: In each iteration, the upper-level optimization task is updated first. and Based on the updated Execute lower-level optimization task updates and The iteration continues until the preset termination condition is met.

[0074] S5. Output the final latent feature representation. With fusion affinity matrix Among them, latent features represent Used to perform clustering or classification tasks.

[0075] Existing multi-view fusion methods still suffer from a core limitation. Mainstream methods typically employ simple strategies, such as weighted averaging or final-layer attention mechanisms, which fail to adequately meet the need for coherent alignment of topology and feature distributions at multiple granularities.

[0076] This invention proposes a novel two-layer optimization framework that formally unifies the joint learning of graph topological consistency and feature space invariance, establishing a new paradigm for multi-view graph representation learning. The invention also designs an efficient alternating optimization mechanism, in which lower-layer tasks jointly optimize view-specific graph structures and latent representations, while upper-layer tasks learn task-adaptive fusion strategies, thereby enhancing the discriminativeness and robustness of the representations.

[0077] This embodiment conducts extensive experiments on benchmark datasets to demonstrate that the multi-view graph fusion representation learning method based on two-layer optimization of the present invention consistently outperforms state-of-the-art baseline methods in clustering tasks, and its effectiveness is verified through convergence and parameter sensitivity analysis.

[0078] The key mathematical symbols used in this invention are explained below.

[0079] Indicates the number of views. This represents the number of samples. The number of features. Represents the real number field.

[0080] For the first The feature matrix of each view. This indicates that multi-view data has One view, One sample and One characteristic. It is a data matrix of multiple views, which is obtained by concatenating the feature matrices of multiple views. .

[0081] To indicate the first Affinity matrix for each view. Represents the set of affinity matrices for a multi-view dataset. To fuse affinity matrices. It is a non-negative matrix (i.e., all elements are non-negative).

[0082] This is a set of view weights. Indicates the first The weight of each view. .

[0083] The weighting coefficients for consistency loss. These are the parameters for the second algorithm. Represents the trace of a matrix. This represents the Frobenius norm. This indicates transpose. For all column vectors and sets .

[0084] Specifically, given a multi-view dataset ( This embodiment attempts to start from data splicing. and evaluation-based multi-graph The obtained fusion affinity matrix In the middle, learn a low-dimensional representation .

[0085] .

[0086] Used to learn a low-dimensional representation The two-level optimization model is constructed as follows.

[0087] .

[0088] In the formula This represents the minimum value. Represents the real number field. This represents the number of samples. The dimension is the dimension of the low-dimensional space. It is the upper-level optimization objective function. This indicates a constraint condition. This represents the parameter that minimizes the objective function. It is the lower-level optimization objective function. To indicate the first Affinity matrix for each view. Indicates the number of views.

[0089] The upper-level optimization objective function is used to obtain a low-dimensional representation. The lower-level optimization objective function is used to update the fusion affinity matrix. The multi-view learning problem is formulated as a two-level minimization problem, solved by alternately updating the upper and lower level optimization tasks. Ideally, the upper level problem has a closed-form solution. Therefore, the two-level optimization problem is simplified.

[0090] Since the upper-level optimization problem (primary objective) depends on the optimal solution of the lower-level optimization problem (constraints), this embodiment first defines the lower-level optimization task and then defines the upper-level optimization task to clarify the " The constraint is that it must be the optimal solution to the lower-level problem.

[0091] In the lower-level optimization task, for the objective function Assuming the learned fusion affinity matrix With affinity matrix set The learned latent feature representations are compatible and normalized to ensure their effectiveness as graph learning methods. Simultaneously, consistency properties drive the learning of latent feature representations. With fusion affinity matrix The consistency relationship between them.

[0092] Therefore, the above problem can be further written in the following form.

[0093] .

[0094] In the formula, The compatibility loss is due to the multi-view dataset and the latent feature representation. The consistency loss is applied to the multi-view dataset and the latent feature representation. The weighting coefficients for consistency loss. for A set of constraints. The multi-view dataset is a data matrix of multiple views. Affinity matrix A set of.

[0095] In the objective function It is a constraint used to ensure the learned fusion affinity matrix It can generate efficient graph structures. Specifically, The constraints include (Row and normalization), indicating The sum of each row of elements is 1, ensuring that the sum of the weights of all edges of each node is 1, which is used to construct the Laplacian matrix in graph regularization. (Non-negativity) indicates All elements in the graph are non-negative, which ensures the physical meaning of the graph edge weights (the similarity should be a non-negative value).

[0096] In the upper-level optimization task, its objective function Specifically, it is defined as a simplified loss applicable to both unsupervised and semi-supervised learning.

[0097] .

[0098] .

[0099] .

[0100] In the formula For sharing the mapping matrix. and These are the indexes of the samples. It is a symmetric normalized affinity matrix. for medium sample and samples The similarity values ​​between them after symmetry adjustment. and They are respectively The row and number row vectors ( Each row vector in the vector represents the encoding of a sample. The degree matrix of the first degree One diagonal element. The degree matrix of the first degree One diagonal element. For the first A labeled sample with one-hot encoding. yes The quantity. These are weighted parameters. This represents the L2 norm. This indicates transpose.

[0101] for The degree matrix is ​​a diagonal matrix. The elements are express The Row vectors. Let be the graph Laplace matrix. .

[0102] The first term in the formula is the graph Laplacian smoothing term (or graph regularization term), the second term is the feature reconstruction term, and the third term is the label fitting term (semi-supervised). Based on consistency, the second term is used to encourage the use of shared latent features. The original multi-view features are reconstructed. The third step aims to obtain a more reliable latent representation when partial labels exist. Essentially, the upper-level optimization acts as a label or pseudo-label propagation stage.

[0103] This embodiment uses a two-layer optimization scheme and an alternating update strategy to solve the formalized multi-view learning problem, thereby optimizing the solution process.

[0104] The lower-level optimization task can be further simplified to the following constraint optimization objective.

[0105] .

[0106] The constraints are: .

[0107] In the formula Indicates the first The weight of each view. These are pre-defined hyperparameters. for and The similarity. To indicate the first Affinity matrix for each view. where is the weighting coefficient for consistency loss. It is the set of view weights to be optimized.

[0108] This embodiment will consider compatibility loss. Defined as weighted fit Consistency loss Set to: It enforces consistency: for any , and The closer the distance between them, the higher the similarity. The larger.

[0109] In addition, constraints Concretized as and This ensures the learned fusion affinity matrix. It is the adjacency matrix of a certain graph. And... , Ensured It can be used as a normalized view weight. Represents the weight vector with all-1 vectors The inner product (i.e., the sum of all elements) is equal to 1. express Each component is non-negative.

[0110] This embodiment uses an alternating update strategy to calculate the optimization variables. and .

[0111] Specifically, step S4 involves updating the lower-level optimization task. and Includes: fixed fusion affinity matrix Update the view weight set through closed-form solution .

[0112] This embodiment uses the Lagrange multiplier method for updating. Lagrange function The definition is as follows.

[0113] .

[0114] In the formula It is a correspondence constraint Lagrange multipliers.

[0115] Therefore, regarding The optimal solution is in Obtained at the saddle point, that is... and Let be the partial derivative of the Lagrange function.

[0116] when hour: .

[0117] In the formula It means that it is always equal to.

[0118] Combination To obtain the optimal and as follows.

[0119] .

[0120] .

[0121] In the formula for The optimal solution. for The optimal solution.

[0122] when When, the optimal solution It can be represented in the following form.

[0123] .

[0124] This indicates that a trivial solution has been obtained, namely, the model training uses only a graph structure with a single view.

[0125] Preferably, step S4 involves updating the lower-level optimization task. and Also includes: Fixed view weight set The fusion affinity matrix is ​​updated using the projective gradient descent method. .

[0126] This embodiment uses the projected gradient method to solve the problem. Considering two constraints and Since it is difficult to solve simultaneously, this embodiment constructs a two-stage projective gradient descent method.

[0127] .

[0128] .

[0129] .

[0130] In the formula For the updated This is the second stage of the projection operation. This is the first stage of the projection operation. For the first The fusion affinity matrix at the next iteration. For the first The learning rate at the next iteration. It is the upper-level optimization objective function At point The gradient at that point. Based on the current feature representation matrix The calculated squared distance matrix between samples. for The Row vectors.

[0131] and They are respectively to feasible sets and Projection operations.

[0132] Given an initial point According to the two-stage projection gradient descent method (i.e. The solution model is iterated until the termination condition is met. The Euclidean projection onto the feasible set is defined as follows.

[0133] .

[0134] Furthermore, using the Lagrange multiplier method, the explicit form of the two projections is as follows.

[0135] .

[0136] .

[0137] Preferably, the upper-level optimization task update is performed in S4. and Includes: Fixed shared mapping matrix By solving the Sylvester equation, we can obtain... The solution.

[0138] Specifically, the upper-level optimization involves updating the simplified loss. and This embodiment solves the problem using an alternating optimization strategy. When the fixed Update The subproblem is finding the optimal solution to the simplified loss.

[0139] .

[0140] This is a second-order optimization function without any constraints. This embodiment... about Taking the derivative, we obtain the following solution model. .

[0141] Setting the above derivative to zero, we obtain Equation 1.

[0142] .

[0143] When all data labels are unavailable, the above equation transforms into Equation 2.

[0144] .

[0145] In the formula It is an identity matrix. This is a transpose. This represents the true label matrix of the labeled samples.

[0146] Equations 1 and 2 are both Sylvester equations, and their closed-form solutions can be obtained using the Bartels-Stewart algorithm. Specifically, the optimal solutions for the semi-supervised and unsupervised cases are obtained by solving equations (16) and (17), respectively.

[0147] Preferably, the upper-level optimization task update is performed in S4. and Also includes: fixed latent feature representation Update the shared mapping matrix .

[0148] when When fixed, solve The subproblem is simplified to the following problem.

[0149] .

[0150] This is a second-order optimization function without any constraints. The closed-form solution can be easily obtained through the following equation: .

[0151] In summary, by summarizing the optimal solutions to all optimization subproblems, the process for solving the joint learning of multi-view feature representation and consistency graph fusion can be summarized as follows: Input , , , And the maximum number of iterations MaxIter, then perform the following steps to output: and .

[0152] 1. Initialize the optimization variables to be solved and .

[0153] 2. Evaluate the graph affinity matrix using some existing kernel method. .

[0154] 3. Number of updates and iterations.

[0155] 4. Use the Barlets-Stewart algorithm to solve Equation 1 (for semi-supervised classification) or Equation 2 (for clustering).

[0156] 5. According to renew To optimize upper-level tasks.

[0157] 6. If convergence fails, continue with subsequent steps; otherwise, stop and output the result. and .

[0158] 7. Use an alternating update strategy to optimize lower-level tasks.

[0159] 8. According to Solution model (fixed) )renew .

[0160] 9. According to Solution model (fixed) )renew .

[0161] 10. If the maximum number of iterations is reached, stop and output the result. and Otherwise, return to step 3.

[0162] To facilitate understanding of the present invention, the application of this embodiment will be described below using specific application scenarios.

[0163] The proposed method is comprehensively evaluated below, and compared with eight existing baseline methods on six public datasets. All experiments were conducted on a PC equipped with an 11th-generation Intel Core i7-11800H processor (2.30GHz), 32GB of memory, and an NVIDIA GeForce RTX 3060 GPU.

[0164] The experiments were conducted on six public benchmark datasets. Detailed dataset specifications are summarized in Table 1.

[0165] Table 2. Multi-view clustering benchmark dataset: sample size, number of views, feature dimensions per view, and number of categories.

[0166]

[0167] Animals: A dataset containing 10,158 animal images across 50 classes. Following standard feature extraction protocols, this example uses pre-computed DECAF and VGG19 representations.

[0168] BDGP: Provides spatiotemporal gene expression profiles of Drosophila embryo slices using fluorescence microscopy, covering developmental stages. Six characteristic modalities (from Gabor to LBP) are extracted from each image to create a dual analytical view.

[0169] COIL20: Contains 1,440 images of 20 objects, each photographed at 72 rotation angles around its vertical axis. The dataset is divided into 20 classes, and each sample is represented in three different viewpoint-induced feature spaces.

[0170] Hdigit: A benchmark set of handwritten digit images containing 10,000 samples, represented by dual complementary views.

[0171] WebKB contains 8,282 academic web pages from the computer science departments of four universities: Cornell, Washington, Wisconsin, and Texas. The pages are divided into six categories (students, staff, faculty, departments, courses, and projects), with text content and hyperlinked views. Experiments use a subset from Washington.

[0172] Wikipedia: Provides a corpus of 693 documents in 10 categories, with each entry having a dual low-dimensional representation: a 10-dimensional and a 128-dimensional feature view.

[0173] To verify the effectiveness of the bi-level optimization theory proposed in this embodiment, this case was compared and evaluated with eight state-of-the-art baseline methods.

[0174] SMVSC proposes a joint anchor graph optimization framework that overcomes the limitations of heuristic sampling through a unified objective, achieving accurate data distribution modeling and enhanced discriminability. Its complexity is suitable for large-scale subspace clustering.

[0175] DSMVC proposes a performance-guaranteed deep learning framework that employs a two-process learning mechanism. This architecture simultaneously extracts complementary information from multiple views, automatically selects discriminative features, and filters out meaningless noise.

[0176] DSRL implements reusable modular networks with learnable piecewise linear activation functions for end-to-end deep sparse regularization, where differentiable modular networks adaptively learn data-driven regularizers to establish parameterized regularization modeling.

[0177] FSMSC introduces a two-module self-guided framework. Its local modules establish a locally consistent representation of shared views, while its global module derives globally discriminative features from the stitched multi-view features. Norm-constrained feature selection is used to guide the process from global to local.

[0178] CVCL employs a two-stage architecture in which a deep autoencoder first extracts view-specific features and then leverages these features through cluster-level cross-view comparisons to discover consistent semantic patterns.

[0179] SLMVGC proposes a unified framework to address two key challenges in multi-view clustering: explicitly modeling the underlying structure in the manifold space through topological learning to overcome the limitations of traditional methods that ignore topology; and enhancing cross-view clustering by leveraging sample-level statistical regularities of similar local structures across views. Figure 1 To the point of being responsive.

[0180] SMVAGC integrates anchor graph construction and alignment into a unified optimization framework, eliminating the traditional two-stage separation. It further adaptively fuses multi-scale anchor graphs through structural alignment regularization to maintain distributional diversity while preserving linear complexity for efficient large-scale processing.

[0181] AGLDR implements a dual-distribution noise removal mechanism to construct a low-rank constrained cross-view structure. Figure 1 An anchor point diagram is used to capture the global structure, while Gaussian and Laplace noise is eliminated through a dual optimization objective.

[0182] To verify the effectiveness of the proposed method, this embodiment compares it with eight existing baseline methods on six public datasets. The results for the four evaluation metrics (ACC, NMI, ARI, and F1-Score) are shown in Table 2.

[0183] Table 2 compares the node clustering performance of six public datasets with eight existing baseline methods.

[0184]

[0185] For a comprehensive evaluation, this example uses six small datasets (including BDGP, ​​COIL20, WebKB, and Wikipedia) and two large benchmarks (Animals and Hdigit). The results are discussed separately for each dataset size.

[0186] On small datasets, the method in this embodiment achieves state-of-the-art ACC on all datasets.

[0187] WebKB: The method in this embodiment achieves... Its accuracy significantly surpassed that of the second-ranked method, DSRL. It leads by 9.92 percentage points.

[0188] COIL20: The proposed method achieved its goal. Its accuracy surpasses that of FSMVC. It leads by 8.93 percentage points.

[0189] Wikipedia: The framework of this embodiment was obtained Its accuracy is superior to that of the second-ranked method, DSMVC. It leads by 4.00 percentage points.

[0190] For large-scale datasets, Table 3 shows that the method in this embodiment maintains the performance of the second method. arrive Its performance advantage proves that its effectiveness remains consistent even on more challenging data.

[0191] Animals: The method in this embodiment achieves... Its accuracy surpassed that of the second-ranked method, DSMVC. It leads by 5.79 percentage points.

[0192] Hdigit: The proposed method has achieved its goal. Its accuracy surpasses that of DSMVC. It leads by 2.93 percentage points.

[0193] In summary, the method in this embodiment consistently outperforms existing baselines on both small and large datasets, and improves the average ACC by [percentage missing] compared to the second-best method. This performance gain is particularly significant because it demonstrates the framework's scalability and adaptability when handling datasets of varying sizes and complexities.

[0194] like Figures 2 to 7 As shown, this embodiment presents a comprehensive convergence analysis of the proposed method on six different datasets (Animals, BDGP, ​​Hdigit, COIL20, WebKB, and Wikipedia). By examining the evolution of training loss, accuracy, and F1 score during optimization, this embodiment aims to empirically verify the stability, efficiency, and robustness of the proposed framework under different data characteristics and complexities.

[0195] On the BDGP dataset, the training loss decreases rapidly with increasing epochs and then stabilizes after convergence. Correspondingly, accuracy and F1 score also show a corresponding upward trend, reaching stable values ​​at approximately 700 epochs. On the Hdigit dataset, the training loss increases sharply at approximately 20 epochs, then gradually decreases until the model converges. Correspondingly, accuracy and F1 score also show synchronous improvement at these key points, demonstrating satisfactory overall convergence characteristics.

[0196] The Wikipedia dataset exhibits rapid convergence, stabilizing within fewer than 1500 epochs. The observed discrepancy between the F1 score and accuracy suggests that the model overfits to the majority class while struggling to effectively identify the minority class. Similar anomalous spikes were also observed in the WebKB and COIL20 datasets.

[0197] WebKB's convergence process is similar to Hdigit's, undergoing two distinct phases before stabilizing at approximately 3000 epochs. In contrast, COIL20 experienced a more gradual convergence, stabilizing at around 3000 epochs. The animals dataset, due to its large-scale nature, exhibited a significantly longer convergence process compared to other benchmarks, only stabilizing after approximately 3000 epochs.

[0198] Although both accuracy and F1 score converged rapidly within 200 epochs, the significant difference between them indicates an inherent imbalance in classification performance, similar to the behavior observed on the Wikipedia dataset, revealing a consistent pattern of inconsistent metrics.

[0199] In summary, the consistent convergence pattern observed across all six datasets provides strong empirical evidence for the robustness of the method in this embodiment. The convergence speed varies appropriately with dataset size and complexity, ranging from rapid convergence on Wikipedia to prolonged but stable convergence on large-scale datasets such as Animals. The framework of this embodiment maintains consistent behavior throughout the optimization trajectory. The differences between accuracy and F1 score observed on the Wikipedia and Animals datasets primarily reflect an inherent imbalance in classification performance rather than indicating a fundamental instability in training. The convergence properties of these systems further demonstrate the model's ability to handle complex learning dynamics and confirm its reliability for multi-view representation learning in diverse real-world scenarios.

[0200] To evaluate the robustness of the proposed method, this embodiment systematically analyzes... and Impact on model performance. Experiments were conducted on six benchmark datasets, among which... Control Updates The regularization strength, Control Updates The constraints.

[0201] Model performance It exhibits low sensitivity, maintaining stable results across all six benchmark datasets with only minor fluctuations, demonstrating strong robustness. A local variation was observed on the BDGP dataset when... Set to 10 and When the value varies between 0.2 and 0.4, performance fluctuations become more pronounced, but this does not diminish the overall stability of the method. Furthermore, the parameters... The algorithm exhibits different behavioral patterns across different datasets. On the Hdigit and Wikipedia datasets, the algorithm updates... It exhibits strong robustness in terms of constraints. For the BDGP and WebKB datasets, when When the value is less than 1, the method maintains stability comparable to Hdigit and Wikipedia, but when... Accuracy dropped significantly when the value was increased to 10 and 100. As for the Animals and COIL20 datasets, accuracy only decreased when... When the accuracy reaches 100, significant performance fluctuations in accuracy begin to appear.

[0202] The optimal parameter region maintained strong consistency across different datasets. These findings indicate that: 1. The update process requires appropriate regularization constraints. 2. The update mechanism is relatively insensitive to changes in constraint strength. 3. Set to approximately 0.1, Setting the parameter to 1 yields stable and excellent performance on most datasets. This analysis provides clear guidance for parameter configuration in practical applications and verifies the parameter stability of the proposed framework.

[0203] This study explores the integration of graph topological consistency and feature space invariance in multi-view learning through a principled two-layer optimization method, demonstrating consistent performance improvements across diverse benchmarks. The framework in this embodiment systematically coordinates view-specific representation learning and global consensus formation through an alternating optimization mechanism between lower-layer representation refinement and upper-layer fusion strategy learning.

[0204] Example 2: The present invention provides a multi-view graph fusion representation learning device based on two-layer optimization, which includes a data module, an initialization module, a model module, an iteration module and an output module.

[0205] The data module is used to acquire a multi-view dataset and concatenate the feature matrices of each view to obtain the multi-view data matrix. .

[0206] The initialization module is used to calculate the initial affinity matrix for each view based on the multi-view dataset using a kernel function, and to initialize the fusion affinity matrix based on the initial affinity matrix. and view weight set Simultaneously, the latent feature representation of multiple views. and shared mapping matrix Perform random initialization.

[0207] The model module is used to build a two-level optimization model based on the data matrix. and fusion affinity matrix Learning latent feature representations The two-layer optimization model comprises an upper-layer optimization task and a lower-layer optimization task. The objective function of the upper-layer optimization task is used to optimize the model with a fixed fusion affinity matrix. Update the latent feature representation under constraints. and shared mapping matrix The objective function of the lower-level optimization task is used to define a fixed latent feature representation. Under the constraints, update the fusion affinity matrix. and view weight set .

[0208] The iteration module is used to perform alternating optimization iterations: in each iteration, the upper-level optimization task is updated first. and Based on the updated Execute lower-level optimization task updates and The iteration continues until the preset termination condition is met.

[0209] The output module is used to output the final latent feature representation. With fusion affinity matrix Among them, latent features represent Used to perform clustering or classification tasks.

[0210] Example 3: This invention provides a multi-view graph fusion representation learning device based on two-layer optimization, characterized in that it includes a processor, a memory, and a computer program stored in the memory. The computer program can be executed by the processor to implement a multi-view graph fusion representation learning method based on two-layer optimization as described in any paragraph of Example 1.

[0211] It is understood that the multi-view graph fusion representation device can be an electronic device with computing power, such as a portable laptop computer, desktop computer, server, smartphone, or tablet computer.

[0212] Obviously, the embodiments described above are only some embodiments of the present invention, and not all embodiments. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without inventive effort are within the scope of protection of the present invention.

[0213] In the several embodiments provided in this invention, it should be understood that the disclosed apparatus and methods can also be implemented in other ways. The apparatus and method embodiments described above are merely illustrative. For example, the flowcharts and block diagrams in the accompanying drawings illustrate the architecture, functionality, and operation of possible implementations of apparatus, methods, and computer program products according to various embodiments of the present invention. In this regard, each block in a flowchart or block diagram may represent a module, segment, or portion of code containing one or more executable instructions for implementing a specified logical function. It should also be noted that in some alternative implementations, the functions marked in the blocks may occur in a different order than those marked in the drawings. For example, two consecutive blocks may actually be executed substantially in parallel, and they may sometimes be executed in reverse order, depending on the functions involved. It should also be noted that each block in a block diagram and / or flowchart, and combinations of blocks in block diagrams and / or flowcharts, can be implemented using a dedicated hardware-based system that performs the specified function or action, or using a combination of dedicated hardware and computer instructions.

[0214] In addition, the functional modules in the various embodiments of the present invention can be integrated together to form an independent part, or each module can exist independently, or two or more modules can be integrated to form an independent part.

[0215] If the aforementioned functions are implemented as software functional modules and sold or used as independent products, they can be stored in a computer-readable storage medium. Based on this understanding, the technical solution of the present invention, or the part that contributes to the prior art, or a part of the technical solution, can be embodied in the form of a software product. This computer software product is stored in a storage medium and includes several instructions to cause a computer device (which may be a personal computer, electronic device, or network device, etc.) to execute all or part of the steps of the methods described in the various embodiments of the present invention. The aforementioned storage medium includes various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory, random access memory, magnetic disks, or optical disks. It should be noted that, in this document, the terms "comprising," "including," or any other variations thereof are intended to cover non-exclusive inclusion, such that a process, method, article, or apparatus that comprises a list of elements includes not only those elements but also other elements not expressly listed, or elements inherent to such a process, method, article, or apparatus. Without further limitations, an element defined by the phrase "comprising one..." does not exclude the presence of other identical elements in the process, method, article, or apparatus that includes said element.

[0216] The terminology used in the embodiments of this invention is for the purpose of describing particular embodiments only and is not intended to be limiting of the invention. The singular forms “a,” “the,” and “the” used in the embodiments of this invention are also intended to include the plural forms unless the context clearly indicates otherwise.

[0217] It should be understood that the term "and / or" used in this article is merely a description of the relationship between related objects, indicating that three relationships can exist. For example, A and / or B can represent: A existing alone, A and B existing simultaneously, and B existing alone. Additionally, the character " / " in this article generally indicates that the preceding and following related objects have an "or" relationship.

[0218] Depending on the context, the word "if" as used here can be interpreted as "when," "when," "in response to determination," or "in response to detection." Similarly, depending on the context, the phrase "if determination" or "if detection (of the stated condition or event)" can be interpreted as "when determination," "in response to determination," "when detection (of the stated condition or event)," or "in response to detection (of the stated condition or event)."

[0219] The terms "first" and "second" used in the embodiments are merely to distinguish similar objects and do not represent a specific ordering of objects. It is understood that "first" and "second" can be interchanged in a specific order or sequence where permitted. It should be understood that the objects distinguished by "first" and "second" can be interchanged where appropriate so that the embodiments described herein can be implemented in an order other than those illustrated or described herein.

[0220] The above description is merely a preferred embodiment of the present invention and is not intended to limit the invention. Various modifications and variations can be made to the present invention by those skilled in the art. Any modifications, equivalent substitutions, improvements, etc., made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A multi-view graph fusion representation learning method based on two-layer optimization, characterized in that, Include: S1. Obtain the multi-view dataset and concatenate the feature matrices of each view to obtain the multi-view data matrix. ; S2. Based on the multi-view dataset, calculate the initial affinity matrix for each view using a kernel function, and initialize the fusion affinity matrix based on the initial affinity matrix. and view weight set Simultaneously, the latent feature representation of multiple views. and shared mapping matrix Perform random initialization; S3. Construct a two-layer optimization model based on the data matrix. and fusion affinity matrix Learning latent feature representations The two-layer optimization model comprises an upper-layer optimization task and a lower-layer optimization task; the objective function of the upper-layer optimization task is used to optimize a fixed fusion affinity matrix. Update the latent feature representation under constraints. and shared mapping matrix The objective function of the lower-level optimization task is used to define a fixed latent feature representation. Under the constraints, update the fusion affinity matrix. and view weight set ; S4. Perform alternating optimization iterations: In each iteration, the upper-level optimization task is updated first. and Based on the updated Execute lower-level optimization task updates and This continues until the preset iteration termination condition is met; S5. Output the final latent feature representation. With fusion affinity matrix Among them, latent features represent Used to perform clustering or classification tasks.

2. The multi-view graph fusion representation learning method based on two-layer optimization according to claim 1, characterized in that, The two-level optimization model is constructed as follows; ； In the formula Indicates the minimum value; Represents the real number field; The number of samples; The dimension of a low-dimensional space; It is the upper-level optimization objective function; Indicates constraints; This represents the parameters that minimize the objective function; It is the lower-level optimization objective function; To indicate the first Affinity matrix of each view; Indicates the number of views.

3. The multi-view graph fusion representation learning method based on two-layer optimization according to claim 1, characterized in that, The lower-level optimization task can be simplified to the following constraint optimization objective; ； The constraints are: ; In the formula Indicates the first The weight of each view; These are pre-defined hyperparameters; for and Similarity; Indicates the minimum value; It is the lower-level optimization objective function; Indicates the number of views; To indicate the first Affinity matrix of each view; Denotes the Frobenius norm; The weighting coefficients for consistency loss; The number of samples; and These are the indexes of the samples; and They are respectively The row and number row vector Each row of vectors represents the encoding of a sample; This represents the L2 norm.

4. The multi-view graph fusion representation learning method based on two-layer optimization according to claim 3, characterized in that, S4 executes lower-level optimization task updates and Includes: fixed fusion affinity matrix Update the view weight set through closed-form solution Among them, weight The solution model is as follows; when hour: ; when hour: ; In the formula for The optimal solution; This represents the parameter that minimizes the objective function.

5. The multi-view graph fusion representation learning method based on two-layer optimization according to claim 3, characterized in that, S4 executes lower-level optimization task updates and Also includes: Fixed view weight set The fusion affinity matrix is ​​updated using the projective gradient descent method. ; ； ； ； ； ； In the formula For the updated ; This is the second stage of the projection operation; This is the first stage of the projection operation; For the first The fusion affinity matrix at the next iteration; For the first Learning rate at the next iteration; It is the upper-level optimization objective function At point gradient at; Based on the current feature representation matrix The calculated squared distance matrix between samples; for The Row vectors; This indicates taking the larger value; This indicates transpose.

6. The multi-view graph fusion representation learning method based on two-layer optimization according to claim 1, characterized in that, Objective function of upper-level optimization task Specifically defined as a simplified loss applicable to unsupervised and semi-supervised learning; ； ； In the formula It is a symmetric normalized affinity matrix; for medium sample and samples The similarity values ​​between them after symmetry adjustment; Degree matrix The One diagonal element; The degree matrix of the first degree One diagonal element; These are weighted parameters; For the first A labeled sample with one-hot encoding; yes Quantity; Indicates the minimum value; and These are the indexes of the samples; and They are respectively The row and number row vector Each row of vectors represents the encoding of a sample; Represents the L2 norm; This represents the Frobenius norm.

7. The multi-view graph fusion representation learning method based on two-layer optimization according to claim 6, characterized in that, S4 performs upper-level optimization tasks and updates. and include: Fixed shared mapping matrix By solving the Sylvester equation, we can obtain... The solution; the form of the Sylvester equation is as follows; ； When labels for all data are unavailable, the Sylvester equation takes the following form; ； In the formula It is the identity matrix; For transpose; This represents the true label matrix of the labeled samples; Fixed latent feature representation Update the shared mapping matrix ; .

8. A multi-view graph fusion representation learning method based on bi-layer optimization according to any one of claims 1 to 7, characterized in that, In step S2, the kernel function is a Gaussian kernel, a polynomial kernel, or a linear kernel; Fusion Affinity Matrix Initialize to the average of the initial affinity matrices of all views; View weight set Initialize to a uniform distribution, meaning each view has an equal weight, equal to one-tenth the number of views; Random initialization specifically involves randomly generating a matrix of a preset shape from a standard normal distribution.

9. A multi-view graph fusion representation learning device based on two-layer optimization, characterized in that, Include: The data module is used to acquire a multi-view dataset and concatenate the feature matrices of each view to obtain the multi-view data matrix. ; The initialization module is used to calculate the initial affinity matrix for each view based on the multi-view dataset using a kernel function, and to initialize the fusion affinity matrix based on the initial affinity matrix. and view weight set Simultaneously, the latent feature representation of multiple views. and shared mapping matrix Perform random initialization; The model module is used to build a two-level optimization model based on the data matrix. and fusion affinity matrix Learning latent feature representations The two-layer optimization model comprises an upper-layer optimization task and a lower-layer optimization task; the objective function of the upper-layer optimization task is used to optimize a fixed fusion affinity matrix. Update the latent feature representation under constraints. and shared mapping matrix The objective function of the lower-level optimization task is used to define a fixed latent feature representation. Under the constraints, update the fusion affinity matrix. and view weight set ; The iteration module is used to perform alternating optimization iterations: in each iteration, the upper-level optimization task is updated first. and Based on the updated Execute lower-level optimization task updates and This continues until the preset iteration termination condition is met; The output module is used to output the final latent feature representation. With fusion affinity matrix Among them, latent features represent Used to perform clustering or classification tasks.

10. A multi-view graph fusion representation learning device based on two-layer optimization, characterized in that, It includes a processor, a memory, and a computer program stored in the memory; the computer program can be executed by the processor to implement a multi-view graph fusion representation learning method based on two-layer optimization as described in any one of claims 1 to 8.

Images