Multi-modal remote sensing data clustering method, equipment and medium
By using a multi-objective multi-view attribute graph learning method, combined with superpixel segmentation and neighbor coding, the graph structure is optimized, which solves the problems of high computational complexity and insufficient robustness in traditional multimodal remote sensing data classification, realizes efficient multimodal remote sensing data clustering, and improves the classification accuracy and scalability of large-scale data.
Patent Information
- Application Number
- CN202510438444.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-09
- Publication Date
- 2025-09-05
AI Technical Summary
Traditional multimodal remote sensing data classification methods have high computational complexity, lack of robustness and low efficiency. They are difficult to expand, especially when processing large-scale data. They are also sensitive to outliers and ignore global relationships, which affects the clustering effect.
A multi-objective multi-view attribute graph learning method is adopted. The graph structure is initialized through entropy rate superpixel segmentation and k-nearest neighbor algorithm. The graph structure is optimized by combining filtering and multi-objective evolutionary algorithm. The spectral clustering algorithm is used to complete the classification. The superpixel segmentation and neighbor coding method are combined to reduce the computational complexity and improve the efficiency of graph structure optimization.
It effectively reduces computational complexity, improves graph structure optimization efficiency and clustering accuracy, and is particularly suitable for the joint analysis and classification of large-scale multimodal remote sensing data, showing stronger discrimination ability and scalability.
Smart Images
Figure CN120599308A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the field of remote sensing data classification, and in particular to a multimodal remote sensing data clustering method. Background Art
[0002] Land cover classification is a core task in remote sensing image analysis, which involves assigning a specific land feature label to each pixel. This task is crucial to the Earth observation community, and with advances in Earth observation technology, the joint classification of multimodal data has become an important research direction.
[0003] Traditional supervised learning methods, including both shallow and deep learning models, have achieved remarkable success in fusing multimodal remote sensing data (such as hyperspectral and LiDAR). However, these methods rely on large amounts of manually annotated data, especially in deep learning frameworks, resulting in high costs and labor-intensive work. Therefore, unsupervised methods, such as Multi-View Clustering (MVC), have gradually attracted attention.
[0004] Multi-view subspace clustering captures the intrinsic structure of multimodal data through low-dimensional subspace representation, but still faces the following challenges:
[0005] 1. Objective conflict and hyperparameter dependence: Traditional multi-view subspace clustering methods combine multiple optimization objectives and rely on hyperparameter adjustment, which can easily lead to suboptimal solutions.
[0006] 2. High computational complexity: Full-map construction requires a lot of computation, making it difficult to scale to large-scale remote sensing data. Anchor graphs reduce computational complexity but ignore spatial characteristics.
[0007] 3. Lack of robustness: Existing methods are sensitive to outliers and are only based on local similarities, ignoring global relationships, which affects the clustering effect. Summary of the Invention
[0008] The purpose of the present invention is to propose a multimodal remote sensing data clustering method, device and medium to solve the technical problems of high computational complexity, lack of robustness and low efficiency in traditional multimodal remote sensing data classification.
[0009] Specifically, the present invention provides a multimodal remote sensing data clustering method, which is specifically a multimodal remote sensing data clustering method based on multi-target multi-view attribute graph learning (MMAGL), which includes the following steps:
[0010] S1, obtain multimodal remote sensing data, and use the entropy rate superpixel segmentation algorithm to generate superpixel segmentation results S;
[0011] S2, using the k-nearest neighbor algorithm to initialize the graph structure of the multimodal remote sensing data for the segmentation result S;
[0012] S3. Filter the graph structure to obtain an optimized graph structure;
[0013] S4, fusing the optimized graph structures to obtain a fused graph, and encoding the fused graph as a population in the evolutionary algorithm;
[0014] S5, using the multi-objective evolutionary algorithm to obtain the optimal fusion graph G by solving the corresponding optimization equation * ;
[0015] S6. In the optimal fusion graph G * The spectral clustering algorithm is applied to complete the classification of the target image.
[0016] A storage medium stores instructions and data for implementing a multimodal remote sensing data clustering method.
[0017] A multimodal remote sensing data clustering device comprises: a processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement a multimodal remote sensing data clustering method.
[0018] The beneficial effects provided by the present invention are as follows: by combining superpixel-based sample reduction and proximity-based population coding methods, not only the computational complexity is effectively reduced, but also the optimization efficiency of the graph structure is improved; in addition, by utilizing a multi-objective evolutionary algorithm, the multi-view subspace clustering problem is transformed into a multi-objective optimization task, taking into account multiple conflicting objectives such as data reconstruction, smoothness, and structural consistency, thereby achieving a global optimal solution. When processing multimodal remote sensing data, this method fully utilizes the complementarity between different modalities and overcomes the limitations of traditional methods in processing complex data structures. Experimental results verify that this method is superior to existing remote sensing benchmark methods in overall accuracy, showing stronger discrimination ability and better scalability, and is particularly suitable for joint analysis and classification tasks of large-scale, multimodal remote sensing data. BRIEF DESCRIPTION OF THE DRAWINGS
[0019] Figure 1 It is a simple flow chart of the method of the present invention;
[0020] Figure 2 It is a schematic diagram of the process of the method of the present invention;
[0021] Figure 3 Schematic diagram of the working of the hardware device according to an embodiment of the present invention;
[0022] Figure 4 Schematic diagram of the comparison between the present invention and the most advanced multimodal clustering method on the TRENTO dataset;
[0023] Figure 5Schematic diagram of the comparison between the present invention and the most advanced multimodal clustering method on the MUUFL dataset;
[0024] Figure 6 Schematic diagram of the comparison between the present invention and the most advanced multimodal clustering method on the HOUSTON 2013 dataset. DETAILED DESCRIPTION
[0025] To make the objectives, technical solutions and advantages of the present invention more clear, the embodiments of the present invention will be further described below with reference to the accompanying drawings.
[0026] Before formally explaining the present invention, the scheme of the present invention is first generally explained for easy understanding.
[0027] Please refer to Figure 1-Figure 2 The present invention provides a multimodal remote sensing data clustering method, comprising:
[0028] S1, obtain multimodal remote sensing data, and use the entropy rate superpixel segmentation algorithm to generate superpixel segmentation results S;
[0029] Specifically, the multimodal remote sensing data of the present invention includes hyperspectral data and LIDAR data.
[0030] In order to simplify the calculation process, improve data expression ability and enhance clustering effect, the present invention uses the entropy rate superpixel segmentation algorithm to generate superpixels, and solves the model based on the segmentation results. The relevant formulas are shown in formula (1) and formula (2):
[0031] S={S i ||S i |=n i ,i=1,2,…,M}(1)
[0032]
[0033] In formula (1), S represents the superpixel set, S i represents the i-th superpixel area, n i Represents superpixel S i The number of pixels in M represents the total number of superpixels, which is much smaller than the total number of pixels N. In formula (2), Q represents the correlation matrix between superpixels and pixels, q ij The value of p depends on j Is it a super pixel S i , if yes, then it is 1, otherwise it is 0.
[0034] The calculation formula of super-pixel node attributes is shown in formula (3):
[0035] X=QX p (3)
[0036] In formula (3), X is the node attribute matrix, X p Represents the pixel-level feature matrix.
[0037] S2, using the k-nearest neighbor algorithm to initialize the graph structure of the multimodal remote sensing data for the segmentation result S;
[0038] It should be noted that when initializing the attribute graph, the present invention first constructs the super-pixel level node S according to the result of super-pixel segmentation. i and the incidence matrix Then, the adjacency relationship is established and the edge set ε is calculated using the k-nearest neighbor (KNN) algorithm to initialize the graph structure of the multimodal remote sensing data, as shown in formula (4):
[0039] ε={(x i ,x j )|x j ∈N k (x i )orx i ∈x k (x j )}(4)
[0040] Finally, the initial attribute graph of multimodal remote sensing data is constructed, corresponding to G H and G L , as shown in formula (5):
[0041] G H =(E H ,A H ,X H ),G L =(E L ,A L ,X L )(5)
[0042] In formula (5), G H , G L They are respectively the graphical representation of multimodal remote sensing data; E H 、E L The corresponding edge set is constructed by the k-nearest neighbor algorithm and self-loop; A H 、A L represents the adjacency matrix; X H 、X L Represents the node attribute matrix.
[0043] S3. Filter the graph structure to obtain an optimized graph structure;
[0044] The filtering in this invention is performed according to a specific formula. The filtering is mainly used to extract discriminative graph embeddings. Its core idea is based on spectral graph convolution, which uses low-pass filtering to reduce noise and enhance the separability of embeddings for more effective clustering. It is specifically expressed as:
[0045] Define the graph signal denoising problem, the goal is to optimize Equation (6) to obtain the graph embedding after denoising:
[0046]
[0047] In formula (6), H H Graph embedding representing hyperspectral data, X H Represents the feature matrix of the original node, L H The adjacency matrix A H The calculated symmetric normalized Laplace matrix, α is the regularization coefficient.
[0048] By taking the partial derivative of formula (6), we get its analytical solution as shown in formula (7):
[0049] H H =(I+αL H ) -1 X H (7)
[0050] In formula (7), I is the unit matrix, (I+αL H ) -1 It is a propagation matrix of a graph signal, defined as formula (8):
[0051] P H =(I+αL H ) -1 (8)
[0052] For the implementation of low-pass filtering, according to formula (6), the frequency response function can be expressed as formula (9):
[0053]
[0054] In formula (9), λ is the eigenvalue of the Laplace matrix L. By adjusting the regularization coefficient α, the degree of filtering of high-frequency components can be controlled.
[0055] S4, fusing the optimized graph structures to obtain a fused graph, and encoding the fused graph as a population in the evolutionary algorithm;
[0056] Specifically, step S4 is as follows:
[0057] Encoding the fusion graph as a population in the evolutionary algorithm makes it possible to use the evolutionary algorithm, which specifically includes:
[0058] By merging the hyperspectral map G H and LiDAR map G L The fusion graph is obtained, which is expressed as formula (10) as follows:
[0059]
[0060] In formula (10), G F Represented as a fused graph; by letting A F is the adjacency matrix of the fusion graph, which is regarded as the initial subspace representation. F Since it has symmetry, we only need to focus on the non-zero elements in the upper triangle.
[0061] In the evolution process of the multi-objective evolutionary algorithm, each individual is encoded as a real-valued vector, which is expressed as formula (11):
[0062] u={u i ∈[-1,1]|i=1,…,l}(11)
[0063] In formula (11), l represents the fusion graph G F The number of edges.
[0064] Through a mapping function φ(u), the individual u is mapped to a subspace representation matrix Z, which is expressed as formula (12):
[0065] Z=φ(u)(12)
[0066] S5, using the multi-objective evolutionary algorithm to obtain the optimal fusion graph G by solving the corresponding optimization equation * ;
[0067] Based on the above formula, a multi-objective optimization problem is carried out, which involves four objective functions, and its formula is expressed as formula (13):
[0068]
[0069] In formula (13), J(H,φ(u)H) represents the similarity loss function of the embedding result, ||u||0 is the sparse constraint, and δ i (L) is the eigenvalue of the Laplace matrix L.
[0070] The present invention optimizes the fusion graph using a multi-objective evolutionary algorithm and obtains the optimal fusion graph by solving the corresponding optimization equation;
[0071] Using multi-objective evolutionary algorithm optimization, first initialize the size to Npop The population is then optimized using a multi-objective evolutionary algorithm; the optimal solution set is obtained through evolutionary search. Used to predict the final cluster labels.
[0072] S6. In the optimal fusion graph G * The spectral clustering algorithm is applied to complete the classification of the target image.
[0073] In the optimal fusion graph G * The spectral clustering algorithm is applied to complete the classification of the target image.
[0074] The optimal fusion graph calculation, the specific steps are as follows:
[0075] Select the optimal solution Constructing the optimal fusion graph Its mathematical model is shown in formula (14):
[0076]
[0077] In formula (14), is the adjacency matrix of the optimal fusion graph. By applying spectral clustering to the graph, we get the superpixel level label: i , and then converted into pixel-level labels through the pixel-superpixel mapping matrix Q, as shown in formula (15):
[0078] Y i =Q T y i (15)
[0079] Y in formula (15) i Represents pixel-level cluster labels.
[0080] See Figure 3 , Figure 3 4 is a schematic diagram of the working of the hardware device of an embodiment of the present invention, wherein the hardware device specifically includes: a multimodal remote sensing data clustering device 401, a processor 402 and a storage medium 403.
[0081] A multimodal remote sensing data clustering device 401: The multimodal remote sensing data clustering device 401 implements the multimodal remote sensing data clustering method.
[0082] Processor 402: The processor 402 loads and executes the instructions and data in the storage medium 403 to implement the multimodal remote sensing data clustering method.
[0083] Storage medium 403: The storage medium 403 stores instructions and data; the storage medium 403 is used to implement the multimodal remote sensing data clustering method.
[0084] As an example, the present invention conducts extensive experiments on three real multimodal remote sensing datasets. Figure 4-Figure 6 .
[0085] pass Figure 4-Figure 6 The results show that the proposed method achieves competitive performance compared with the state-of-the-art methods. The experimental results show that MMAGL has the following advantages:
[0086] (1) MMAGL can handle multiple conflicting objectives simultaneously, avoiding the problem of neglecting the complex interactions between objectives when traditional methods simplify it into a weighted single-objective optimization, thereby improving the quality of subspace representation.
[0087] (2) MMAGL significantly compresses the decision space and optimizes computational efficiency by introducing superpixel-based sample reduction and proximity-based population coding, enabling multi-objective evolutionary algorithms to run effectively on large-scale datasets.
[0088] (3) MMAGL achieves sample reduction through superpixel segmentation, uses spatial priors to reduce computational complexity, and introduces structural priors through a proximity-based population coding method, ensuring that multimodal data can maintain good spatial consistency and structural information when clustering.
[0089] (4) MMAGL redefines the self-expression model and combines it with the contrastive learning method. By introducing generalized spectral graph convolution, it achieves efficient graph filtering, enhances the discriminative ability of graph structure, and improves the clustering effect, especially when processing complex data, it can better distinguish samples of different categories.
[0090] (4) MMAGL is highly adaptable and versatile, and is a flexible framework that can adapt to any type of multi-objective evolutionary algorithm. This allows the method to be combined with a variety of optimization algorithms, further enhancing its application potential in the joint classification task of multimodal remote sensing data.
[0091] The beneficial effects of the present invention are as follows: by combining superpixel-based sample reduction and proximity-based population coding methods, not only the computational complexity is effectively reduced, but also the optimization efficiency of the graph structure is improved; in addition, by utilizing a multi-objective evolutionary algorithm, the multi-view subspace clustering problem is transformed into a multi-objective optimization task, taking into account multiple conflicting objectives such as data reconstruction, smoothness, and structural consistency, thereby achieving a global optimal solution. When processing multimodal remote sensing data, this method fully utilizes the complementarity between different modalities and overcomes the limitations of traditional methods in processing complex data structures. Experimental results verify that this method is superior to existing remote sensing benchmark methods in overall accuracy, showing stronger discrimination ability and better scalability, and is particularly suitable for joint analysis and classification tasks of large-scale, multimodal remote sensing data.
[0092] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, improvements, etc. made within the spirit and principles of the present invention should be included in the scope of protection of the present invention.
Claims
1. A multimodal remote sensing data clustering method, characterized by: The following steps are involved: S1, obtain multimodal remote sensing data, and use the entropy rate superpixel segmentation algorithm to generate superpixel segmentation results S; S2, using the k-nearest neighbor algorithm to initialize the graph structure of the multimodal remote sensing data for the segmentation result S; S3. Filter the graph structure to obtain an optimized graph structure; S4, fusing the optimized graph structures to obtain a fused graph, and encoding the fused graph as a population in the evolutionary algorithm; S5, using the multi-objective evolutionary algorithm to obtain the optimal fusion graph G by solving the corresponding optimization equation * ; S6. In the optimal fusion graph G * The spectral clustering algorithm is applied to complete the classification of the target image.
2. The multimodal remote sensing data clustering method according to claim 1, wherein: The segmentation result S in step S1 is as follows: S={S i ||S i |=n i ,i=1,2,…,M}(1) In formula (1), S represents the superpixel set, S i represents the i-th superpixel area, n i Represents superpixel S i The number of pixels in, M represents the total number of superpixels, which is much smaller than the total number of pixels N; In formula (2), Q represents the correlation matrix between superpixels and pixels, q ij The value of p depends on j Is it a super pixel S i , if yes, then it is 1, otherwise it is 0.
3. The multimodal remote sensing data clustering method according to claim 2, wherein: The graph structure in step S2 is as follows: G H =(E H ,A H ,X H ),G L =(E L ,A L ,X L )(5) In formula (5), G H , G L are the graph representations of multimodal remote sensing data, where G H is the graph representation of hyperspectral data, G L is a graphical representation of LIDAR data; E H 、E L The corresponding edge set is constructed by the k-nearest neighbor algorithm and self-loop; A H 、A L represents the adjacency matrix; X H 、X L Represents the node attribute matrix.
4. The multimodal remote sensing data clustering method according to claim 3, wherein: In step S3, filtering is achieved by optimizing formula (6), which is shown as follows: In formula (6), H H Represents the graph embedding of hyperspectral data, that is, the original superpixel graph G is embedded by graph filtering technology H The low-dimensional representation after optimizing the node attributes, X H Represents the feature matrix of the original node, L H The adjacency matrix A H The calculated symmetric normalized Laplace matrix, α is the regularization coefficient; Tr represents the trace of the matrix.
5. The multimodal remote sensing data clustering method according to claim 3, wherein: The calculation process of obtaining the fusion graph in step S4 is as follows: In formula (7), G F Represented as a fused graph; by letting A F is the adjacency matrix of the fused graph, which is regarded as the initial subspace representation.
6. The multimodal remote sensing data clustering method according to claim 5, wherein: The encoding process in step S4 is as follows: Each individual is encoded as a real-valued vector, which is expressed as formula (8): in={in i ∈[-1,1]|i=1,…,l}(8) In formula (8), l represents the fusion graph G F The number of edges.
7. The multimodal remote sensing data clustering method according to claim 6, wherein: The objective function of the multi-objective evolutionary algorithm in step S5 is as follows: In formula (9), J(H,φ(u)H) represents the similarity loss function of the embedding result, ||u||0 is the sparse constraint, and δ i (L) is the eigenvalue of the Laplace matrix L; φ(u) is a mapping function.
8. The multimodal remote sensing data clustering method according to claim 7, wherein: The expression of the optimal fusion graph in step S6 is as follows: In formula (10), is the adjacency matrix of the optimal fusion graph. By applying spectral clustering to the graph, we get the superpixel-level label y i , and then converted into pixel-level labels through the pixel-superpixel mapping matrix Q, thus completing the target image classification.
9. A storage medium, characterized in that: The storage medium stores instructions and data for implementing a multimodal remote sensing data clustering method as described in any one of claims 1 to 8.
10. A multimodal remote sensing data clustering device, characterized by: include: A processor and a storage medium; the processor loads and executes instructions and data in the storage medium to implement a multimodal remote sensing data clustering method as described in any one of claims 1 to 8.