An unsupervised multi-modal remote sensing image feature classification method based on double random graph

Abstract

The application discloses a kind of based on double random graph's unsupervised multi-modal remote sensing image ground object classification method, comprising the following steps: one, obtains multi-modal remote sensing image and carries out superpixel segmentation, obtains the anchor point corresponding to multi-modal remote sensing image;Two, adaptive projection dimension reduction between any modal remote sensing image to anchor point, to obtain adaptive projection matrix and affinity matrix;Three, based on adaptive projection matrix and affinity matrix obtains double random matrix;Four, based on double random matrix constructs multi-modal remote sensing image fusion clustering graph model and optimizes solution and obtains optimal clustering index matrix;Five, based on optimal clustering index matrix, obtains the ground object category corresponding to each pixel point of remote sensing image.The application design is reasonable, based on double random matrix constructs multi-modal remote sensing image fusion clustering graph model to realize the acquisition of clustering indication matrix under unsupervised, so as to realize ground object classification according to clustering index matrix.

Description

Technical Field

[0001] This invention belongs to the field of remote sensing image processing technology, specifically relating to an unsupervised multimodal remote sensing image land cover classification method based on dual random graphs. Background Technology

[0002] Hyperspectral ( HS The images provide rich spectral and spatial details, while the lidar images ( LiDAR The data provides supplementary elevation information. While this data interprets some characteristics of ground features, data from a single sensor is often insufficient for accurate identification of ground objects. Although hyperspectral images contain multiple consecutive spectral bands and help provide detailed material characterization for applications such as urban planning, environmental monitoring, and mineral exploration, they cannot reliably distinguish objects with similar spectral features. This limitation can be addressed by integrating... LiDAR The data has been effectively processed. In practical applications, hyperspectral images are often combined with multispectral images... LiDAR Combining with other data sources can enhance surface observation capabilities and achieve accurate identification of ground features. However, current multimodal remote sensing image clustering directly fuses images and uses spectral clustering to classify the fused images, resulting in high computational complexity of the iterative process and reduced clustering accuracy.

[0003] Therefore, there is currently a lack of a well-designed unsupervised multimodal remote sensing image land cover classification method based on double random graphs. This method obtains a double random matrix by adaptive projection dimensionality reduction of multimodal remote sensing data, and constructs a multimodal remote sensing image fusion clustering graph model based on the double random matrix to obtain the clustering indicator matrix under unsupervised conditions. This allows for land cover classification based on the clustering indicator matrix, improving iteration efficiency and land cover identification accuracy. Summary of the Invention

[0004] The technical problem to be solved by this invention is to address the shortcomings of the prior art by providing an unsupervised multimodal remote sensing image land cover classification method based on a dual random graph. The method has simple steps and a reasonable design. It obtains a dual random matrix by adaptive projection dimensionality reduction of multimodal remote sensing data, and constructs a multimodal remote sensing image fusion clustering graph model based on the dual random matrix to obtain the clustering indicator matrix under unsupervised conditions. Thus, land cover classification is achieved based on the clustering indicator matrix, improving iteration efficiency and land cover identification accuracy.

[0005] To solve the above-mentioned technical problems, the technical solution adopted by the present invention is: an unsupervised multimodal remote sensing image land cover classification method based on dual random graphs, the method comprising the following steps: Step 1: Acquire multimodal remote sensing images and perform superpixel segmentation to obtain the anchor points corresponding to the multimodal remote sensing images; Step 2: Adaptive projection dimensionality reduction from any modal remote sensing image to the anchor point to obtain the adaptive projection matrix and affinity matrix; Step 3: Obtain a double random matrix based on the adaptive projection matrix and affinity matrix; Step 4: Construct a multimodal remote sensing image fusion clustering graph model based on a double random matrix and optimize the solution to obtain the optimal clustering index matrix; Step 5: Based on the optimal clustering index matrix, obtain the land cover category corresponding to each pixel in the remote sensing image.

[0006] The aforementioned unsupervised multimodal remote sensing image land cover classification method based on dual random graphs is further refined in step one, which is as follows: Step 101: Acquire the 1st modal remote sensing image, ..., the vth modal remote sensing image, ..., the mth modal remote sensing image; where m and v are both positive integers, and 1 ≤ v ≤ m; the size of the vth modal remote sensing image is... W represents the width of the remote sensing image, and H represents the height of the remote sensing image. This represents the feature dimension of the v-th modal remote sensing image; Step 102: Combine the m modal remote sensing images according to the number of channels to obtain a high-dimensional remote sensing image of W×H×D; where, ; Step 103: Convert the high-dimensional remote sensing image of W×H×D into a two-dimensional matrix; where the number of rows in the two-dimensional matrix is ​​W×H and the number of columns in the two-dimensional matrix is ​​D. Step 104: Use principal component analysis to reduce the dimensionality of the two-dimensional matrix and convert the two-dimensional matrix of the first principal component into a grayscale image of the principal components with a size of W×H. Step 105: Use the entropy rate superpixel segmentation method to segment the principal component grayscale image according to the set number of superpixel blocks I, to obtain I superpixel blocks; where I is a positive integer; Step 106, according to The i-th anchor point corresponding to the i-th superpixel block on the v-th modal remote sensing image is obtained. ;in, Let represent the feature data of the e-th pixel in the i-th superpixel block of the v-th modal remote sensing image, where the value of e ranges from 1 to 2. The value of i ranges from 1 to I. This represents the total number of pixels in the i-th superpixel block. The size is ; Step 107: Process the v-th modal remote sensing image to obtain the data matrix corresponding to the v-th modal remote sensing image. ,and ;in, This represents the feature data of the first pixel in the v-th modal remote sensing image. This represents the feature data of the nth pixel in the vth modal remote sensing image. Let N represent the feature data of the Nth pixel in the v-th modal remote sensing image, where n ranges from 1 to N, and N represents the total number of pixels in the remote sensing image, and N = W × H. The size is , The size is , T represents transpose.

[0007] The above-mentioned unsupervised multimodal remote sensing image land cover classification method based on dual random graphs further includes step two, which is as follows: Step 201: Set the adaptive projection matrix corresponding to the v-th modal remote sensing image. The affinity matrix between pixels and anchor points corresponding to the v-th modal remote sensing image Construct an adaptive projection map learning model between pixels and anchor points of the v-th modal remote sensing image. Where min represents the minimum value and st represents the constraint condition. The size is , This represents the dimension after dimensionality reduction, and Less than positive integers, The size is N×I, express The affinity vector of the nth pixel. The size is I×1, express The affinity value between the middle and the i-th anchor point. Denotes the regularization parameter, and The value ranges from 10 to 100. Let represent a vector, where all elements of the vector are 1, and The size is I×1, Represents the identity matrix. The size is , Denotes the Gram matrix, and ; Step 202: Optimize the adaptive projection map learning model between pixels and anchor points of the v-th modal remote sensing image using the alternating direction multiplier method to obtain the adaptive projection matrix corresponding to the v-th modal remote sensing image. The affinity matrix between pixels and anchor points corresponding to the v-th modal remote sensing image .

[0008] The above-mentioned unsupervised multimodal remote sensing image land cover classification method based on dual random graphs is further refined in step three, which is as follows: Construct the transformation affinity matrix corresponding to the v-th modal remote sensing image. , The size is N×I, and and according to The double random matrix corresponding to the v-th modal remote sensing image is obtained. ;in, This represents the transformation diagonal matrix corresponding to the v-th modal remote sensing image. Its size is I×I, and The i-th diagonal element is denoted as ,and , The size is N×N.

[0009] The above-mentioned unsupervised multimodal remote sensing image land cover classification method based on dual random graphs, further, step four, the specific process is as follows: Step 401: Set the clustering index matrix The spectral embedding matrix of the v-th modal remote sensing image A multimodal remote sensing image fusion clustering graph model was constructed based on a double random matrix. ①; where max represents the maximum value and st represents the constraint condition. The size is N×C, and the clustering index matrix is... The size is N×C, where C represents the number of land cover categories. Representing the clustering index matrix The element value in the cluster index matrix is ​​1 or 0. The row number in the matrix is ​​the pixel index, and the clustering index matrix is... The number of columns in the index is a categorical index; This represents the weight of the v-th modal remote sensing image; Represents the balance coefficient, and The value range is 10 to 100; Represents the trace of a matrix; Represents the identity matrix. The size is C×C; Step 402: Perform a transformation affinity matrix on the v-th modal remote sensing image. Perform singular value decomposition and arrange the eigenvalues ​​in descending order. Denote the eigenvectors corresponding to the first C eigenvalues ​​as the initial spectral embedding matrix corresponding to the v-th modal remote sensing image. ; Step 403: Input the initial spectral embedding matrix corresponding to the v-th modal remote sensing image. The initial weights of each modal remote sensing image are set to... The optimal clustering index matrix is ​​obtained by optimizing the clustering graph model of multimodal remote sensing image fusion. ; where the initial value of the weight of the v-th modal remote sensing image. for .

[0010] The above-mentioned unsupervised multimodal remote sensing image land cover classification method based on dual random graphs, further, step 403, is as follows: Step 4031, Order and Substituting into the multimodal remote sensing image fusion clustering graph model①, the objective function is transformed into... ②, then obtain Clustering index matrix after the first iteration ;in, If the element in row e′ and column f is the largest, then the clustering index matrix after the first iteration is... The element in the e′-th row and f-th column is 1, otherwise it is zero; e′ takes values ​​from 1 to N, and f takes values ​​from 1 to C; Step 4032, Order and Substituting the data into the multimodal remote sensing image fusion clustering graph model, the objective function is transformed into... ③; Step 4033: Construct the Lagrange function for equation ③ , ;in, Represents the Lagrange multipliers. Let C represent a diagonal matrix of size C×C; Step 4034, Obtain time Let be the spectral embedding matrix of the v-th modal remote sensing image after the first iteration. ; Step 4035, Order and Substitute the data into the multimodal remote sensing image fusion clustering graph model, and convert it into... ④; Step 4036, Order The constant corresponding to the v-th modal remote sensing image Next, transform equation ④ into ⑤; Step 4037: Solve equation ⑤ using the Cauchy-Schwarz inequality to obtain the weights of the v-th modal remote sensing image after the first iteration. ;in, , Let h be a constant corresponding to the h-th modal remote sensing image, where h is a positive integer and its value ranges from 1 to m; Step 4038: Based on the clustering index matrix after the first iteration The spectral embedding matrix of the v-th modal remote sensing image after the first iteration The weights of the v-th modal remote sensing image after the first iteration Following steps 4031 to 4037, a second iteration is performed to obtain the clustering index matrix after the second iteration. The spectral embedding matrix of the v-th modal remote sensing image after the second iteration The weights of the v-th modal remote sensing image after the second iteration ; Step 4039: Repeat step 4038 multiple times for the next iteration to obtain the clustering index matrix after the k-th iteration. The spectral embedding matrix of the v-th modal remote sensing image after the k-th iteration The weights of the v-th modal remote sensing image after the k-th iteration And the clustering index matrix after the (k+1)th iteration The spectral embedding matrix of the v-th modal remote sensing image after the (k+1)-th iteration The weights of the v-th modal remote sensing image after the (k+1)-th iteration Where k is a positive integer greater than 1; Step 40310: The clustering index matrix after the kth iteration... The spectral embedding matrix of the v-th modal remote sensing image after the k-th iteration The weights of the v-th modal remote sensing image after the k-th iteration And the clustering index matrix after the (k+1)th iteration The spectral embedding matrix of the v-th modal remote sensing image after the (k+1)-th iteration The weights of the v-th modal remote sensing image after the (k+1)-th iteration Substituting these values ​​into the objective function of equation ①, we obtain the maximum value of the (k+1)th iteration and the maximum value of the kth iteration. If the difference between the (k+1)th and kth iteration maximum values ​​is less than 0.05, the iteration ends. The clustering index matrix after the (k+1)th iteration is then calculated. This is denoted as the optimal clustering index matrix.

[0011] Compared with the prior art, the present invention has the following advantages: 1. The method of this invention has simple steps and reasonable design. First, the anchor points corresponding to the multimodal remote sensing images are obtained. Second, adaptive projection dimensionality reduction between any modal remote sensing image and the anchor points is used to obtain the adaptive projection matrix and affinity matrix. Next, a double random matrix is ​​obtained based on the adaptive projection matrix and affinity matrix. Then, a multimodal remote sensing image fusion clustering graph model is constructed based on the double random matrix and optimized to obtain the optimal clustering index matrix. Finally, based on the clustering index matrix, the land cover category corresponding to each pixel point of the remote sensing image is obtained, which improves the iteration efficiency and land cover identification accuracy, and has good robustness.

[0012] 2. This invention first performs channel stitching on the data of multimodal remote sensing images to form a high-dimensional remote sensing map, and then performs dimensionality reduction and superpixel segmentation. Multiple superpixel blocks are obtained using the entropy rate superpixel segmentation method, and the anchor point corresponding to each superpixel block in each modal remote sensing image is obtained, which facilitates the subsequent traversal of each modal remote sensing image and each anchor point.

[0013] 3. This invention constructs an adaptive projection map learning model between pixels and anchor points of the v-th modal remote sensing image, traverses all anchor points, obtains the affinity matrix between pixels and anchor points corresponding to the v-th modal remote sensing image, and improves computational efficiency by reducing the dimensionality of the adaptive projection matrix.

[0014] 4. This invention constructs a double random matrix corresponding to the v-th modality remote sensing image based on the transformation affinity matrix corresponding to the v-th modality remote sensing image, thereby improving the clustering effect of subsequent modality weighting and clustering indicator joint optimization, enhancing robustness, and making it suitable for complex multimodal data. Furthermore, it uses... replace This reduces the storage and computational burden on computers.

[0015] 5. This invention constructs a multimodal remote sensing image fusion clustering graph model based on the double random matrix corresponding to each modality of remote sensing image, and iteratively solves to obtain the optimal clustering index matrix, realizing unsupervised learning of ground feature recognition. In addition, it can adaptively weight multimodal information, effectively distinguish the differences in importance of different modal data, which helps to accurately characterize ground features and improve clustering and classification capabilities.

[0016] In summary, the method of this invention is simple in steps and reasonable in design. It obtains a double random matrix by adaptive projection dimensionality reduction of multimodal remote sensing data, and constructs a multimodal remote sensing image fusion clustering graph model based on the double random matrix to obtain the clustering indicator matrix under unsupervised conditions. Thus, it achieves land cover classification based on the clustering indicator matrix, improving iteration efficiency and land cover identification accuracy.

[0017] The technical solution of the present invention will be further described in detail below with reference to the accompanying drawings and embodiments. Attached Figure Description

[0018] Figure 1 This is a flowchart of the method of the present invention.

[0019] Figure 2 This is a remote sensing image of the present invention when m is 2.

[0020] Figure 3 This is a classification result of remote sensing imagery when m is 2 in this invention.

[0021] Figure 4 This is a remote sensing image of m=3 in this invention.

[0022] Figure 5 This is a classification result of remote sensing imagery when m is 3 in this invention. Detailed Implementation

[0023] like Figure 1 As shown, the present invention provides an unsupervised multimodal remote sensing image land cover classification method based on dual random graphs, the method comprising the following steps: Step 1: Acquire multimodal remote sensing images and perform superpixel segmentation to obtain the anchor points corresponding to the multimodal remote sensing images; Step 2: Adaptive projection dimensionality reduction from any modal remote sensing image to the anchor point to obtain the adaptive projection matrix and affinity matrix; Step 3: Obtain a double random matrix based on the adaptive projection matrix and affinity matrix; Step 4: Construct a multimodal remote sensing image fusion clustering graph model based on a double random matrix and optimize the solution to obtain the optimal clustering index matrix; Step 5: Based on the optimal clustering index matrix, obtain the land cover category corresponding to each pixel in the remote sensing image.

[0024] In this embodiment, step one, the specific process is as follows: Step 101: Acquire the 1st modal remote sensing image, ..., the vth modal remote sensing image, ..., the mth modal remote sensing image; where m and v are both positive integers, and 1 ≤ v ≤ m; the size of the vth modal remote sensing image is... W represents the width of the remote sensing image, and H represents the height of the remote sensing image. This represents the feature dimension of the v-th modal remote sensing image; Step 102: Combine the m modal remote sensing images according to the number of channels to obtain a high-dimensional remote sensing image of W×H×D; where, ; Step 103: Convert the high-dimensional remote sensing image of W×H×D into a two-dimensional matrix; where the number of rows in the two-dimensional matrix is ​​W×H and the number of columns in the two-dimensional matrix is ​​D. Step 104: Use principal component analysis to reduce the dimensionality of the two-dimensional matrix and convert the two-dimensional matrix of the first principal component into a grayscale image of the principal components with a size of W×H. Step 105: Use the entropy rate superpixel segmentation method to segment the principal component grayscale image according to the set number of superpixel blocks I, to obtain I superpixel blocks; where I is a positive integer; Step 106, according to The i-th anchor point corresponding to the i-th superpixel block on the v-th modal remote sensing image is obtained. ;in, Let represent the feature data of the e-th pixel in the i-th superpixel block of the v-th modal remote sensing image, where the value of e ranges from 1 to 2. The value of i ranges from 1 to I. This represents the total number of pixels in the i-th superpixel block. The size is ; Step 107: Process the v-th modal remote sensing image to obtain the data matrix corresponding to the v-th modal remote sensing image. ,and ;in, This represents the feature data of the first pixel in the v-th modal remote sensing image. This represents the feature data of the nth pixel in the vth modal remote sensing image. Let N represent the feature data of the Nth pixel in the v-th modal remote sensing image, where n ranges from 1 to N, and N represents the total number of pixels in the remote sensing image, and N = W × H. The size is , The size is , T represents transpose.

[0025] In this embodiment, step two is as follows: Step 201: Set the adaptive projection matrix corresponding to the v-th modal remote sensing image. The affinity matrix between pixels and anchor points corresponding to the v-th modal remote sensing image Construct an adaptive projection map learning model between pixels and anchor points of the v-th modal remote sensing image. Where min represents the minimum value and st represents the constraint condition. The size is , This represents the dimension after dimensionality reduction, and Less than positive integers, The size is N×I, express The affinity vector of the nth pixel. The size is I×1, express The affinity value between the middle and the i-th anchor point. Denotes the regularization parameter, and The value ranges from 10 to 100. Let represent a vector, where all elements of the vector are 1, and The size is I×1, Represents the identity matrix. The size is , Denotes the Gram matrix, and ; Step 202: Optimize the adaptive projection map learning model between pixels and anchor points of the v-th modal remote sensing image using the alternating direction multiplier method to obtain the adaptive projection matrix corresponding to the v-th modal remote sensing image. The affinity matrix between pixels and anchor points corresponding to the v-th modal remote sensing image .

[0026] In this embodiment, step three is as follows: Construct the transformation affinity matrix corresponding to the v-th modal remote sensing image. , The size is N×I, and and according to The double random matrix corresponding to the v-th modal remote sensing image is obtained. ;in, This represents the transformation diagonal matrix corresponding to the v-th modal remote sensing image. Its size is I×I, and The i-th diagonal element is denoted as ,and , The size is N×N.

[0027] In this embodiment, step four is as follows: Step 401: Set the clustering index matrix The spectral embedding matrix of the v-th modal remote sensing image A multimodal remote sensing image fusion clustering graph model was constructed based on a double random matrix. ①; where max represents the maximum value and st represents the constraint condition. The size is N×C, and the clustering index matrix is... The size is N×C, where C represents the number of land cover categories. Representing the clustering index matrix The element value in the cluster index matrix is ​​1 or 0. The row number in the matrix is ​​the pixel index, and the clustering index matrix is... The number of columns in the index is a categorical index; This represents the weight of the v-th modal remote sensing image; Represents the balance coefficient, and The value range is 10 to 100; Represents the trace of a matrix; Represents the identity matrix. The size is C×C; Step 402: Perform a transformation affinity matrix on the v-th modal remote sensing image. Perform singular value decomposition and arrange the eigenvalues ​​in descending order. Denote the eigenvectors corresponding to the first C eigenvalues ​​as the initial spectral embedding matrix corresponding to the v-th modal remote sensing image. ; Step 403: Input the initial spectral embedding matrix corresponding to the v-th modal remote sensing image. The initial weights of each modal remote sensing image are set to... The optimal clustering index matrix is ​​obtained by optimizing the clustering graph model of multimodal remote sensing image fusion. ; where the initial value of the weight of the v-th modal remote sensing image. for .

[0028] In this embodiment, step 403 is specifically performed as follows: Step 4031, Order and Substituting into the multimodal remote sensing image fusion clustering graph model①, the objective function is transformed into... ②, then obtain Clustering index matrix after the first iteration ;in, If the element in row e′ and column f is the largest, then the clustering index matrix after the first iteration is... The element in the e′-th row and f-th column is 1, otherwise it is zero; e′ takes values ​​from 1 to N, and f takes values ​​from 1 to C; Step 4032, Order and Substituting the data into the multimodal remote sensing image fusion clustering graph model, the objective function is transformed into... ③; Step 4033: Construct the Lagrange function for equation ③ , ;in, Represents the Lagrange multipliers. Let C represent a diagonal matrix of size C×C; Step 4034, Obtain time Let be the spectral embedding matrix of the v-th modal remote sensing image after the first iteration. ; Step 4035, Order and Substitute the data into the multimodal remote sensing image fusion clustering graph model, and convert it into... ④; Step 4036, Order The constant corresponding to the v-th modal remote sensing image Next, transform equation ④ into ⑤; Step 4037: Solve equation ⑤ using the Cauchy-Schwarz inequality to obtain the weights of the v-th modal remote sensing image after the first iteration. ;in, , Let h be a constant corresponding to the h-th modal remote sensing image, where h is a positive integer and its value ranges from 1 to m; Step 4038: Based on the clustering index matrix after the first iteration The spectral embedding matrix of the v-th modal remote sensing image after the first iteration The weights of the v-th modal remote sensing image after the first iteration Following steps 4031 to 4037, a second iteration is performed to obtain the clustering index matrix after the second iteration. The spectral embedding matrix of the v-th modal remote sensing image after the second iteration The weights of the v-th modal remote sensing image after the second iteration ; Step 4039: Repeat step 4038 multiple times for the next iteration to obtain the clustering index matrix after the k-th iteration. The spectral embedding matrix of the v-th modal remote sensing image after the k-th iteration The weights of the v-th modal remote sensing image after the k-th iteration And the clustering index matrix after the (k+1)th iteration The spectral embedding matrix of the v-th modal remote sensing image after the (k+1)-th iteration The weights of the v-th modal remote sensing image after the (k+1)-th iteration Where k is a positive integer greater than 1; Step 40310: The clustering index matrix after the kth iteration... The spectral embedding matrix of the v-th modal remote sensing image after the k-th iteration The weights of the v-th modal remote sensing image after the k-th iteration And the clustering index matrix after the (k+1)th iteration The spectral embedding matrix of the v-th modal remote sensing image after the (k+1)-th iteration The weights of the v-th modal remote sensing image after the (k+1)-th iteration Substituting these values ​​into the objective function of equation ①, we obtain the maximum value of the (k+1)th iteration and the maximum value of the kth iteration. If the difference between the (k+1)th and kth iteration maximum values ​​is less than 0.05, the iteration ends. The clustering index matrix after the (k+1)th iteration is then calculated. This is denoted as the optimal clustering index matrix.

[0029] In this embodiment, the value of I is 169.

[0030] In this embodiment, during the solution process in step 202, the affinity matrix between the pixels and anchor points corresponding to the v-th modal remote sensing image is pre-set. The initial value is Substitute into the iterative solution; where, The element value is the Euclidean distance between any pixel and any anchor point.

[0031] In this embodiment, it should be noted that when solving the spectral embedding matrix of the v-th modal remote sensing image after the first iteration in step 4034, the Lagrange multipliers are obtained using orthogonal constraints. Then, the spectral embedding matrix of the v-th modal remote sensing image is obtained after the first iteration. .

[0032] In this embodiment, it should be noted that in the optimal clustering index matrix, the row number of the element with a value of 1 is the pixel number, and the column number of the element with a value of 1 is the land feature category number to which that pixel number belongs.

[0033] In this embodiment, it should be noted that, further, the regularization parameter... Set to 10, balance coefficient The value is 100 and can be adjusted; there is no specific limitation.

[0034] In this embodiment, when m=2, the first modal remote sensing image obtained from the MUUFL Gulfport dataset is a hyperspectral image with 64 bands. The value is 64; the second modal remote sensing image is a lidar image, then... The value is 1. The image size W×H of the MUUFL Gulfport dataset is 325×220.

[0035] like Figure 2 As shown, the left image is the first hyperspectral image. HS The middle image is a LiDAR image. SAR The image on the right is the actual label map of the first land cover classification. GT The MCDLT (Dynamic Multi-Level Graph Fusion Clustering Model), SBGC (Spatial Spectral Feature Consistency Clustering Model), and the method of this invention were used for processing, respectively, to obtain the corresponding results. Figure 3The identification results in (a) to (c) are shown. The processing times for the MCDLT method, SBGC method and the method of the present invention are 14.85 seconds, 16.79 seconds and 12.74 seconds, respectively. The accuracy of the MCDLT method, SBGC method and the method of the present invention relative to the first land cover classification true label map is 42.02%, 57.12% and 70.08%, respectively.

[0036] In this embodiment, when m=3, the first modal remote sensing image obtained from the Augsburg dataset is a hyperspectral image. The value is 64; the second modal remote sensing image is a dual-polarization synthetic aperture radar image, then... The value is 4; the third modal remote sensing image is a digital surface model image, then The value is 1; the digital surface model image is a grayscale image, the dimension of the dual-polarization synthetic aperture radar image feature data is 4, namely the real and imaginary parts of the off-diagonal elements of VV intensity, VH intensity and SAR covariance, and the image size W×H of the Augsburg dataset is 332×485.

[0037] like Figure 4 As shown, the second hyperspectral image is displayed from left to right. HS Dual-polarization synthetic aperture radar image SAR Digital surface model image DSM Second land cover classification real label map GT The FPFC (Projected Fuzzy Clustering Model), SBGC (Spatial Spectral Consistency Clustering Model), and the method of this invention were respectively used for processing to obtain... Figure 5 The identification results are shown in (a) to (c). The processing times for the FPFC method, SBGC method, and the method of this invention are 28.06 seconds, 28.36 seconds, and 24.32 seconds, respectively; and the accuracies of the FPFC method, SBGC method, and the method of this invention relative to the second land cover classification true label map are 81.58%, 76.29%, and 84.71%, respectively.

[0038] In this embodiment, the entropy rate superpixel segmentation method is used to segment and obtain superpixel blocks. This method groups spatially related pixels into coherent superpixels in an unsupervised manner based on the local correlation of pixels, ensuring that most pixels within the same superpixel block belong to the same land cover category. Therefore, a superpixel block is a locally homogeneous region, in which most pixels usually belong to the same land cover category, and thus the anchor points generated by different superpixel blocks exhibit significant discriminative power.

[0039] In summary, the method of this invention is simple in steps and reasonable in design. It obtains a double random matrix by adaptive projection dimensionality reduction of multimodal remote sensing data, and constructs a multimodal remote sensing image fusion clustering graph model based on the double random matrix to obtain the clustering indicator matrix under unsupervised conditions. Thus, it achieves land cover classification based on the clustering indicator matrix, improving iteration efficiency and land cover identification accuracy.

[0040] The above description is merely a preferred embodiment of the present invention and does not constitute any limitation on the present invention. Any simple modifications, alterations, or equivalent structural changes made to the above embodiments based on the technical essence of the present invention shall still fall within the protection scope of the present invention.

Claims

1. A method for unsupervised multimodal remote sensing image land cover classification based on dual random graphs, characterized in that, The method includes the following steps: Step 1: Acquire multimodal remote sensing images and perform superpixel segmentation to obtain the anchor points corresponding to the multimodal remote sensing images; Step 2: Adaptive projection dimensionality reduction from any modal remote sensing image to the anchor point to obtain the adaptive projection matrix and affinity matrix; Step 3: Obtain a double random matrix based on the adaptive projection matrix and affinity matrix; Step 4: Construct a multimodal remote sensing image fusion clustering graph model based on a double random matrix and optimize the solution to obtain the optimal clustering index matrix; Step 5: Based on the optimal clustering index matrix, obtain the land cover category corresponding to each pixel in the remote sensing image; Step three, the specific process is as follows: Construct the transformation affinity matrix corresponding to the v-th modal remote sensing image. , The size is N×I, and and according to The double random matrix corresponding to the v-th modal remote sensing image is obtained. ;in, This represents the transformation diagonal matrix corresponding to the v-th modal remote sensing image. Its size is I×I, and The i-th diagonal element is denoted as ,and , The size is N×N; This represents the affinity matrix between pixels and anchor points corresponding to the v-th modal remote sensing image. The size is N×I, express The affinity vector of the nth pixel. The size is I×1, express The affinity value between the i-th anchor point and the n-th anchor point, where n ranges from 1 to N, i ranges from 1 to I, N represents the total number of pixels in the remote sensing image, and I represents the number of superpixel blocks set; 1 ≤ v ≤ m, and m and v are both positive integers. Step four, the specific process is as follows: Step 401: Set the clustering index matrix The spectral embedding matrix of the v-th modal remote sensing image A multimodal remote sensing image fusion clustering graph model was constructed based on a double random matrix. ①; where max represents the maximum value and st represents the constraint condition. The size is N×C, and the clustering index matrix is... The size is N×C, where C represents the number of land cover categories. Represents the clustering index matrix The element value in the cluster index matrix is ​​1 or 0. The row number in the matrix is ​​the pixel index, and the clustering index matrix is... The number of columns in the index is a categorical index; This represents the weight of the v-th modal remote sensing image; Represents the balance coefficient, and The value range is 10 to 100; Represents the trace of a matrix; Represents the identity matrix. The size is C×C; Step 402: Perform a transformation affinity matrix on the v-th modal remote sensing image. Perform singular value decomposition and arrange the eigenvalues ​​in descending order. Denote the eigenvectors corresponding to the first C eigenvalues ​​as the initial spectral embedding matrix corresponding to the v-th modal remote sensing image. ; Step 403: Input the initial spectral embedding matrix corresponding to the v-th modal remote sensing image. The initial weights of each modal remote sensing image are set to... The optimal clustering index matrix is ​​obtained by optimizing the clustering graph model of multimodal remote sensing image fusion. ; where the initial value of the weight of the v-th modal remote sensing image. for .

2. The unsupervised multimodal remote sensing image land cover classification method based on dual random graphs according to claim 1, characterized in that: Step one, the specific process is as follows: Step 101: Acquire the first modal remote sensing image, ..., the vth modal remote sensing image, ..., the mth modal remote sensing image; wherein, the size of the vth modal remote sensing image is... W represents the width of the remote sensing image, and H represents the height of the remote sensing image. This represents the feature dimension of the v-th modal remote sensing image; Step 102: Combine the m modal remote sensing images according to the number of channels to obtain a high-dimensional remote sensing image of W×H×D; where, ; Step 103: Convert the high-dimensional remote sensing image of W×H×D into a two-dimensional matrix; where the number of rows in the two-dimensional matrix is ​​W×H and the number of columns in the two-dimensional matrix is ​​D. Step 104: Use principal component analysis to reduce the dimensionality of the two-dimensional matrix and convert the two-dimensional matrix of the first principal component into a grayscale image of the principal components with a size of W×H. Step 105: Use the entropy rate superpixel segmentation method to segment the principal component grayscale image according to the set number of superpixel blocks I, to obtain I superpixel blocks; where I is a positive integer; Step 106, according to The i-th anchor point corresponding to the i-th superpixel block on the v-th modal remote sensing image is obtained. ;in, Let represent the feature data of the e-th pixel in the i-th superpixel block of the v-th modal remote sensing image, where the value of e ranges from 1 to 2. , This represents the total number of pixels in the i-th superpixel block. The size is ; Step 107: Process the v-th modal remote sensing image to obtain the data matrix corresponding to the v-th modal remote sensing image. ,and ;in, This represents the feature data of the first pixel in the v-th modal remote sensing image. This represents the feature data of the nth pixel in the vth modal remote sensing image. Let N represent the feature data of the Nth pixel in the vth modal remote sensing image, where N = W × H. The size is , The size is , T represents transpose.

3. The unsupervised multimodal remote sensing image land cover classification method based on dual random graphs according to claim 2, characterized in that: Step two, the specific process is as follows: Step 201: Set the adaptive projection matrix corresponding to the v-th modal remote sensing image. The affinity matrix between pixels and anchor points corresponding to the v-th modal remote sensing image Construct an adaptive projection map learning model between pixels and anchor points of the v-th modal remote sensing image. Where min represents the minimum value and st represents the constraint condition. The size is , Indicates the dimension after dimensionality reduction, and Less than positive integers, This represents the regularization parameter, and The value ranges from 10 to 100. Let represent a vector, where all elements of the vector are 1, and The size is I×1, Represents the identity matrix. The size is , Denotes the Gram matrix, and ; Step 202: Optimize the adaptive projection map learning model between pixels and anchor points of the v-th modal remote sensing image using the alternating direction multiplier method to obtain the adaptive projection matrix corresponding to the v-th modal remote sensing image. The affinity matrix between pixels and anchor points corresponding to the v-th modal remote sensing image .

4. The unsupervised multimodal remote sensing image land cover classification method based on dual random graphs according to claim 1, characterized in that: Step 403, the specific process is as follows: Step 4031, Order and Substituting into the multimodal remote sensing image fusion clustering graph model①, the objective function is transformed into... ②, then obtain Clustering index matrix after the first iteration ;in, If the element in row e′ and column f is the largest, then the clustering index matrix after the first iteration is... The element in the e′-th row and f-th column is 1, otherwise it is zero; e′ takes values ​​from 1 to N, and f takes values ​​from 1 to C; Step 4032, Order and Substituting the data into the multimodal remote sensing image fusion clustering graph model, the objective function is transformed into... ③; Step 4033: Construct the Lagrange function for equation ③ , ;in, Represents the Lagrange multipliers. Let C represent a diagonal matrix of size C×C; Step 4034, Obtain time Let be the spectral embedding matrix of the v-th modal remote sensing image after the first iteration. ; Step 4035, Order and Substitute the data into the multimodal remote sensing image fusion clustering graph model, and convert it into... ④; Step 4036, Order The constant corresponding to the v-th modal remote sensing image Next, transform equation ④ into ⑤; Step 4037: Solve equation ⑤ using the Cauchy-Schwarz inequality to obtain the weights of the v-th modal remote sensing image after the first iteration. ;in, , Let h be a constant corresponding to the h-th modal remote sensing image, where h is a positive integer and its value ranges from 1 to m; Step 4038: Based on the clustering index matrix after the first iteration The spectral embedding matrix of the v-th modal remote sensing image after the first iteration The weights of the v-th modal remote sensing image after the first iteration Following steps 4031 to 4037, a second iteration is performed to obtain the clustering index matrix after the second iteration. The spectral embedding matrix of the v-th modal remote sensing image after the second iteration The weights of the v-th modal remote sensing image after the second iteration ; Step 4039: Repeat step 4038 multiple times for the next iteration to obtain the clustering index matrix after the k-th iteration. The spectral embedding matrix of the v-th modal remote sensing image after the k-th iteration The weights of the v-th modal remote sensing image after the k-th iteration And the clustering index matrix after the (k+1)th iteration The spectral embedding matrix of the v-th modal remote sensing image after the (k+1)-th iteration The weights of the v-th modal remote sensing image after the (k+1)-th iteration Where k is a positive integer greater than 1; Step 40310: The clustering index matrix after the kth iteration... The spectral embedding matrix of the v-th modal remote sensing image after the k-th iteration The weights of the v-th modal remote sensing image after the k-th iteration And the clustering index matrix after the (k+1)th iteration The spectral embedding matrix of the v-th modal remote sensing image after the (k+1)-th iteration The weights of the v-th modal remote sensing image after the (k+1)-th iteration Substituting these values ​​into the objective function of equation ①, we obtain the maximum value of the (k+1)th iteration and the maximum value of the kth iteration. If the difference between the (k+1)th and kth iteration maximum values ​​is less than 0.05, the iteration ends. The clustering index matrix after the (k+1)th iteration is then calculated. This is denoted as the optimal clustering index matrix.

Images