A method for classifying ground objects in spectral images by combining spatial-spectral correlation and mixed-order similarity

Abstract

A method for classifying ground objects in spectral images by combining spatial-spectral correlation and mixed-order similarity belongs to the field of remote sensing image processing and classification. First, the first principal component extracted from the hyperspectral data set is segmented through entropy rate superpixel analysis, and the d-dimensional latent features extracted from each region are stacked to obtain a latent feature matrix. Then, the total objective function is obtained through learning the inter-band correlation coefficient matrix, learning the inter-band similarity matrix of hyperspectral bands, and learning the spatial-spectral joint correlation, and the coefficient matrix and the inter-band similarity matrix are solved until the value of the total objective function converges. Finally, spectral clustering is performed according to the obtained inter-band similarity matrix, the bands with the largest information entropy are selected to form a subset of bands, the k-nearest neighbor algorithm is used to classify the obtained subset of bands to obtain a classification result, and the classification accuracy is calculated; compared with other methods, the present invention improves the classification accuracy and has more robust performance.

Description

Technical Field

[0001] The present invention belongs to the field of remote sensing image processing and classification, and particularly relates to a method for classifying ground objects in spectral images by combining spatial-spectral correlation and mixed-order similarity. Background Art

[0002] In the field of remote sensing image processing and classification, hyperspectral image data is a widely available data. Performing dimensionality reduction processing and classification analysis on hyperspectral data can reduce the computational time complexity, remove noise interference in the dataset, and mine potential information such as the proportion of ground object coverage and geomorphic features. However, due to the large dimension of the original spectral image and the high information redundancy of adjacent bands, it leads to a huge amount of calculation and susceptibility to noise in the classification process. Some existing classification methods directly use the original spectral image or only use simple dimensionality reduction methods, without fully utilizing the spatial texture information of the original spectral image and the information between each band. The selected bands also cannot achieve an ideal classification effect, which poses a huge challenge to the problem of classifying hyperspectral image data. Therefore, effective and novel technical methods are needed to solve the classification problem of hyperspectral image data. Summary of the Invention

[0003] To solve the above problems, the present invention provides a method for classifying ground objects in spectral images by combining spatial-spectral correlation and mixed-order similarity. The method includes the steps of:

[0004] Extract the first principal component from the hyperspectral image, divide the first principal component into multiple regions, extract the potential features of each region, and stack the potential features of each region for each band to obtain a potential feature matrix.

[0005] Learn the inter-band correlation coefficient matrix through the obtained potential feature matrix to obtain a coefficient matrix, and map the potential feature matrix into the Gaussian kernel space, polynomial kernel space, and inverse polynomial kernel space respectively. Superimpose the obtained different similarity matrices to obtain a new similarity matrix.

[0006] Calculate the mixed-order similarity matrix through the new similarity matrix, perform learning on the inter-band similarity matrix of the hyperspectral bands to obtain an inter-band similarity matrix, and perform spatial-spectral joint correlation learning on the coefficient matrix and the inter-band similarity matrix to update the inter-band similarity matrix.

[0007] Obtain the total objective function through inter-band correlation coefficient matrix learning, inter-band similarity matrix learning of hyperspectral bands, and spatial-spectral joint correlation learning, and iteratively solve the obtained total objective function until the value of the total objective function converges.

[0008] Perform spectral clustering on the inter-band similarity matrix to obtain the band clustering results, calculate the information entropy of each band, select the band with the largest information entropy in each cluster to form a subset of wavebands, and classify the obtained subset of wavebands through the k-nearest neighbor algorithm to obtain the classification results.

[0009] According to the obtained classification results, calculate the classification accuracy on the spectral image dataset.

[0010] Furthermore, the expression for dividing the first principal component into N regions is:

[0011]

[0012] where H represents the spectral image data, and r i represents the i-th region after segmentation.

[0013] The expression for each region to learn the regional latent feature is:

[0014]

[0015] where Y (i) represents the latent feature matrix of the i-th region, I is the d-dimensional identity matrix, and L (i) represents the symmetric normalized Laplacian matrix of the i-th region's latent feature. The L (i) is defined as follows:

[0016]

[0017] where W (i) is the similarity matrix constructed by the k-nearest neighbor graph and Euclidean distance for the latent feature of the i-th region, and D (i) is the degree matrix of W (i) The original spectral data can be re-expressed as Stack the d-dimensional latent features of each region of each band to obtain the latent feature matrix

[0018] Furthermore, the expression for learning the inter-band correlation coefficient matrix is:

[0019]

[0020] where ‖·‖ F is the F-norm, and ‖·‖ * is the nuclear norm.

[0021] The three different similarity matrices R1, R2, and R3 obtained by mapping the latent feature matrix into the Gaussian kernel space, polynomial kernel space, and inverse polynomial kernel space respectively are expressed as follows:

[0022]

[0023] R2 = K2(f i , f j ) = (f i T f j + c) 2

[0024]

[0025] where ||·||2 is the 2-norm, f i and f j represent the i-th row and j-th row elements of F, respectively.

[0026] The expression formula for superimposing the three different similarity matrices is:

[0027]

[0028] where R n is the n-th similarity matrix, and the similarity matrix R s is the result of superimposing R1, R2, and R3.

[0029] Furthermore, the expression formula for the mixed-order similarity matrix R is:

[0030]

[0031] where and represent the p-th order and (p - 1)-th order matrices of R s , respectively.

[0032] The expression formula for learning the similarity matrix between hyperspectral bands is:

[0033]

[0034] where ‖·‖ F is the F-norm, and S is the similarity matrix between bands.

[0035] The expression formula for joint spatial-spectral correlation learning is:

[0036]

[0037] where z i and z j represent the i-th column and j-th column of the coefficient matrix Z, respectively, and S ij represents the element in the i-th row and j-th column of S.

[0038] Furthermore, the expression formula for the total objective function is:

[0039]

[0040] The augmented Lagrangian method is used, and the expression formula for introducing the auxiliary variable L is as follows:

[0041]

[0042] The expression formulas for the coefficient matrix Z and the auxiliary variable L are as follows:

[0043]

[0044] where Y1 is the Lagrange multiplier, μ>0 is the penalty parameter, I is the identity matrix, is the transpose of the Laplacian matrix of matrix L.

[0045] The expression formula for the coefficient matrix after being solved by the Sylvester equation is as follows:

[0046]

[0047] where the expression of Y1 is Y1 = Y1 + μ(Z - L), and the expression of μ is μ = min(ρμ, μ max ), ρ>1 is the iteration step size, and μ max represents the maximum value of μ.

[0048] The solution for the S value in the total objective function is as follows:

[0049]

[0050] where R ij represents the element in the i-th row and j-th column of R. According to the above alternating solution method, the total objective function is iteratively solved until the value of the total objective function converges.

[0051] Furthermore, spectral clustering is performed on the band similarity matrix S based on the solution results to obtain the band clustering results, and the information entropy of each band is calculated. Then, according to the band clustering results and the information entropy, the band with the largest information entropy in each cluster is selected to form a subset of bands, and the k-nearest neighbor algorithm is used for classification through the subset of bands to obtain the classification results.

[0052] Furthermore, based on the obtained classification results, the classification accuracy rate on the spectral image dataset is calculated.

[0053] The present invention provides a method for classifying ground objects in spectral images by combining spatial-spectral correlation and mixed-order similarity, which has the following advantages:

[0054] (1) The method uses principal component analysis and entropy rate superpixel segmentation to divide the hyperspectral data into regions, and extracts the potential features of each region. The obtained potential features can make more full use of the spatial information of the hyperspectral image and reduce the noise interference.

[0055] (2) The method adopts the method of mapping the similarity matrix between bands of the hyperspectral image to the Gaussian kernel space, polynomial kernel space and inverse polynomial kernel space. The obtained different similarity matrices can better capture the local patterns and subtle features of the data, and more flexibly express different feature combinations and interactions, so as to fully consider the global feature trend and more effectively reduce the influence of outliers and noise.

[0056] (3) The method adopts a mixed-order similarity calculation method. The obtained mixed-order similarity matrix can more fully consider the spectral information between bands of the hyperspectral image and more effectively reduce the noise interference at the same time.

[0057] (4) The method adopts joint spatial-spectral correlation learning, which can more effectively fuse the spatial and spectral information of the hyperspectral image data. The bands in the finally obtained subset of wavebands are more representative. BRIEF DESCRIPTION OF THE DRAWINGS

[0058] In order to more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the following will briefly introduce the drawings required for use in the description of the embodiments or the prior art. Obviously, the following drawings are only some embodiments of the present invention. For those of ordinary skill in the art, other drawings can be obtained based on these drawings without creative efforts.

[0059] Figure 1 is a flowchart of the method for classifying ground objects in spectral images by combining joint spatial-spectral correlation and mixed-order similarity provided by the present invention;

[0060] Figure 2 is a schematic diagram of the Indian Pines dataset with a size of 145 * 145 pixels and containing 200 effective bands. DETAILED DESCRIPTION OF THE EMBODIMENTS

[0061] To make the purpose, technical solutions and advantages of the present invention clearer, the present invention will be further described in detail below in conjunction with the specific embodiments and with reference to the accompanying drawings. It should be understood that these descriptions are exemplary and not intended to limit the scope of the present invention. In addition, in the following description, the descriptions of well-known structures and technologies are omitted to avoid unnecessarily confusing the concepts of the present invention.

[0062] Exemplary Method

[0063] Such as Figure 1, the present invention provides a method for classifying ground objects in spectral images by combining spatial-spectral correlation and mixed-order similarity. The steps of the method are as follows:

[0064] Step S110: Obtain the first principal component of the spectral image data through principal component analysis, and segment the obtained first principal component into N regions by entropy rate superpixel segmentation. The expression formula is as follows:

[0065]

[0066] where H represents the spectral image data, and r i represents the i-th region after segmentation.

[0067] Learn the regional latent features of each region through spectral embedding learning. The expression formula is as follows:

[0068]

[0069] where Y (i) represents the latent feature matrix of the i-th region, I is the d-dimensional identity matrix, and L (i) represents the symmetric normalized Laplacian matrix of the latent features of the i-th region.

[0070] The L (i) The expression formula is as follows:

[0071]

[0072] where W (i) is the similarity matrix constructed by the latent features of the i-th region through the k-nearest neighbor graph and Euclidean distance, and D (i) is the degree matrix of W (i) ; the original spectral data can be re-expressed as Stack the d-dimensional latent features of each region in each band to obtain the latent feature matrix

[0073] Step S120: Learn the coefficient matrix Z from the obtained latent feature matrix F through the inter-band correlation coefficient matrix. The expression formula is as follows:

[0074]

[0075] where ‖·‖ F is the F norm, and ‖·‖ * is the nuclear norm. The nuclear norm is used to constrain Z so that Z has a low-rank structure.

[0076] By mapping the potential feature matrix F into the Gaussian kernel space, polynomial kernel space, and inverse polynomial kernel space respectively, three different similarity matrices R1, R2, and R3 are obtained, and their expression formulas are as follows:

[0077]

[0078] R2 = K2(f i , f j ) = (f i T f j + c) 2

[0079]

[0080] where ||·||2 is the 2-norm, and f i and f j represent the elements of the i-th row and j-th row of F respectively.

[0081] The similarity matrix R s is obtained by superimposing the three similarity matrices, and its expression formula is as follows:

[0082]

[0083] where R n is the n-th similarity matrix, and the similarity matrix R s is the result of superimposing R1, R2, and R3.

[0084] Step S130: Calculate its mixed-order similarity matrix R through the similarity matrix R s , and its expression formula is as follows:

[0085]

[0086] where and represent the p-th order and p - 1-th order matrices of R s respectively.

[0087] Based on the mixed-order similarity matrix R, perform hyperspectral inter-band similarity matrix learning, and its expression formula is as follows:

[0088]

[0089] where ‖·‖ F is the F-norm, and S is the inter-band similarity matrix.

[0090] Learn the coefficient matrix Z and the inter-band similarity matrix S through spatial-spectral joint correlation learning, and its expression formula is as follows:

[0091]

[0092] wherein, z i and z j represent the i-th column and the j-th column of the coefficient matrix Z respectively, and S ij represents the element at the i-th row and the j-th column of S.

[0093] Step S140: Obtain the total objective function through inter-band correlation coefficient matrix learning, hyperspectral inter-band similarity matrix learning, and spatial-spectral joint correlation learning. The expression formula is as follows:

[0094]

[0095] Iteratively solve the coefficient matrix Z for the total objective function by the alternating solution method and the gradient descent method. Use the augmented Lagrangian method and introduce the auxiliary variable L. The expression formula is as follows:

[0096]

[0097] Adopt the alternating solution strategy for the formula obtained after using the Lagrangian method to obtain the expressions for Z and L. The expression formula is as follows:

[0098]

[0099] wherein, Y1 is the Lagrange multiplier, μ > 0 is the penalty parameter, I is the identity matrix, is the transpose of the Laplacian matrix of matrix L.

[0100] Solve for Z through the Sylvester equation to obtain the expression for Z. The expression formula is as follows:

[0101]

[0102] where the expression for Y1 is Y1 = Y1 + μ(Z - L), and the expression for μ is μ = min(ρμ, μ max ), ρ > 1 is the iteration step size, and μ max represents the maximum value of μ.

[0103] Solve for the value of S in the total objective function by using the alternating solution method. The expression formula is as follows:

[0104]

[0105] where, R ij represents the element at the i-th row and the j-th column of R.

[0106] Iteratively solve the total objective function by the above alternating solution method until the value of the total objective function converges.

[0107] Step S150: Obtain the band clustering result through spectral clustering, calculate the information entropy of each band, select the band with the maximum information entropy in each cluster through the band clustering result and information entropy to form a subset of wavebands, and classify the subset of wavebands through the k-nearest neighbor algorithm to obtain the classification result.

[0108] Step S160: Calculate the classification accuracy rate on the hyperspectral dataset according to the obtained classification result.

[0109] Through this embodiment, first divide the hyperspectral dataset into N regions, then extract potential features to obtain a potential feature matrix, obtain the total objective function through inter-band correlation coefficient matrix learning, hyperspectral inter-band similarity matrix learning, and spatial-spectral joint correlation learning, then solve the coefficient matrix and the inter-band similarity matrix according to the total objective function, then perform spectral clustering and construct a subset of wavebands according to the information entropy of each band, and finally use the k-nearest neighbor algorithm to classify according to the subset of wavebands to obtain the final classification result. After obtaining the classification result, calculate the classification accuracy rate on the dataset.

[0110] Further explanation, assuming that a hyperspectral image dataset is classified according to this embodiment, a classification result with an accuracy rate higher than most methods will be obtained.

[0111] Specific embodiment results

[0112] This embodiment uses the publicly available Indian Pines dataset and uses 200 effective bands therein. The details of the dataset are described as follows:

[0113] The Indian Pines dataset covers farmland, a small amount of forest, buildings and other ground objects in the northwest of Indiana. This dataset contains the following two files:

[0114] The hyperspectral image data file is a three-dimensional data matrix with dimensions of 145 * 145 * 220. 145 * 145 represents the spatial size of the entire scene, that is, it contains 21025 pixel points; the spectral vector corresponding to each pixel contains the reflectance values of 220 bands, and 200 effective bands are retained; each pixel corresponds to a ground resolution of approximately 20 meters * 20 meters.

[0115] The ground truth label file is a two-dimensional matrix of 145 * 145. Each label pixel corresponds one-to-one with the corresponding pixel position in the hyperspectral data; each element in the matrix takes an integer between 0-16, representing the ground object category of an element.

[0116] To verify the superiority of this embodiment (Ours), this embodiment is compared with several existing spectral image land cover classification methods, including methods such as TOF, OPBS, ASPS_MN, and ONR. The values of three metrics, namely OA, AA, and Kappa, will be compared when these methods use three classifiers, KNN, RF, and SVM, on the IndianPines public dataset. The specific data comparisons are shown in Table 1.

[0117] Table 1 Classification accuracy of Indian Pines dataset (%)

[0118]

[0119] From the data comparisons in the above table, it can be clearly seen that Ours achieves the best performance and significantly improves the classification accuracy of spectral image land cover. The quantitative results fully illustrate the superiority of Ours because Ours can make more full use of spatial information and spectral information and better fuse the two, thus obtaining a subset of wavebands containing more comprehensive information, and then achieving an ideal classification effect. Ours performs dimensionality reduction on the hyperspectral image to obtain a more ideal subset of wavebands, which can reduce the computational time complexity and remove the noise interference in the dataset at the same time, thereby improving the classification accuracy of the hyperspectral data image. A large number of experiments show that this method is superior to existing methods. Regarding the parameter settings of this embodiment, in all experiments, the ranges of parameters α and β are uniformly tuned between 1e-8 and 1e4, the optimal order of the mixed-order similarity is adjusted between 3 and 8 orders, the potential feature dimension is set to 5, and the number of neighbors is set to 15.

[0120] This embodiment proposes a spectral image land cover classification method that combines spatial-spectral correlation and mixed-order similarity for land cover classification analysis of common hyperspectral images in life. The potential feature matrix of the hyperspectral image is obtained by stacking d-dimensional potential features, and then the total objective function is obtained through inter-band correlation coefficient matrix learning, hyperspectral inter-band similarity matrix learning, and spatial-spectral joint correlation learning. The coefficient matrix and the inter-band similarity matrix are iteratively solved for the total objective function, then spectral clustering is performed and the information entropy is calculated to obtain a subset of wavebands, and finally the final classification result is obtained through the k-nearest neighbor algorithm. The experimental results using three classifiers on the public dataset Indian Pines show that this embodiment has a higher classification accuracy and better superiority compared with other methods.

[0121] It should be understood that the above specific embodiments of the present invention are only used for exemplary illustration or explanation of the principles of the present invention, and do not constitute a limitation on the present invention. Therefore, any modifications, equivalent replacements, improvements, etc. made without departing from the spirit and scope of the present invention shall be included within the protection scope of the present invention. In addition, the appended claims of the present invention are intended to cover all variations and modifications that fall within the scope and boundaries of the appended claims, or equivalent forms of such scope and boundaries.

Claims

1. A spectral image object classification method combining spatial-spectral correlation and mixed-order similarity, characterized in that: The method comprises the steps of: A principal component analysis is performed on a hyperspectral data set to extract its first principal component, and the obtained first principal component is segmented by entropy rate superpixel to segment the spectral image into N regions; for each region, its d-dimensional potential features are extracted; The d-dimensional potential features of each region in each band are stacked to obtain the potential feature matrix F of the hyperspectral image data; According to the obtained potential feature matrix F, the inter-band correlation coefficient matrix is ​​learned to obtain the coefficient matrix Z; F is mapped to the Gaussian kernel space, polynomial kernel space and inverse polynomial kernel space respectively to obtain three different similarity matrices R1, R2, and R3, which are superimposed to obtain the similarity matrix R s ; According to the similarity matrix R s Calculate its mixed order similarity matrix R; perform hyperspectral inter-band similarity matrix learning based on the mixed order similarity matrix R to obtain the inter-band similarity matrix S; perform spatial-spectral joint correlation learning based on the coefficient matrix Z and the inter-band similarity matrix S to update the inter-band similarity matrix S; The total objective function is obtained according to the inter-band correlation coefficient matrix learning, the inter-band similarity matrix learning of the hyperspectral spectrum and the spatial-spectral joint correlation learning; the coefficient matrix Z and the inter-band similarity matrix S are iteratively solved for the obtained total objective function using the alternating solution method and the gradient descent method until the total objective function value converges; According to the solution results, the similarity matrix S between bands is spectrally clustered to obtain the band clustering results, the information entropy of each band is calculated, and the band with the largest information entropy in each cluster is selected to form a band subset; Classification is performed using the k nearest neighbor algorithm according to the band subset to obtain the classification result; According to the obtained classification results, the classification accuracy rate on the spectral image dataset is calculated.

2. The spectral image object classification method based on joint spatial-spectral correlation and mixed-order similarity according to claim 1 is characterized in that: The principal component analysis is performed on a spectral image data to obtain the first principal component of the spectral data; the entropy rate superpixel segmentation is performed on the obtained first principal component to divide the spectral image into N regions, as shown below: Where H represents the spectral image data, r i represents the i-th region after segmentation; for each region, its regional potential features are learned by spectral embedding, and the learned regional potential features are as follows: Among them, Y (i) represents the potential feature matrix of the i-th region, I is the d-dimensional unit matrix, L (i) represents the symmetric normalized Laplacian matrix of the potential features of the i-th region, L (i) The specific definitions are as follows: Among them, W (i) is the similarity matrix constructed by the k nearest neighbor graph and Euclidean distance of the potential features of the i-th region, D (i) W (i) The original spectral data can be re-expressed as H∈R b*N*d , stack the d-dimensional potential features of each region of each band to obtain the potential feature matrix F∈R b*Nd .

3. The spectral image object classification method based on joint spatial-spectral correlation and mixed-order similarity according to claim 1 is characterized in that: According to the obtained potential feature matrix F, the inter-band correlation coefficient matrix is ​​learned to obtain the coefficient matrix Z. The inter-band correlation coefficient matrix learning is defined as follows: in,‖·‖ F is the F-norm, ‖·‖ * is the nuclear norm, and Z is constrained by the nuclear norm so that Z has a low-rank structure; then, F is mapped to the Gaussian kernel space, the polynomial kernel space, and the inverse polynomial kernel space, respectively, to obtain three different similarity matrices R1, R2, and R3, which are defined as follows: R2=K2(f i ,f j )=(f i T f j +c) 2 Among them, ‖·‖2 is the 2-norm, f i and f j Respectively represent the i-th and j-th row elements of F; after obtaining R1, R2, and R3, the three similarity matrices are superimposed as shown below: Among them, R n is the nth similarity matrix, similarity matrix R s It is the result of superposition of R1, R2 and R3.

4. The spectral image object classification method based on joint spatial-spectral correlation and mixed-order similarity according to claim 1 is characterized in that: According to the similarity matrix R s Calculate its mixed-order similarity matrix R as follows: in, and Respectively represent R s The p-order and p-1-order matrices are used; according to the mixed-order similarity matrix R, the similarity matrix between hyperspectral bands is learned, which is defined as follows: in,‖·‖ F is the F norm, S is the inter-band similarity matrix; the spatial-spectral joint correlation learning is performed according to the coefficient matrix Z and the inter-band similarity matrix S, as shown below: Among them, z i and z j denote the i-th and j-th columns of the coefficient matrix Z, respectively, S ij Represents the element in the i-th row and j-th column of S.

5. The spectral image object classification method combining spatial-spectral correlation and mixed-order similarity according to claim 1 is characterized in that: The total objective function is obtained based on the inter-band correlation coefficient matrix learning, the hyperspectral inter-band similarity matrix learning, and the spatial-spectral joint correlation learning. The total objective function is defined as follows: The coefficient matrix Z of the total objective function is iteratively solved by the alternating solution method and the gradient descent method. The augmented Lagrangian method is used to introduce the auxiliary variable L, as shown below: Next, we adopt the alternating solution strategy to obtain the expressions for Z and L as follows: Where Y1 is the Lagrange multiplier, μ>0 is the penalty parameter, I is the identity matrix, is the transpose of the Laplace matrix of matrix L; then, by solving Z through the Sylvester equation, we can get the expression of Z as follows: The expression of Y1 is Y1=Y1+μ(ZL), and the expression of μ is μ=min(ρμ,μ max ), ρ>1 is the iteration step, μ max represents the maximum value of μ; The alternating solution method is used at the same time to solve the S value in the total objective function, as shown below: Among them, R ij Represents the i-th row and j-th column element of R; according to the above alternating solution method, the total objective function is iteratively solved until the total objective function value converges.

6. The spectral image object classification method combining spatial-spectral correlation and mixed-order similarity according to claim 1 is characterized in that: According to the solution results, the similarity matrix S between bands is spectrally clustered to obtain the band clustering results; the information entropy of each band is calculated; according to the band clustering results and the information entropy, the band with the largest information entropy in each cluster is selected to form a band subset; The k nearest neighbor algorithm is used to classify the band subsets to obtain the classification results.

7. The spectral image object classification method combining spatial-spectral correlation and mixed-order similarity according to claim 1 is characterized in that: According to the obtained classification results, the classification accuracy rate on the spectral image dataset is calculated.

Images