Joint sparse representation hyperspectral image classification method based on dual neighborhood constraints

Abstract

The invention relates to the technical field of remote sensing image processing, in particular to a joint sparse representation hyperspectral image classification method based on dual domain constraints, which comprises the following steps: preprocessing: carrying out spectral feature-based wave band grouping on hyperspectral image data, and then carrying out MNF data dimension reduction on the grouped hyperspectral image data; extracting a main component feature map by using morphology; performing superpixel segmentation on the hyperspectral image by using an improved watershed algorithm, and performing FCM clustering on the hyperspectral image; adaptive selection of the neighborhood is carried out through weight calculation under double constraints of the obtained superpixel neighborhood and the clustering field; multi-view angles of the superpixel field, the clustering field and the constraint field are used for joint sparse representation; and a majority voting method is adopted to integrate classification results, and the classification results are adjusted through a correction rule, so that the classification effect of the hyperspectral image is improved, and the classification precision of edge pixels is improved.

Description

Technical Field

[0001] The present invention relates to the technical field of remote sensing image processing, and in particular to a joint sparse representation hyperspectral image classification method based on dual neighborhood constraints. Background Art

[0002] Hyperspectral images (HSI) continuously capture detailed spectral information of target objects during the imaging process. As a result, each pixel contains a large amount of spectral data, providing more comprehensive spectral dimension information than traditional multispectral images. This characteristic gives hyperspectral images significant advantages in fields such as land use classification, crop identification, mineral resource exploration, environmental monitoring, and military reconnaissance. However, due to the high dimensionality and complexity of the data, how to efficiently and accurately extract useful information from it and achieve accurate image classification is one of the major research challenges in this field.

[0003] In early research, the spectral dimension was considered the most important basis for classification. Consequently, many classification algorithms focused on modeling spectral information at the pixel level. Commonly used methods include traditional supervised classification algorithms such as support vector machines (SVMs), multinomial logistic regression (MLR), naive Bayesian classifiers, and decision trees. While these methods have excellent generalization capabilities for solving linear or weakly nonlinear problems, they ignore the structural relationships between pixels in the spatial dimension. This leads to poor classification performance in images with high background noise or blurred object edges, particularly when identifying edge pixels and non-uniform regions.

[0004] As research continues to deepen, scholars have gradually realized that relying solely on spectral information is insufficient to fully express the differences between objects, especially when different objects have similar spectral characteristics, or when the same object exhibits different spectral responses due to changes in external conditions. Therefore, incorporating spatial structure information into the classification model has become a key breakthrough in improving classification accuracy. Based on this idea, a series of spatial-spectral joint classification methods have been proposed. These methods can be roughly divided into three categories: the first is to enhance spatial features through image preprocessing methods (such as image filtering, dimensionality reduction, principal component extraction, and MNF); the second is to optimize the initial classification results through post-processing methods (such as conditional random fields and smoothing operations); and the third is to integrate spatial and spectral information into a unified classification framework to form a comprehensive modeling strategy.

[0005] Among the three methods mentioned above, joint spatial-spectral models have received particular attention, with sparse representation-based classification models showing promising development potential. Sparse Representation Classification (SRC) is a signal processing technique that has emerged in recent years. Its core concept is that a sample to be classified can be reconstructed from a small number of representative samples in a training set through a linear combination. This method constructs sparse coding coefficients, exploits the sparse correlations between training and test samples, and determines pixel categories through the reconstructed residuals. Compared with traditional methods, SRC is more resistant to noise, requires fewer training samples, and exhibits greater adaptability and robustness.

[0006] Although sparse representation classification methods have shown significant advantages in hyperspectral image classification, they still have several shortcomings. Traditional sparse representations typically only utilize spectral information and ignore the constraints of spatial neighborhood relationships, resulting in poor classification results at image edges and in regions with complex structures. At the same time, due to the use of superpixels or monotone neighborhood methods, the image classification preprocessing step makes minimal use of spectral information. Summary of the Invention

[0007] To solve the above technical problems, the present invention provides a joint sparse representation hyperspectral image classification method based on dual neighborhood constraints, comprising:

[0008] S1. Input hyperspectral image data and preprocess the hyperspectral image data using band grouping based on spectral features and MNF;

[0009] S2. performing erosion and dilation processing on the principal component image of the image by using an image processing method based on morphological reconstruction;

[0010] S3. Perform watershed superpixel segmentation on the morphologically processed feature map to obtain superpixel neighborhoods; and simultaneously perform Fuzz C-mean clustering on the morphologically processed feature map to obtain cluster neighborhoods.

[0011] S4. Double neighborhood constraints are applied to the generated superpixel neighborhood and clustering area;

[0012] S5. Construct a joint sparse representation model based on the dual-constrained neighborhood, superpixel neighborhood, and cluster neighborhood, and calculate the reconstruction residual to obtain the classification result;

[0013] S6. Use the majority voting method to unify the classification results and obtain the final classification results.

[0014] Beneficial effects of the present invention:

[0015] By combining spatial and spectral information, the present invention demonstrates that using PCA (Principal Component Analysis) alone for data dimensionality reduction is less effective than using MNF (Minimum Noise Fraction) for noise reduction. This is because MNF is essentially a two-step principal component transformation, requiring significantly more computational effort than PCA. Therefore, when processing hyperspectral images, the present invention can first perform band grouping before performing MNF data dimensionality reduction preprocessing. This approach has the advantages of first performing band clustering based on spectral feature selection to more effectively utilize inter-spectral correlations in hyperspectral images, further compress the spectral images, and remove redundancy between spectra. Finally, MNF data dimensionality reduction further removes redundant and noisy information, thereby improving data readability and classification accuracy. An improved watershed algorithm is used to perform superpixel segmentation on hyperspectral images, and then FCM clustering is performed on the hyperspectral images. The neighborhood is adaptively selected through weight calculation under the dual constraints of the formed cluster neighborhood and the superpixel field. The superpixel field, cluster field, and constraint field are used for joint sparse representation. The classification results are fused and adjusted by correcting the rules, thereby improving the hyperspectral image classification effect and improving the classification accuracy of edge pixels. BRIEF DESCRIPTION OF THE DRAWINGS

[0016] Figure 1 This is a flow chart of the joint sparse representation hyperspectral image classification based on dual neighborhood constraints of the present invention;

[0017] Figure 2 A schematic diagram of constructing a superpixel map in step S3 of the present invention;

[0018] Figure 3 Schematic diagram of the dual neighborhood constraint in step S4 of the present invention. DETAILED DESCRIPTION

[0019] The following will clearly and completely describe the technical solutions in the embodiments of the present invention in conjunction with the accompanying drawings. Obviously, the described embodiments are only part of the embodiments of the present invention, not all of the embodiments. Based on the embodiments of the present invention, all other embodiments obtained by ordinary technicians in this field without making creative efforts are within the scope of protection of the present invention.

[0020] A joint sparse representation hyperspectral image classification method based on dual neighborhood constraints, such as Figure 1 As shown, the steps are as follows:

[0021] S1. Input hyperspectral data and preprocess the hyperspectral image data using band grouping and MNF based on spectral features;

[0022] S2. Principal component plot after morphological processing;

[0023] S3. Performing improved watershed superpixel segmentation on the feature map after morphological processing to obtain superpixel neighborhoods; and performing Fuzz C-mean clustering on the feature map after morphological processing to obtain cluster neighborhoods;

[0024] S4. Double neighborhood constraints are applied to the generated superpixel neighborhood and clustering area;

[0025] S5. Construct a joint sparse representation model for the dual-constrained neighborhood, superpixel neighborhood, and cluster neighborhood, and calculate the reconstruction residual to obtain the classification result;

[0026] S6. Use the majority voting method to unify the classification results and obtain the final classification results.

[0027] In one embodiment, band clustering and grouping based on spectral feature selection is first performed to more effectively utilize the inter-spectral correlation of hyperspectral images, further compress spectral images, and remove redundancy between spectra. Finally, MNF data dimensionality reduction can further remove redundant information and noise information in the data.

[0028] Specifically, the band grouping based on spectral features and the MNF preprocessing of hyperspectral image data in step S1 include:

[0029] Calculate the spectral angular distance (SAD) of the band. The band grouping metric based on spectral characteristics is the spectral angular distance (SAD).

[0030] By treating the spectrum of each pixel in the hyperspectral image as a high-dimensional vector, the similarity between the spectra is measured by calculating the angle between the two vectors. The smaller the angle, the more similar the two spectra are. We can group the two bands into one group, which can reduce the amount of calculation for subsequent classification work. It can be expressed as:

[0031] SAD(B i ,B j )=cos (-1) ((B i ·B j ) / (||B i |‖‖|B j ||))

[0032] Among them B i and B j A spectral vector representing two bands;

[0033] Based on the SAD metric, the band grouping can be expressed as:

[0034] First, calculate the similarity matrix S between all band pairs, then the distance matrix D = 1-S: S = [s ij ] where s ij =SAD(B i ,B j ); Then, a clustering algorithm is applied to divide the bands into K groups, and the distance matrix D is directly used to assign the bands to the nearest cluster center.

[0035] Perform MNF dimensionality reduction. The MNF process is the transformation matrix:

[0036] A=WU′

[0037] Where A is the change matrix, W is the noise whitening matrix, and U′ is the signal subspace projection matrix;

[0038] The transformation result is:

[0039] y=A T x

[0040] The MNF after grouping (transformation of each group) is:

[0041]

[0042] Among them, y (k) represents the MNF transformation result of the k-th group of bands, represents the transpose of the MNF transformation matrix of group k, x (k) A pixel vector representing the kth band in the original hyperspectral data.

[0043] The final result can be expressed as:

[0044]

[0045] in It represents the first m components retained after the MNF transformation of the k-th group of bands (arranged in descending order of signal-to-noise ratio).

[0046] Furthermore, the image processing method based on morphological reconstruction in step S2 mainly includes the following key steps: First, a rectangular structuring element of a preset size is used to perform morphological processing on the preprocessed principal component image. This process involves basic erosion and dilation operations. This processing method can effectively enhance the spatial differences between adjacent pixels in the image, thereby significantly improving the subsequent recognition of spectral feature edges. The principal component feature map after morphological processing is eroded and expanded, expressed as:

[0047]

[0048]

[0049] Among them, Θ represents the corrosion operation, represents the expansion operation, I represents the principal component feature map after morphological processing, B represents the structural element, represents the image of B, x represents the two-dimensional coordinate vector in the image (e.g., x = (i, j)), and b represents the offset relative to the center point. Morphological erosion effectively separates weakly connected adjacent objects, while dilation fills and repairs microscopic gaps. The combination of these two basic operations constitutes the core processing mechanism of morphological opening and closing operations.

[0050] Specifically, the morphological opening operation O(I) and closing operation ρ(I) are:

[0051]

[0052] Furthermore, for step S3, this study adopts a watershed algorithm based on

[15] to achieve superpixel segmentation. This method first performs morphological preprocessing on the feature map, and then completes the division of the superpixel area based on the pixel gradient features, aiming to simultaneously maintain the compactness and boundary conformity of the superpixel.

[0053] Specifically, first, the gradient map (enhanced edge) is calculated. The gradient magnitude in the gradient metric Sobel operator is: G = |G x |+|G y |, where G x is the horizontal gradient, G y is the vertical gradient; find the local minimum point (as the seed / initial water source); simulate the flooding process and record which area of ​​"water" each pixel is flooded by; the confluence of water in different areas is marked as the boundary (watershed line). The schematic diagram of superpixel construction on the typical dataset Indian Pines is shown below. Figure 2 As shown, the super pixel neighborhood L(X) is obtained.

[0054] Preferably, after superpixel division, the entire image is divided into more than a dozen blocks, and each block is represented by sampling a different value.

[0055] In one embodiment, the simultaneous execution of the FuzzC-mean clustering algorithm on the morphologically processed feature maps in step S3 includes:

[0056] S11. For the pixel to be measured, select a window n×n;

[0057] S12. Initialize the membership matrix U using random numbers in the interval [0,1] and satisfy the formula:

[0058]

[0059] where u jrepresents the membership probability of sample i to cluster j, and N represents the number of pre-set target categories.

[0060] S13. Calculate the cluster center using the formula:

[0061]

[0062] S14. Update member membership, the formula is:

[0063]

[0064] S14. Calculate the objective function, which is:

[0065]

[0066] where u ij is x j The membership of the members in the i-th cluster, the sum of the membership is 1; c i represents the cluster center; n represents the number of samples; Represents the weighted index. dist(x j ,c i ) is the cluster center c i and the neighboring pixel x j The spectral cosine distance between them is expressed as;

[0067]

[0068] Among them G c As a local similarity measure, it can effectively suppress noise interference while maintaining the detailed features of the image. Its expression is:

[0069]

[0070] Among them, is the mean value of pixels in the neighborhood;

[0071] S15. Repeat steps S13 and S14 until J m The value of is less than a certain tolerance value ε or no longer decreases for a given test pixel, where the termination condition is:

[0072]

[0073] Where k is the number of iterations and ε is the error threshold.

[0074] Furthermore, for the constraint rule of step S4, its superpixel neighborhood is L(X), its cluster neighborhood is L(Y), then the overlapping part of the superpixel neighborhood and the cluster neighborhood is L(U), Figure 3 Demonstrates the composition of dual neighborhood constraints.

[0075] Specifically, the FCM clustering algorithm can be used to determine the membership matrix U, which represents the cluster average weight of each pixel in the cluster. In order to use this membership matrix to assign weights to new neighborhoods, we need to convert the cluster center weights into test pixel weights.

[0076] The weight of the test pixel is U(x), and the membership matrix update expression is:

[0077]

[0078] In the formula, ξ is the threshold value. By adjusting the size of ξ, the neighborhood weight distribution can be controlled. After obtaining the new membership matrix, the new neighborhood is weighted using the membership matrix to obtain the weighted neighborhood L W =L(U)*U W .

[0079] In one embodiment, a joint sparse representation is constructed using a constrained neighborhood, further integrating spatial and spectral information into the sparse representation process.

[0080] Specifically, for the joint sparse representation construction in step S5, the process is as follows:

[0081] S21. Obtain the weighted constraint neighborhood L through step S4 U

[0082] S22. The superpixel neighborhood L X , cluster neighborhood L Y , constrained neighborhood L U They are used to construct a joint sparse representation model. The construction steps are as follows: the SOMP algorithm is used to solve the sparse coefficient objective function formula:

[0083]

[0084] st‖S‖ row,0 ≤K0

[0085] in, Represents the objective function of solving sparse coefficients, D represents the dictionary matrix, S represents the sparse coefficient matrix, L represents the observation signal matrix, st represents the constraint condition, row represents row sparsity (number of non-zero rows per column), and K0 represents the sparsity constraint (maximum number of non-zero rows allowed);

[0086] S23. Calculate the residual formula of each neighborhood reconstruction separately:

[0087]

[0088] where r c (L) represents the reconstruction residual of the cth class, Dc represents the sub-dictionary of class c (e.g. extracting class-related atoms from D), represents the sparse coefficient corresponding to the sth class, and C represents the total number of classes;

[0089] S24. Complete classification based on the reconstructed residuals. The formula is:

[0090]

[0091] Among them, Class(L) represents the predicted class label of signal L.

[0092] Furthermore, the comprehensive judgment method mentioned in step S6 is to adopt a majority voting method.

[0093] Specifically, the rule is: when the classification results of each neighborhood are different, the classification result of the constrained neighborhood is used as the classification result of the test pixel; we then use superpixels to refine the classification result to obtain the final classification. We use superpixels with known labels and training sample information to refine the classification result. First, we calculate the number of objects in each category in the superpixel classification result L = [l1,l2,...,l c ]. Where c is the number of object types. We then calculate the number of training examples in a superpixel.

[0094] Preferably, there are three distribution situations of superpixels at this time:

[0095] 1. If a superpixel does not contain any training examples, all test pixels within the superpixel will be assigned to the class with the largest number of objects;

[0096] 2. If a superpixel contains the category of a training example and the number is greater than 1, all test pixels in the superpixel are assigned to the category of the training example;

[0097] 3. If a superpixel contains training examples of multiple categories, the category of the test pixel in the superpixel will not change.

[0098] The final classification map corrected by the above correction rules is taken as the final classification result.

[0099] While embodiments of the present invention have been shown and described, it will be appreciated by those skilled in the art that various changes, modifications, substitutions, and variations may be made to these embodiments without departing from the principles and spirit of the invention, and that the scope of the invention is defined by the appended claims and their equivalents.

Claims

1. A joint sparse representation hyperspectral image classification method based on dual neighborhood constraints, characterized by: include: S1. Input hyperspectral image data and preprocess the hyperspectral image data using band grouping based on spectral features and MNF; S2. performing erosion and dilation processing on the principal component image of the image by using an image processing method based on morphological reconstruction; S3. Perform watershed superpixel segmentation on the morphologically processed feature map to obtain superpixel neighborhoods; and simultaneously perform Fuzz C-mean clustering on the morphologically processed feature map to obtain cluster neighborhoods. S4. Double neighborhood constraints are applied to the generated superpixel neighborhood and clustering area; S5. Construct a joint sparse representation model based on the dual-constrained neighborhood, superpixel neighborhood, and cluster neighborhood, and calculate the reconstruction residual to obtain the classification result; S6. Use the majority voting method to unify the classification results and obtain the final classification results.

2. The method for hyperspectral image classification based on joint sparse representation and dual neighborhood constraints according to claim 1, characterized in that: Hyperspectral image data preprocessing using spectral feature-based band grouping and MNF, including: Band grouping preprocessing based on spectral characteristics: The spectrum distance SAD of the band is calculated. The band grouping metric based on spectral characteristics is spectrum distance SAD: SAD(B i ,B j )=cos (-1) ((B i ·B j ) / (||B i |‖‖|B j ||)); among them, SAD(B i ,B j ) indicates band B i and B j The spectrum between them is called distance, B i and B j Represents the spectral vector of two bands i and j; The similarity matrix S between all band pairs is calculated based on the spectral distance SAD of the bands = [s ij ], obtain the distance matrix D = 1-S through the similarity matrix, where s ij =SAD(B i ,B j ), s ij Indicates band B i and B j The similarity between them is calculated; the clustering algorithm is applied to divide the bands into K groups, and the distance matrix D is directly used to assign the bands to the nearest cluster center; MNF pretreatment: The MNF calculation process is the transformation matrix A = WU'; where A represents the change matrix, W represents the noise whitening matrix, and U' represents the signal subspace projection matrix; Transform each group of bands through the transformation matrix k=1,...,K, and finally the transformation results of K groups of bands are obtained Among them, y (k) represents the MNF transformation result of the k-th group of bands, represents the transpose of the MNF transformation matrix of group k, x (k) Represents the pixel vector of the kth band in the original hyperspectral data, It represents the first m components retained after the MNF transformation of the k-th group of bands.

3. The method for hyperspectral image classification based on joint sparse representation and dual neighborhood constraints according to claim 1, characterized in that: The main component image of the image is eroded and expanded by an image processing method based on morphological reconstruction, including: Among them, Θ represents the corrosion operation, represents the expansion operation, I represents the principal component feature map after morphological processing, B represents the structural element, Represents the image of B, x represents the two-dimensional coordinate vector in the image, and b represents the offset relative to the center point.

4. The method for hyperspectral image classification based on joint sparse representation and dual neighborhood constraints according to claim 1, characterized in that: Perform watershed superpixel segmentation on the feature map after morphological processing to obtain the superpixel neighborhood, including: Calculate the gradient map, and the gradient amplitude G in the Sobel operator is: G=|G x |+|G y |, where G x represents the horizontal gradient, G y represents the vertical gradient; Find the local minimum point as the initial water source, simulate the flooding process, and record which area of ​​"water" each pixel is flooded by; mark the confluence of water from different areas as the boundary, and finally obtain the superpixel neighborhood L(X).

5. The method for hyperspectral image classification based on joint sparse representation and dual neighborhood constraints according to claim 1, characterized in that: Perform Fuzz C-mean clustering on the feature map after morphological processing to obtain the cluster neighborhood, including: S11. For the pixel to be measured, select a window n×n; S12. Initialize the membership matrix U with random numbers in the interval [0,1] and satisfy Among them, u j represents the membership probability of sample i to cluster j, and N represents the number of pre-set target categories; S13. Calculate cluster centers: Among them, u ij Represents x j The membership of the members in the i-th cluster, the sum of the membership is 1, n represents the number of samples, and m represents the weighted index; S14. Update member membership: Among them, dist(x j ,c i ) represents the cluster center c i and the neighboring pixel x j spectral cosine distance between them; S15. Calculate the objective function: Among them, G c represents the local similarity measure; S16. Repeat steps S13 and S15 until the template function J m The value of is less than the error threshold ε or no longer decreases for a given test pixel: Where k represents the number of iterations, ε represents the error threshold, represents the membership degree of the k-th group member.

6. The method for hyperspectral image classification based on joint sparse representation and dual neighborhood constraints according to claim 1, characterized in that: The generated superpixel neighborhood and clustering area are subjected to dual neighborhood constraints, including: The membership matrix U is determined by the FCM clustering algorithm. The matrix represents the cluster average weight of each pixel in the cluster, and the cluster center weight is converted into the test pixel weight U(x); Update the membership matrix: Among them, U w represents the updated membership matrix, U(i) represents the membership matrix of member i, and ξ represents the threshold; After obtaining the new membership matrix, use the membership matrix to weight the new neighborhood to obtain the weighted neighborhood L w =L(U)*U w .

7. The method for hyperspectral image classification based on joint sparse representation and dual neighborhood constraints according to claim 1, characterized in that: A joint sparse representation model is constructed based on the dual-constrained neighborhood, superpixel neighborhood, and cluster neighborhood, and the reconstruction residual is calculated to obtain the classification results, including: S21: Use the SOMP algorithm to solve the sparse coefficient objective function: in, Represents the objective function of solving sparse coefficients, D represents the dictionary matrix, S represents the sparse coefficient matrix, L represents the observation signal matrix, st represents the constraint condition, row represents row sparsity, K0 represents sparsity constraint; S22: Calculate the reconstruction residual of each neighborhood separately: Among them, r c (L) represents the reconstruction residual of the cth class, D c represents the sub-dictionary of category c, represents the sparse coefficient corresponding to the sth class, and C represents the total number of classes; S23: Complete classification based on reconstruction residuals Among them, Class(L) represents the predicted class label of the observation signal L.

8. The method for hyperspectral image classification based on joint sparse representation and dual neighborhood constraints according to claim 1, characterized in that: The classification results are unified using the majority voting method to obtain the final classification results, including: When the classification results of each neighborhood are different, the classification result of the constrained neighborhood is used as the classification result of the test pixel, and then superpixels are used to refine the classification result to obtain the final classification.

Images