A hyperspectral remote sensing image classification method of clustering and attention double-branch linkage
By employing a dual-path approach combining clustering and attention, and integrating deep learning and clustering theory, the problems of intra-class differences and inter-class similarities in hyperspectral remote sensing image classification were solved, thereby achieving accurate classification and improved generalization capabilities for hyperspectral images.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- GUANGZHOU COLLEGE OF COMMERCE
- Filing Date
- 2026-03-30
- Publication Date
- 2026-06-09
AI Technical Summary
Hyperspectral remote sensing image classification suffers from problems such as large intra-class differences and high inter-class similarities, which are difficult to solve effectively with existing technologies. In particular, when there is insufficient small sample label data, the classification accuracy and generalization ability are insufficient.
A hyperspectral remote sensing image classification method with dual-branch linkage of clustering and attention is adopted. Combining deep learning and clustering theory, the inherent structure of the data is discovered through clustering theory, pseudo-label data is provided, and multi-scale multi-branch feature extraction and attention enhancement modules are designed to remove band redundancy information and improve feature classification accuracy.
It effectively solves the problems of insufficient small sample label data and redundant interference of band information, improves the classification accuracy and generalization ability of hyperspectral images, reduces intra-class differences and inter-class similarities, and obtains more accurate feature representations.
Smart Images
Figure CN122176527A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of image processing technology, specifically relating to a hyperspectral remote sensing image classification method that combines clustering and attention in a dual-branch linkage. Background Technology
[0002] Hyperspectral remote sensing images represent the spatial and spectral information of ground objects in the form of data cubes. Therefore, in addition to spatial information, they can also obtain rich spectral feature information. Spectral characteristic curves reflect the material properties of ground objects, and each material has a different spectral characteristic curve, which is the physical basis for the widespread use of hyperspectral images in ground object classification. Currently, classification technology based on hyperspectral remote sensing images is widely used in precision agriculture, environmental monitoring, mineral exploration, food safety, and military reconnaissance. However, hyperspectral image data cubes contain a large number of bands, with high correlation between bands and a lot of redundant information. This easily leads to phenomena such as "different spectra for the same object" or "same spectra for different objects." Therefore, when using hyperspectral remote sensing images for ground object classification, the common phenomenon of large intra-class differences or high inter-class similarities poses a significant challenge to the accuracy of hyperspectral remote sensing image classification.
[0003] Currently, there are numerous research results and application cases in hyperspectral remote sensing image classification. These theories and methods can be broadly divided into traditional methods and deep learning methods. Traditional classification methods mainly rely on manually designed features and classifiers to achieve spectral classification. Examples include supervised methods such as Support Vector Machine (SVM), Maximum Likelihood Classification (MLC), and Random Forest (RF), as well as unsupervised methods such as K-means and Fuzzy C. When using traditional classification methods for feature extraction and classification of hyperspectral remote sensing images, the general approach is to first use methods such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA) to reduce the dimensionality of the hyperspectral image data cube, and then perform feature extraction and classification on the reduced-dimensional image. Although traditional methods can achieve good results in low-dimensional feature spaces, their ability to represent and model nonlinear spectral features is limited, making it difficult to solve the widespread intra-class differences and inter-class similarities in hyperspectral remote sensing images. Summary of the Invention
[0004] To address the problems of existing technologies, this invention provides a hyperspectral remote sensing image classification method that combines clustering and attention in a dual-branch approach. This method fully integrates the advantages of deep learning and clustering theories, as well as the benefits of attention mechanisms and multi-scale theory. Through clustering theory, it not only discovers the inherent structure of the data, alleviating the problem of insufficient labeled samples, but also provides pseudo-label data for deep learning, guiding feature learning and extraction. Statistical features remove redundant information between bands, maintaining physical correlation while reducing interference with the model, thus improving the accuracy of feature classification. By designing a multi-scale, multi-branch feature extraction concept and an attention enhancement module, it can better perceive local or global features, obtaining more accurate representational features and information. This invention combines clustering principles with deep learning models, effectively leveraging the advantages of unsupervised feature learning and supervised classification. It not only solves the problems of insufficient small-sample labeled data and redundant band information interference, but also allows for the input of relevant physical characteristics and data structure information, thereby improving the classification accuracy and generalization ability of hyperspectral images.
[0005] To achieve the above objectives, the present invention provides the following solution: A hyperspectral remote sensing image classification method that combines clustering and attention in a dual-branch approach includes: Step 1: Input the original hyperspectral remote sensing image; Step 2: Preprocess the hyperspectral remote sensing image; wherein, the preprocessing includes: band removal and filtering; Step 3: Perform adaptive clustering on the preprocessed hyperspectral remote sensing image to obtain cluster feature maps and pseudo-label data; Step 4: Use a 1D convolutional neural network model to extract features from the clustering feature map and output a 1D convolutional feature map; Step 5: Perform attention mechanism processing on the preprocessed hyperspectral remote sensing image by fusing variance statistical features and spatial features, and output attention mechanism feature map; Step 6: Perform multi-scale 2D depthwise convolution on the attention feature map to output a 2D convolutional feature map; Step 7: Perform a fully connected concatenated process on the 1D convolutional feature map and the 2D convolutional feature map; Step 8: Based on the fully connected processing results and pseudo-label data, classify the feature pixels into categories and output the classified result image.
[0006] Preferably, step 2 includes: Input includes Hyperspectral remote sensing images in the band; Based on the hyperspectral remote sensing images, calculate the mean and variance of the pixel values for each band of the image; Calculate the mean of the variances of all image bands based on the variances. The threshold is calculated based on the mean. The variance of each band of hyperspectral remote sensing image is compared with a threshold. If it is greater than or equal to the threshold, it is retained; otherwise, it is discarded. Gaussian filtering is applied to the retained band images.
[0007] Preferably, step 3 includes: The hyperspectral images preprocessed using principal component analysis or linear discriminant analysis are then subjected to dimensionality reduction. The Mean-Shift algorithm is used to perform coarse clustering on the dimensionality-reduced hyperspectral image data; The K-means algorithm is used to perform fine clustering on the coarsely clustered hyperspectral image data, and the pixel classes are labeled to obtain cluster feature images and pseudo-label data.
[0008] Preferably, step 5 includes: Input the preprocessed hyperspectral image; Max pooling is performed on the preprocessed hyperspectral image to obtain the first feature map. ; The preprocessed hyperspectral image is subjected to average pooling to obtain the second feature map. ; According to the first feature map Second feature map Three-dimensional convolution, normalization, and activation operations are performed separately, and then fusion is performed by addition to obtain a spatial domain feature enhancement map. ; Enhanced map based on spatial domain features Calculate the variance feature map of the preprocessed hyperspectral image. ; Variance feature map Performing two consecutive cascaded 3D convolutions, normalization, and activation operations yields a feature map with enhanced variance features. ; Enhanced map based on spatial domain features Feature maps with variance feature enhancement The weighted feature map is obtained; Based on the weight feature map, the attention mechanism feature map is obtained. .
[0009] Preferably, step 6 includes: Input image based on attention mechanism enhancement ; Image enhancement based on attention mechanism Multi-scale convolution, normalization, and activation operations are performed, along with summation and fusion. Then, normalization and max pooling are applied to obtain the first two-dimensional convolutional feature map. ; For the first two-dimensional convolutional feature map Multi-scale convolution, normalization, and activation operations are performed, along with summation and fusion. Then, normalization and max pooling are applied to obtain the second two-dimensional convolutional feature map. ; For the second two-dimensional convolutional feature map Multi-scale convolution, normalization, and activation operations are performed, along with summation and fusion. Then, normalization and max pooling are applied to obtain the third-dimensional convolutional feature map. , As a preferred option, in step 8, the Softmax function is used to design the classification, and each feature pixel is classified and divided with reference to the pseudo-label data to obtain the classification result image.
[0010] Preferably, in step 4, after the cluster feature map is convolved by three cascaded convolutional modules, it is then subjected to batch normalization, activation, and max pooling operations to obtain a 1D convolutional feature map.
[0011] Compared with the prior art, the beneficial effects of the present invention are as follows: This invention preprocesses the input hyperspectral remote sensing image by performing band removal and filtering to remove bands that affect and interfere with hyperspectral image classification, and removes corresponding Gaussian noise through Gaussian filtering. It utilizes clustering theory to obtain corresponding clustering feature maps and pseudo-label data, which not only mines the data structure and provides labeled data for subsequent classification, overcoming the dependence on unlabeled small samples, but also uses the clustering feature maps, which contain data structure information, as features to guide feature extraction and classification in a 1D convolutional neural network. The preprocessed hyperspectral image is then processed by an attention mechanism based on statistical features, followed by feature extraction using a multi-scale 2D convolutional neural network. The feature maps from two different branches are then fully connected, and finally classified using a classifier and pseudo-label data. The output is the land cover classification or segmentation image of the hyperspectral remote sensing image. This invention not only effectively removes band images that affect and interfere with land cover classification in hyperspectral remote sensing image cubes, but more importantly, it effectively preserves the structural information of hyperspectral image data. It also generates effective pseudo-label data, overcoming the problem of scarce labeled samples. Furthermore, by constructing a variance attention module, a spatial-spectral dual-branch deep convolutional neural network, and a multi-scale branch feature extraction structure, it achieves cluster-guided spatial-spectral linkage modeling. This effectively leverages the advantages of combining unsupervised feature learning and supervised feature extraction, effectively solving the common problems of intra-class discrepancies, inter-class similarities, and insufficient generalization ability in hyperspectral remote sensing image classification models. Therefore, it provides an efficient solution for small-sample hyperspectral remote sensing image classification. Verification experiments using actual hyperspectral remote sensing images have yielded excellent results, demonstrating that this invention has advantages such as good preservation of detail information, low misclassification rate, and wide applicability for small-sample hyperspectral remote sensing image classification, and has good promotion and application value. The main contributions of this invention are reflected in the following aspects.
[0012] Compared with the prior art, the beneficial effects of the present invention are as follows: (1) The idea of introducing deep convolutional neural network models into clustering theory and introducing attention mechanism into variance statistical features is proposed. The concept of linking and fusing unsupervised feature learning and supervised feature extraction is proposed.
[0013] (2) A method for extracting and classifying features of hyperspectral remote sensing images based on a dual-branch convolutional neural network model with clustering theory and attention mechanism (DBCNN-CTAM algorithm) is proposed, which is helpful to solve the problems of small sample size and data label dependence in hyperspectral image classification, as well as the problems of intra-class differences and inter-class similarity.
[0014] (3) A band removal and filtering algorithm (BEF algorithm) is proposed. By analyzing and processing the statistical features of hyperspectral remote sensing images, redundant and interfering band image data are removed, and dimensionality reduction and sample screening are achieved. This helps to reduce the impact of foreign object homospectral data images on the processing and classification, and improves data processing efficiency and final classification accuracy.
[0015] (4) An adaptive clustering pseudo-label data input algorithm (ACPL algorithm) is proposed, which is beneficial to alleviate the problem of insufficient labeled samples and small sample size in hyperspectral image data.
[0016] (5) A feature extraction algorithm based on clustered feature images (CIFE algorithm) is proposed, which is beneficial to the internal structure information of input data, improves the classification accuracy of hyperspectral remote sensing images, and reduces inter-class similarity.
[0017] (6) An attention mechanism enhancement algorithm based on statistical variance features (SVFAE algorithm) is proposed, which is beneficial to enhance image spatial features and facilitates feature extraction and pixel classification processing.
[0018] (7) A multi-scale two-dimensional convolutional neural network model algorithm (MT-CNN algorithm) is proposed, which is conducive to obtaining more accurate and comprehensive spatial spectrum information and improving classification accuracy.
[0019] (8) In the DBCNN-CTAM algorithm, SVFAE algorithm and MT-CNN algorithm, dual-branch structure and multi-scale, multi-unit convolution module were designed respectively, with the aim of extracting more comprehensive and accurate spatial spectrum information as much as possible. Attached Figure Description
[0020] To more clearly illustrate the technical solution of the present invention, the drawings used in the embodiments are briefly introduced below. Obviously, the drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0021] Figure 1 This is a flowchart of the hyperspectral remote sensing image classification method according to an embodiment of the present invention; Figure 2 This is a block diagram illustrating the principle of the ACPL algorithm in this invention. Figure 3 This is a block diagram illustrating the principle of the CIFE algorithm in this invention. Figure 4 This is a block diagram illustrating the principle of the SVFAE algorithm in this invention. Figure 5 This is a block diagram illustrating the principle of the MT-CNN algorithm in this invention. Detailed Implementation
[0022] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of the present invention.
[0023] To make the above-mentioned objects, features and advantages of the present invention more apparent and understandable, the present invention will be further described in detail below with reference to the accompanying drawings and specific embodiments.
[0024] Example 1 like Figure 1 As shown, this invention provides a hyperspectral remote sensing image classification method (i.e., the DBCNN-CTAM algorithm) that combines clustering and attention in a dual-branch approach, comprising: Step 1: Input the original hyperspectral remote sensing image; Step 2: Preprocess the hyperspectral remote sensing image, the preprocessing including: band removal and filtering; Step 3: Perform adaptive clustering on the preprocessed hyperspectral remote sensing image to obtain cluster feature maps and pseudo-label data; Step 4: Use a 1D convolutional neural network model to extract features from the clustering feature map and output a 1D convolutional feature map; Step 5: Perform attention mechanism processing on the preprocessed hyperspectral remote sensing image by fusing variance statistical features and spatial features, and output attention mechanism feature map; Step 6: Perform multi-scale 2D depthwise convolution on the attention feature map to output a 2D convolutional feature map; Step 7: Perform a fully connected concatenated process on the 1D convolutional feature map and the 2D convolutional feature map; Step 8: Based on the fully connected processing results and pseudo-label data, classify the feature pixels into categories and output the classified result image.
[0025] As one embodiment of the present invention, in step 1, the hyperspectral remote sensing image is input by a satellite-borne or airborne (UAV) imager. However, these hyperspectral remote sensing images have undergone radiometric and geometric correction processing and lack corresponding ground label data. The land cover types in the hyperspectral imaging area are generally not very complex, mainly consisting of regional targets.
[0026] As one embodiment of the present invention, in step 2, the bands of the hyperspectral remote sensing image are removed and filtered. The present invention proposes a new band removal and filtering algorithm (BEF algorithm) to complete the removal and filtering of hyperspectral remote sensing image bands; including: Step 2.1: Use the BEF algorithm to remove band data in hyperspectral remote sensing images that exhibit foreign object spectral similarity, and then filter the remaining band data. The specific steps are as follows; Step 2.1.1: Input contains Hyperspectral remote sensing images in [number] bands. The image size for each band is [size]. , and These represent the height and width of the image, respectively. Step 2.1.2: Calculate the mean value of each band image pixel using equation (1). and variance ; (1) in, Represents variance. This represents the size of the image, which is the sum of the total number of pixels in the image. Represents the grayscale value of the image. This represents the mean value of all pixels in the image. The ordinal number representing the band number, and , This represents the number of bands in the entire hyperspectral remote sensing image set. and They represent the first The variance and mean of each band image; Step 2.1.3: Calculate the mean variance of all band images using equation (2). ; (2) Step 2.1.4: Set the discrimination threshold using equation (3). ; (3) In the formula It is an adjustable factor that can be set to different values such as 0.1 or 0.2, depending on the actual hyperspectral image data; Step 2.1.5: Identification and Removal of Foreign Objects in the Same Spectral Band. Calculate the variance of each band image. Compare with threshold ,if Then save this If the image is valid, then discard the image for that band. Step 2.2: Filter the remaining hyperspectral band images. At this time, the noise in the hyperspectral images is mainly Gaussian noise. Therefore, the Gaussian filtering algorithm is selected to filter the hyperspectral images. Step 2.2.1: Setting the filter window. The key to Gaussian filtering lies in setting the filter window. If the filter window is too large, it will easily smooth out details; if the filter window is too small, noise removal will be unsatisfactory. Based on practical experience, the window size should be set to... or This is a suitable approach; multiple windows of different scales can be selected for fusion processing. Step 2.2.2: Perform filtering processing, setting the window to filter each hyperspectral band image one by one; Step 2.2.3: Save the results and output the preprocessed hyperspectral remote sensing image; As one embodiment of the present invention, in step 3, an adaptive hyperspectral image data adaptive clustering and pseudo-label input algorithm (ACPL algorithm) is proposed, such as... Figure 2 As shown, it includes: Step 3.1: Dimensionality reduction. The preprocessed hyperspectral image is then subjected to dimensionality reduction. Many methods exist for dimensionality reduction, such as Principal Component Analysis (PCA) and Linear Discriminant Analysis (LDA). Here, we choose PCA based on variance ranking for dimensionality reduction, which effectively preserves the variance of the original image. The main steps of the PCA algorithm are as follows: Step 3.1.1: Data standardization (mean removal) processing. In order to eliminate the difference in the units of different features, each feature is usually standardized before PCA processing, that is, each feature is subtracted from its mean to center the data, as shown in equation (4). (4) In the formula It is the raw data. This is the data after removing the mean; Step 3.1.2: Calculate the covariance matrix. The mean-removed data is used to calculate the covariance matrix, which reflects the correlation between various features. For the mean-removed data matrix... (Assuming there is) One sample and (features), its covariance matrix for: (5) Step 3.1.3: Eigenvalue decomposition, through the covariance matrix Eigenvalue decomposition yields eigenvalues and eigenvectors. Eigenvalues represent the variance of the principal components (i.e., the new coordinate axis directions) in the data, while eigenvectors represent the directions of these principal components. Covariance matrix. eigenvalues The feature vector is Then select the eigenvectors corresponding to the largest few eigenvalues. For hyperspectral remote sensing images, the first three eigenvectors usually contain more than 95% of the information. Step 3.1.4: Select principal components. After PCA transformation, the eigenvectors corresponding to the three largest eigenvalues of the hyperspectral remote sensing image usually retain the largest variance information in the data. The ACPL algorithm selects the coefficient maps of the three components to synthesize a hyperspectral remote sensing image for clustering and pseudo-label data generation. Step 3.2: Coarse Clustering Processing. The Mean-Shift algorithm is used to perform coarse clustering on the dimensionality-reduced hyperspectral remote sensing image to obtain a rough estimate of the number of clusters. Because the Mean-Shift algorithm does not require specifying the number of clusters, it can be used as an adaptive method to determine the number of land cover types in the hyperspectral remote sensing image. To improve computational speed, the ACPL algorithm only uses pixel intensity features for calculation, ignoring spatial coordinate features; Step 3.3: Fine-grained clustering. The K-means algorithm is used to perform fine-grained clustering of the hyperspectral remote sensing image, adjusting the parameters... The initial number of clusters is used as the K-means algorithm. Then, the K-means algorithm is run to perform clustering processing on the image, while generating class label data. The clustering results and pseudo-label data are saved for subsequent feature extraction and pixel classification. The specific process is as follows: Step 3.3.1: Set the initial number of cluster centers ,Right now An initial class, each class is , ,at this time ; Step 3.3.2: Divide the initial classes. Based on the number of classes, randomly divide them into... An initial cluster Calculate the number of samples in each cluster. ; Step 3.3.3: Calculate the mean of each class. ; Step 3.3.4: Calculate each sample With clusters Between Values are determined, and comparisons and attribution are performed, where... The clustering criterion function is calculated using equation (6); (6) Step 3.3.5: Iterate continuously until all samples are available. The classification and categorization are complete, and If no further changes occur, the algorithm terminates; otherwise, return to step 3.4.4. Step 3.4: Output clustering feature map, clustering result feature map As the output feature map.
[0027] As one embodiment of the present invention, in step 4, a clustering feature image feature extraction algorithm (CIFE algorithm) is designed, such as... Figure 3 As shown, using clustered feature images as guides for image feature extraction is an innovative contribution of the DBCNN-CTAM algorithm. The clustered feature maps are sequentially processed through three convolutional modules for convolutional feature extraction, and the final output is the convolutional feature map. The three convolutional modules form a cascade and act as a branch for feature extraction; specifically including: Step 4.1: Input cluster feature map ; Step 4.2: Perform the first stage of convolutional feature extraction. This stage uses 16 convolutional kernels, each with a size of [missing information]. After the convolution operation, batch normalization, activation, and max pooling operations are performed. The input feature map is... The output feature map is ; Step 4.3: Perform the second stage of convolutional feature extraction. This stage uses 32 convolutional kernels, each with a size of [missing information]. After the convolution operation, batch normalization, activation, and max pooling operations are performed. The input feature map is... The output feature map is ; Step 4.4: Perform the third stage of convolutional feature extraction. This stage uses 64 convolutional kernels, each with a size of [missing information]. After the convolution operation, batch normalization, activation, and max pooling operations are performed. The input feature map is... The output feature map is ; Step 4.5: Output the feature map extracted by the one-dimensional convolutional network ; As one embodiment of the present invention, in step 5, a statistical variance feature attention enhancement algorithm, namely the SVFAE algorithm, is designed, such as... Figure 4 As shown, it is used as a parallel branch to extract features; specifically including: Step 5.1: Input the preprocessed hyperspectral image ,and , , and These represent the image height, width, and number of channels (bands), respectively. Step 5.2: Process the input hyperspectral image Spatial domain attention mechanism enhancement processing is performed by establishing a two-branch structure to process the input image separately. Perform max pooling and average pooling. Step 5.2.1: Max pooling processing, for the input image Max pooling is performed, with a pooling window size of [size missing]. The movement step size is 2, zero-padding is performed, and the first feature map is output. ; Step 5.2.2: Process the first feature map Process it, use The convolution kernels are used to perform 3D convolution operations to further refine the features, followed by normalization and activation operations; Step 5.2.3: Average pooling processing for the input image. Perform average pooling, with a pooling window size of [size missing]. The movement step size is 2, zero-padding is performed, and the second feature map is output. ; Step 5.2.4: Process the second feature map Process it, use The convolution kernels are used to perform 3D convolution operations to further refine the features, followed by normalization and activation operations; Step 5.2.5: Obtain the first feature map from the two branches. Second feature map The fusion process is performed to obtain a weighted feature map with enhanced spatial features. It can be represented by equation (7), here Represents the convolution kernel and its size, indicating Convolution operation, ReLU represents the pooling operation for the input features, while ReLU represents the activation operation. (7) Step 5.3: Establish the branch structure for the second feature extraction, see... Figure 4 Introducing statistical feature variance into the spatial attention mechanism is an innovation of this invention. (Input feature map) ; Step 5.3.1: Obtain the variance feature map Based on size By using a sliding window to process and calculate variance features, the input image can be obtained. Variance feature map , Step 5.3.2: Process the feature map The first convolution process is performed, using a 3D convolutional unit to analyze the variance feature map. Convolution is used to extract features, and the size of the convolution kernel is [value missing]. After performing the convolution operation, normalization and activation processing are then performed. Step 5.3.3: Process the feature map A second convolution process is performed, using a 3D convolutional unit to process the variance feature map. Perform convolution again, with the kernel size being... After the convolution operation, normalization and activation processing are performed. The two 3D convolutional units are connected in a cascaded manner, and the output is an enhanced map of statistical variance features. It can be expressed by equation (8); (8) Step 5.4: Merge the spatial features obtained from the two branch paths. and variance characteristics The fusion process is performed using equation (9), where and For the weights, to simplify calculations, we can set them as follows: , The output feature map after fusion; (9) Step 5.5: Obtain the image enhanced by the variance attention mechanism. , weight feature map and input image Perform multiplication to obtain the output image. Equation (10) is used to express that the output is the attention-enhanced image, which is used for subsequent 2D convolution feature extraction. (10) Step 5.5: Output variance feature attention mechanism to enhance the image ; In one embodiment of the present invention, in step 6, a multi-scale two-dimensional convolutional neural network feature extraction branch is designed. The variance feature attention mechanism enhancement and the two-dimensional convolutional neural network feature extraction together constitute the second branch of the DBCNN-CTAM algorithm, namely the multi-scale two-dimensional convolutional neural network feature extraction algorithm, or MT-CNN algorithm. Its principle block diagram is shown below. Figure 5 As shown, the algorithm designs three cascaded two-dimensional convolutional modules, each containing three different scales of convolutional neural network structures to extract features, making the extracted information more accurate and comprehensive. Each two-dimensional convolutional module contains three convolutional layers arranged in parallel, with kernel sizes of [missing information]. , and The first two-dimensional convolutional module has 16 convolutional kernels, the second two-dimensional convolutional module has 32 convolutional kernels, and the third two-dimensional convolutional module has 64 convolutional kernels; specifically including: Step 6.1: Enhance the image using an input variance feature attention mechanism. ; Step 6.2: Convolution and feature extraction of the first two-dimensional convolution module, including three parallel convolutional networks of different scales. The output of the first two-dimensional convolutional feature map is as follows. ; Step 6.2.1: Perform convolution operations on the first-scale convolutional network, with the kernel size being... The number of convolution kernels is 16. After convolution, normalization and activation processing are required. The input image is... ; Step 6.2.2: Perform convolution operations on the second-scale convolutional network, with the kernel size being... The number of convolution kernels is 16. After convolution, normalization and activation processing are required. The input image is... ; Step 6.2.3: Perform convolution operations on the third-scale convolutional network, with the kernel size being... The number of convolution kernels is 16. After convolution, normalization and activation processing are required. The input image is... ; Step 6.2.4: Fusing features at different scales. The three parallel multi-scale extracted features are added together (i.e., weights are the same). After fusion, normalization and pooling are performed. Zero-padding is performed during pooling. The pooling window size is [size missing]. With a stride of 2, the first 2D convolutional feature map output after processing is: It can be described by equation (11); (11) in, Represents the convolution kernel. , and This indicates the kernel size, and 16 represents the number of kernels, which conforms to... ReLU represents convolution, ReLU represents activation, and Pol represents pooling. Step 6.3: Convolution and feature extraction of the second two-dimensional convolution module, including three parallel convolutional networks of different scales. The processed output is the second two-dimensional convolutional feature map. ; Step 6.3.1: Perform convolution operations on the first-scale convolutional network, with the kernel size being... The number of convolutional kernels is 32. After convolution, normalization and activation processing are required. The input first two-dimensional convolutional feature map is... ; Step 6.3.2: Perform convolution operations on the second-scale convolutional network, with the kernel size being... The number of convolutional kernels is 32. After convolution, normalization and activation processing are required. The input first two-dimensional convolutional feature map is... ; Step 6.3.3: Perform convolution operations on the third-scale convolutional network, with the kernel size being... The number of convolutional kernels is 32. After convolution, normalization and activation processing are required. The input first two-dimensional convolutional feature map is... ; Step 6.3.4: Fusing features at different scales. The three parallel multi-scale extracted features are added together (i.e., weights are the same). After fusion, normalization and pooling are performed. Zero-padding is performed during pooling. The pooling window size is [size missing]. With a stride of 2, the processed output second 2D convolutional feature map is: It can be described by equation (12); (12) Step 6.4: Convolution and feature extraction of the third 2D convolution module, including three parallel convolutional networks of different scales. The output of the processed third 2D convolutional feature map is as follows. ; Step 6.4.1: Perform convolution operations on the first-scale convolutional network, with the kernel size being... The number of convolutional kernels is 64. After convolution, normalization and activation processing are required. The input second-dimensional convolutional feature map is... ; Step 6.4.2: Perform convolution operations on the second-scale convolutional network, with the kernel size being... The number of convolutional kernels is 64. After convolution, normalization and activation processing are required. The input second-dimensional convolutional feature map is... ; Step 6.4.3: Perform convolution operations on the third-scale convolutional network, with the kernel size being... The number of convolutional kernels is 64. After convolution, normalization and activation processing are required. The input second-dimensional convolutional feature map is... ; Step 6.4.4: Fusing features at different scales. The three parallel multi-scale extracted features are added together (i.e., weights are the same). After fusion, normalization and pooling are performed. Zero-padding is performed during pooling. The pooling window size is [size missing]. With a stride of 2, the processed output third-dimensional convolutional feature map is: It can be described using equation (13); (13) Step 6.5: Output the third two-dimensional convolutional feature map after processing by the multi-scale deep convolutional neural network. ; As one embodiment of the present invention, step 7 includes: Step 7.1: Convert the feature map and feature map Perform full connection processing; Step 7.2: Input feature map And expand it into a column by column vectors, let the feature map The size is So after processing The size is ,and ; Step 7.3: Input feature map And expand it into a column by column vectors, let the feature map The size is So after processing The size is ,and ; Step 7.4: Put and Connecting them directly into a column is equivalent to concatenating them, which completes the full connection process of the two feature maps. As one embodiment of the present invention, step 8 includes: Step 8.1: Design a classifier, using the Softmax function to design a multi-class classifier; Step 8.2: Input pseudo-label data; Step 8.3: Perform discrimination processing on the fully connected data elements to complete the class division of feature pixels.
[0028] This invention proposes an algorithm for feature extraction and classification of hyperspectral remote sensing images, namely the DBCNN-CTAM algorithm, which can solve the problems of scarce sample labels and missing internal structure of data in small samples. First, the invention uses the BEF algorithm to preprocess the input hyperspectral remote sensing image to remove band data with heterogeneous spectral phenomena and filters the remaining band data to improve image quality. Second, a dual-branch structure is established for feature processing and extraction, with the preprocessed hyperspectral remote sensing image as input. In the first branch, the ACPL algorithm is used to input clustering feature images and pseudo-label data, followed by the CIFE algorithm to input the internal structure information of the data, outputting the feature map of the first branch. In the second branch, the SVFAE algorithm is used to perform spatial feature enhancement processing on the input hyperspectral image. This algorithm also incorporates two dual-branch structures to facilitate feature extraction. Then, the processed feature map is used to extract features using the MT-CNN algorithm, which is a three-branch multi-scale convolutional structure, outputting the feature map of the second branch. Finally, the feature maps of the two branches are fully connected in a cascaded manner, and a classifier is used to classify the pixels in the feature maps, outputting a classification map. Verification experiments were conducted using actual hyperspectral remote sensing images, and good results were obtained. This demonstrates that the present invention is indeed effective and can effectively solve some practical problems in the classification of small-sample hyperspectral remote sensing images, thereby improving the quality of hyperspectral images and fully realizing their inherent value. Therefore, the present invention has significant potential for widespread application.
[0029] This invention fully considers the characteristics of hyperspectral imaging mechanisms and hyperspectral remote sensing images, as well as the advantages of deep convolutional neural network models, clustering theory, and attention mechanisms, and the advantages of multi-scale models and multi-branch structures for feature extraction. Specific innovations are as follows: (1) Highly targeted This invention targets hyperspectral remote sensing images, specifically small sample data that lacks corresponding label data. In practical applications, it is difficult to obtain label data for hyperspectral remote sensing images, while deep learning models heavily rely on label data. Therefore, this invention fully utilizes the characteristics of hyperspectral image data itself and the advantages of deep learning models to effectively extract features and classify small sample hyperspectral remote sensing images, while maintaining minimal intra-class differences and inter-class similarities. Thus, this invention is highly targeted and applicable.
[0030] (2) Good design concept This invention introduces clustering feature maps and statistical variance attention feature maps, enabling better feature extraction and the acquisition of structural information within the data and high-frequency detail information of ground features. Simultaneously, a multi-branch structure is designed to facilitate comprehensive feature input. Each module employs a dual-branch structure; for example, the DBCNN-CTAM and SVFAE algorithms utilize dual-branch structures three times, while the MT-CNN algorithm employs a three-way parallel structure. This multi-branch structure design allows for the acquisition of both local and global information, as well as fine-grained and coarse-grained features. Furthermore, a multi-scale convolutional model is designed to better extract local features from different perspectives.
[0031] This invention introduces an optimized framework for the linked feature extraction of clustered feature images and hyperspectral images. By designing a dual-branch network structure, it achieves the fusion extraction of clustering constraints and multi-scale features. The introduction of the linkage between statistical variance features and attention mechanism can improve the input capability and classification accuracy of spatial-spectral features. By organically combining clustering algorithms with deep learning algorithms, it can fully leverage the advantages of unsupervised feature learning and supervised classification algorithms, thereby improving the generalization ability of the model and the application value of hyperspectral images.
[0032] (3) Good treatment effect By processing and applying different hyperspectral remote sensing images, this invention has achieved good results, maintaining good classification details and information with a relatively low error rate.
[0033] The embodiments described above are merely preferred embodiments of the present invention and are not intended to limit the scope of the present invention. Various modifications and improvements made to the technical solutions of the present invention by those skilled in the art without departing from the spirit of the present invention should fall within the protection scope defined by the claims of the present invention.
Claims
1. A hyperspectral remote sensing image classification method that combines clustering and attention in a dual-branch approach, characterized in that, include: Step 1: Input the original hyperspectral remote sensing image; Step 2: Preprocess the hyperspectral remote sensing image; wherein, the preprocessing includes: band removal and filtering; Step 3: Perform adaptive clustering on the preprocessed hyperspectral remote sensing image to obtain cluster feature maps and pseudo-label data; Step 4: Use a 1D convolutional neural network model to extract features from the clustering feature map and output a 1D convolutional feature map; Step 5: Perform attention mechanism processing on the preprocessed hyperspectral remote sensing image by fusing variance statistical features and spatial features, and output attention mechanism feature map; Step 6: Perform multi-scale 2D depthwise convolution on the attention feature map to output a 2D convolutional feature map; Step 7: Perform a fully connected concatenated process on the 1D convolutional feature map and the 2D convolutional feature map; Step 8: Based on the fully connected processing results and pseudo-label data, classify the feature pixels into categories and output the classified result image.
2. The hyperspectral remote sensing image classification method with clustering and attention dual-branch linkage as described in claim 1, characterized in that, Step 2 includes: Input includes Hyperspectral remote sensing images in the band; Based on the hyperspectral remote sensing images, calculate the mean and variance of the pixel values for each band of the image; Calculate the mean of the variances of all image bands based on the variances. The threshold is calculated based on the mean. The variance of each band of hyperspectral remote sensing image is compared with a threshold. If it is greater than or equal to the threshold, it is retained; otherwise, it is discarded. Gaussian filtering is applied to the retained band images.
3. The hyperspectral image classification method as described in claim 2, characterized in that, Step 3 includes: The hyperspectral images preprocessed using principal component analysis or linear discriminant analysis are then subjected to dimensionality reduction. The Mean-Shift algorithm is used to perform coarse clustering on the dimensionality-reduced hyperspectral image data; The K-means algorithm is used to perform fine clustering on the coarsely clustered hyperspectral image data, and the pixel classes are labeled to obtain cluster feature images and pseudo-label data.
4. The hyperspectral image classification method as described in claim 3, characterized in that, Step 5 includes: Input the preprocessed hyperspectral image; Max pooling is performed on the preprocessed hyperspectral image to obtain the first feature map. ; The preprocessed hyperspectral image is subjected to average pooling to obtain the second feature map. ; According to the first feature map Second feature map Three-dimensional convolution, normalization, and activation operations are performed separately, and then fusion is performed by addition to obtain a spatial domain feature enhancement map. ; Enhanced map based on spatial domain features Calculate the variance feature map of the preprocessed hyperspectral image. ; Variance feature map Performing two consecutive cascaded 3D convolutions, normalization, and activation operations yields a feature map with enhanced variance features. ; Enhanced map based on spatial domain features Feature maps with variance feature enhancement The weighted feature map is obtained; Based on the weight feature map, the attention mechanism feature map is obtained. .
5. The hyperspectral image classification method as described in claim 4, characterized in that, Step 6 includes: Input attention-based image enhancement ; Image enhancement based on attention mechanism Multi-scale convolution, normalization, and activation operations are performed, along with summation and fusion. Then, normalization and max pooling are applied to obtain the first two-dimensional convolutional feature map. ; For the first two-dimensional convolutional feature map Multi-scale convolution, normalization, and activation operations are performed, along with summation and fusion. Then, normalization and max pooling are applied to obtain the second two-dimensional convolutional feature map. ; For the second two-dimensional convolutional feature map Multi-scale convolution, normalization, and activation operations are performed, along with summation and fusion. Then, normalization and max pooling are applied to obtain the third-dimensional convolutional feature map. .
6. The hyperspectral remote sensing image classification method with clustering and attention dual-branch linkage as described in claim 5, characterized in that, In step 8, the Softmax function is used to design the classification. Referring to the pseudo-label data, each feature pixel is classified and divided to obtain the classification result image.
7. The hyperspectral remote sensing image classification method with clustering and attention dual-branch linkage as described in claim 6, characterized in that, In step 4, after the cluster feature map is convolved by three cascaded convolutional modules, it undergoes batch normalization, activation, and max pooling operations to obtain a 1D convolutional feature map.