Hyperspectral image classification method for complex environment

Abstract

The invention relates to the technical field of hyperspectral remote sensing image processing and classification, and discloses a hyperspectral image classification method for a complex environment. According to the invention, based on the multi-scale attention mechanism reconstruction network, through combination of an information redistribution module, local and global attention mechanisms and residual connection, through multi-level information redistribution, feature selection and multi-scale fusion, the accuracy and robustness in an image classification task are effectively improved. The network can automatically learn important features and perform information fusion in global and local levels, so that the network has better expression ability and generalization ability when processing complex image data.

Description

Technical Field

[0001] The present invention relates to the technical field of hyperspectral remote sensing image processing and classification, and in particular to a hyperspectral image classification method for complex environments. Background Art

[0002] Currently, the Yellow River wetlands, as a key ecological conservation area in my country, boast extremely rich biodiversity and ecosystem services. However, with the intensification of human activities and climate change, wetland ecosystems face increasingly severe challenges, and wetland protection and monitoring have become key topics in ecological research. Hyperspectral remote sensing technology, by providing continuous spectral band information, can accurately classify different landform types, providing strong support for wetland ecological protection and resource management. UAV-mounted hyperspectral remote sensing equipment enables observations with both high spatial and spectral resolution, and through effective image stitching, large-scale remote sensing images can still be obtained.

[0003] Existing hyperspectral image classification technologies have the following significant defects in complex environment applications: First, existing hyperspectral image classification methods usually adopt single-scale convolution operations or use some basic attention mechanisms (such as spatial attention). When faced with complex hyperspectral images, a single scale cannot fully capture the rich local details and global structural information in the image; second, important features in the image are sometimes not evenly distributed, and directly extracting features through conventional convolutional layers may miss some important local and global information; in addition, in deep neural networks, especially when using deep convolutional networks, as the number of layers increases, information may be lost, and the spatial information of the feature map may be gradually compressed, resulting in the loss of details and affecting the classification performance.

[0004] The invention patent with application number CN201810191507.8, "A multi-scale spatial-spectral collaborative classification method for hyperspectral images", provides a multi-scale spatial-spectral collaborative classification method for hyperspectral images, step one: feature extraction of the original hyperspectral image H to obtain a spectral information set Hspec composed of band subsets; step two: extracting multi-scale spatial information from the spectral information set Hspec to obtain multiple groups of multi-scale spatial information data sets Hspet; step three: fusing and preliminarily classifying the multiple groups of multi-scale spatial information data sets Hspet with the spectral information set Hspec to obtain a preliminary classification result map Qinit; step four: decomposing the preliminary classification result map Qinit into k probability result maps P; step five: post-processing the k probability result maps P to obtain the final classification result map Ofin, thereby completing the spatial-spectral collaborative classification of the hyperspectral image.

[0005] This existing technology reduces the complexity of the model by extracting multi-scale spatial information. Although it reduces training time and computing resource consumption and improves classification efficiency, it lacks an adaptive weight adjustment mechanism and is difficult to highlight the characteristics of key features in complex terrain. When applied to the hyperspectral image classification of the Yellow River wetland, the classification results will be affected. Summary of the Invention

[0006] The purpose of this invention is to provide a hyperspectral image classification method for complex environments, which can achieve efficient classification of hyperspectral images in large-scale scenes and complex environments through multi-scale feature extraction and information redistribution technology.

[0007] The present invention adopts the following technical solutions:

[0008] A hyperspectral image classification method for complex environments, comprising:

[0009] S1: Acquire and preprocess UAV hyperspectral remote sensing image data;

[0010] S2: Label the different types of objects in the preprocessed image data and divide them into training set, validation set and test set;

[0011] S3: Build a convolutional neural network model, including:

[0012] The data preprocessing module is used to perform feature extraction and nonlinear transformation on the input UAV hyperspectral images in the training set;

[0013] The information redistribution module is used to normalize the features output by the data preprocessing module and redistribute the high entropy and low entropy features of the features using scaling parameters and offset parameters;

[0014] The local attention module is used to extract local spatial information from the features output by the information redistribution module, generate spatial local attention weights, and perform weighted adjustment on the input features to obtain locally weighted features;

[0015] The global attention module is used to obtain globally weighted features from the features output by the local attention module through global convolution, global average pooling, and fully connected layers;

[0016] The feature fusion module is used to calculate the residual features and fuse the local weighted features, the global weighted features and the residual features to obtain a fused feature map;

[0017] The classification module is used to obtain pooled features by global pooling of the fused feature map. The pooled features are then passed through the first fully connected layer to obtain the category prediction results and output them;

[0018] S4: Validate the convolutional neural network model using the UAV hyperspectral image validation set;

[0019] S5: Input the test set of drone hyperspectral images to be classified and obtain the classification results of drone hyperspectral images.

[0020] Preferably, in step S1, image data is acquired by a hyperspectral camera and a visible light camera onboard the UAV, and preprocessing includes grayscale conversion and filtering noise reduction processing, as well as image stitching based on feature point matching.

[0021] Preferably, in step S2, the pixels are divided into a training set, a test set, and a validation set in a ratio of 1:2:2.

[0022] Preferably, in step S3, the data preprocessing module performs a convolution operation on the input drone hyperspectral image to output a feature map, performs batch normalization on the output feature map, and then applies a ReLU activation function to obtain features with specific information.

[0023] Preferably, in step S3, the information redistribution module performs batch normalization (BN) on the features output by the data preprocessing module to obtain normalized features; the information redistribution module continuously adjusts the scaling parameters and offset parameters through training and learning, dynamically adjusts the normalized features, and generates high-entropy partial features; subtracts the high-entropy partial features from the normalized features to obtain low-entropy partial features; and adds the high-entropy partial features and the low-entropy partial features and outputs them to the local attention module.

[0024] Preferably, in step S3, the local attention module obtains a local feature map by performing a convolution operation on the features output by the information redistribution module, and performs batch normalization and Sigmoid activation function on the local feature map to generate spatial local attention weights, and then multiplies the spatial local attention weights with the features output by the information redistribution module to obtain locally weighted features.

[0025] Preferably, in step S3, the global attention module obtains a global feature map by performing a convolution operation on the locally weighted features, and performs global pooling on the global feature map to generate a global feature vector, which is then processed by the first fully connected layer and the second fully connected layer of the fully connected layer to obtain the spatial global attention weight, and the spatial global attention weight is multiplied by the locally weighted features for global weighting to obtain the globally weighted features.

[0026] Preferably, in step S3, the feature fusion module adjusts the number of channels of the input drone hyperspectral image to be consistent with the number of channels of the output feature of the global attention module through a convolution operation to obtain a residual feature.

[0027] Preferably, in step S3, the classification module obtains pooled features by globally pooling the feature maps output by the feature fusion module; and then maps the pooled features to the category space through the first fully connected layer to obtain a category prediction result.

[0028] Preferably, in step S5, the test set of image data to be classified is sequentially input into the trained convolutional neural network model to obtain the category prediction result of each pixel point, and all category prediction results are spliced in the order of the original image to obtain the classification result of the drone hyperspectral image.

[0029] The beneficial effects of the present invention are as follows: an information redistribution module is designed, which continuously adjusts the scaling parameters and offset parameters through training and learning, dynamically adjusts the input features, optimizes the feature performance of the high-entropy part, and reduces the influence of the low-entropy part; combines local and global attention mechanisms, and uses convolution kernels of different sizes to extract details and global features respectively. This multi-scale attention mechanism can adaptively adjust the focus of the model according to the needs of different regions, thereby significantly improving the recognition ability of hyperspectral image details and global patterns, which is crucial for the classification of land objects in complex scenes; in hyperspectral image classification, a large amount of band and detail information needs to be processed. Through the residual connection module, it is ensured that each layer can effectively transmit information, thereby improving the training effect of the deep network. Especially in the deep network, the residual connection provides an effective solution. BRIEF DESCRIPTION OF THE DRAWINGS

[0030] Figure 1 This is a flow chart of the hyperspectral image classification method for complex environments of the present invention;

[0031] Figure 2 Schematic diagram of the multi-scale attention network in this invention. DETAILED DESCRIPTION

[0032] The present invention is described in detail below with reference to the accompanying drawings and embodiments:

[0033] As attached Figures 1 to 2 , is a flow chart of the hyperspectral image classification method for complex environments of the present invention, which is as follows:

[0034] S1: Acquire and preprocess UAV hyperspectral remote sensing image data;

[0035] S2: Label the different types of objects in the preprocessed image data and divide them into training set, validation set and test set;

[0036] S3: Construct a convolutional neural network model, which includes a data preprocessing module, an information redistribution module, a local attention module, a global attention module, a feature fusion module, and a classification module;

[0037] The data preprocessing module is used to perform feature extraction and nonlinear transformation on the input UAV hyperspectral images in the training set;

[0038] The information redistribution module is used to normalize the features output by the data preprocessing module and redistribute the high-entropy and low-entropy features of the features using the scaling and offset parameters that are continuously adjusted through training and learning;

[0039] The local attention module is used to extract local spatial information from the features output by the information redistribution module, generate spatial local attention weights, and perform weighted adjustment on the input features to enhance the importance of local areas;

[0040] The global attention module is used to obtain globally weighted features from the features output by the local attention module through global convolution, global average pooling, and fully connected layers;

[0041] The feature fusion module is used to calculate the residual features and fuse the local weighted features, the global weighted features and the residual features to obtain a fused feature map;

[0042] The classification module is used to obtain pooled features by global pooling of the fused feature map. The pooled features are then passed through the first fully connected layer to obtain the category prediction results and output them;

[0043] S4: Validate the convolutional neural network model using the obtained UAV hyperspectral image validation set;

[0044] S5: Input the test set of drone hyperspectral images to be classified and obtain the classification results of drone hyperspectral images.

[0045] In this embodiment, in step S1, image data with high spatial and spectral resolution can be obtained by using the drone's onboard hyperspectral camera and visible light camera, and visible light images are used for identification and confirmation, and statistics are performed on each category in the hyperspectral data.

[0046] In the present invention, the pre-processing operation in step S1 specifically includes the following steps:

[0047] S101: Grayscale the input image and filter it for noise reduction. First, the spectral reflectance is adjusted based on the standard spectral curve to reduce the influence of environmental factors. Then, SG smoothing filtering is used to remove random noise in the hyperspectral data to improve data quality. Finally, multiplicative scatter correction is used to eliminate the non-uniformity of the sensor response, so that the data more accurately reflects the spectral characteristics of the ground object.

[0048] S102: stitching the two images based on feature point matching; that is, detecting key feature points in the images, finding matching point pairs of the two images, and calculating the transformation relationship between the two images, thereby seamlessly stitching the images together.

[0049] In the present invention, the step S2 marks different types of objects in the image and randomly divides each marked pixel point; the pixel points are divided into a training set, a test set, and a validation set in a ratio of 1:2:2, and at least 400 sample points are selected for marking in each category. The more labeled samples and the higher the quality, the more accurate the model will perform in subsequent classification tasks.

[0050] In this embodiment, this division method ensures the representativeness of training samples and test samples, laying a solid foundation for subsequent model training and performance evaluation.

[0051] In the present invention, the data preprocessing module in step S3 first extracts low-level features from the drone hyperspectral image x input to the convolutional neural network model through a convolution operation with a convolution kernel of 3×3, outputs a feature map, and performs batch normalization on the output feature map to accelerate the training process and stabilize the dynamic range of the training, reduce the internal covariance shift, and then apply the ReLU activation function to obtain the feature x with specific information. stem The formula is as follows:

[0052] x stem =ReLU(BN(Conv2D(x,64,3,1,1))) (1)

[0053] Among them, x stem Represents the features with specific information output by the data preprocessing module; x represents the input drone hyperspectral image; Conv2D(x,64,3,1,1) means applying a 3×3 convolution operation and outputting 64 channels; BN() represents the batch normalization function; ReLU() is the activation function.

[0054] In this embodiment, hyperspectral images are usually composed of hundreds of bands, each band representing different spectral information. This process helps to reduce computational complexity and ensure the computational efficiency of the model. The above operations can accelerate the training process and stabilize the dynamic range of training, reduce internal covariance shift, ensure output nonlinearity, and enhance the model's ability to express complex features.

[0055] The information redistribution module in step S3 processes the feature x output by the data preprocessing module. stem Perform batch normalization BN processing to obtain the normalized feature x norm The formula is as follows:

[0056] xnorm =BN(x stem ) (2)

[0057] The information redistribution module continuously adjusts the scaling parameter scale and the offset parameter shift through training learning to adjust the normalized feature x norm Dynamic adjustment is performed to enhance the expression of high entropy features and suppress the influence of low entropy features. The generated high entropy feature high_entropy is:

[0058] high_entropy=scale×x norm +shift (3)

[0059] By normalizing the feature x norm Subtract the high entropy feature high_entropy from the low entropy feature low-entropy; the formula is as follows,

[0060] low-entropy=x norm -high-entropy (4)

[0061] Add the high entropy feature high_entropy and the low entropy feature low-entropy to get the feature x after information redistribution irm , then the high entropy part is enhanced and the low entropy part is suppressed; the formula is as follows,

[0062] x irm =high-entropy+low-entropy (5)

[0063] In this embodiment, the information redistribution module is designed to strengthen the features of the high-entropy part while reducing the influence of the low-entropy part features, so that the model can better utilize the features with higher information entropy in the image to optimize the expressiveness of the features, and enable the network to more flexibly adjust the degree of attention to different features; the high-entropy part features often contain more information (such as complex backgrounds or boundaries), while the low-entropy part features may represent some less useful information. By dynamically adjusting the feature weights, the model can highlight more important information and improve classification accuracy. In addition, the use of batch normalization helps to stabilize network training and improve the convergence speed of the model.

[0064] The local attention module in step S3 is used to focus on the feature x output by the information redistribution module. irm Perform a convolution operation with a convolution kernel of 5×5 to obtain a local feature map, perform batch normalization on the local feature map to stabilize the data distribution, and generate the spatial local attention weight AttentionMap through the Sigmoid activation function local ,

[0065] AttentionMap local =σ(BN(Conv2D local (x irm ))) (6)

[0066] Among them, AttentionMap local Represents the spatial local attention weight, and the value represents the importance of each local area. AttentionMap local ∈[0,1], the closer to 1, the more important the feature at that position is, and vice versa; σ() represents the Sigmoid activation function; Conv2D local () indicates the application of a convolution operation with a convolution kernel of 5×5; BN() indicates the batch normalization function;

[0067] The calculated spatial local attention weight AttentionMap local and the feature x output by the information redistribution module irm Multiply and weight to obtain the local weighted feature x local .

[0068] x local =x irm ×AttentionMap local (7)

[0069] In this embodiment, this step can perform weighted processing on key areas in the hyperspectral image, so that important spatial and spectral features are highlighted, and the model's sensitivity to subtle differences in complex scenes is enhanced; the local attention mechanism focuses on small areas in the image, highlighting those detailed features that are crucial for classification; for hyperspectral images, local details such as textures and boundaries often contain important category information, and the local attention mechanism can accurately locate these key areas, improving the model's sensitivity to subtle differences in complex scenes.

[0070] The global attention module in step S3 takes the locally weighted feature x local The global feature map AttentionMap is calculated by the convolution operation with a convolution kernel of 11×11 global ,

[0071] AttentionMap global =Conv2D global (x local ) (8)

[0072] Among them, AttentionMap global Represents the global feature map; Conv2Dglocal () indicates the application of a convolution operation with a convolution kernel of 11×11;

[0073] Then the global feature map AttentionMap global Perform global pooling to obtain the global feature vector Pooled, which represents the context information of the entire image.

[0074] Pooled=GlobalPool(AttentionMap global ) (9)

[0075] Among them, Pooled represents the global feature vector; GlobalPool() represents the global pooling function;

[0076] The global feature vector after global pooling is processed by the fully connected layers fc1 and fc2. fc1 is the first fully connected layer, which converts the global feature vector into a lower-dimensional feature space for feature abstraction; fc2 is the second fully connected layer, which further processes the features output by fc1 to obtain the spatial global attention weight fc2_out.

[0077] fc1-out=ReLU(fc1(Pooled)) (10)

[0078] fc2_out=σ(fc2(fc1_out)) (11)

[0079] Among them, Pooled represents the global feature vector; fc1() represents the first fully connected layer function; ReLU() is the activation function; fc1-out represents the features output from the first fully connected layer fc1; fc2() represents the second fully connected layer function; σ() represents the Sigmoid activation function; fc2-out represents the spatial global attention weight;

[0080] Combine the spatial global attention weight fc2_out with the locally weighted feature x local Multiply them together for global weighting to obtain the globally weighted feature x global , further improving the expression of global information;

[0081] x global =x local ×fc2_out (12)

[0082] In this embodiment, the global attention mechanism is used to capture the overall structural information of the hyperspectral image, enabling the model to not only identify local details but also gain an understanding of the overall image structure. This multi-scale feature extraction method can improve the model's ability to recognize category boundaries and global patterns, thereby effectively reducing classification errors.

[0083] The feature fusion module in step S3 adjusts the number of channels of the UAV hyperspectral image x input to the convolutional neural network model through a convolution operation with a convolution kernel of 1×1, so that it is consistent with the locally weighted feature x in the current processing feature. local and the globally weighted features x global The number of channels is consistent to obtain the residual feature x residual ;

[0084] x residual =Conv2D residual (x,1×1) (13)

[0085] Where x represents the UAV hyperspectral image input to the convolutional neural network model; x residual Represents residual features; Conv2D residual (x,1×1) represents the application of a convolution operation with a kernel of 1×1 to the UAV hyperspectral image input to the convolutional neural network model;

[0086] The locally weighted feature x local , the globally weighted feature x global and residual features x residual Add together to form a fusion feature map x final The formula is as follows:

[0087] x final =x local +x global +x residual (14)

[0088] In this embodiment, the addition operation helps the network retain the original input information while effectively fusing the processed features; the residual connection module ensures that important information is not lost in the deep network, effectively solving the problems of gradient vanishing and information loss in the deep network; in this way, the network can be trained more stably and avoid performance degradation.

[0089] The classification module in step S3 is used to fusion feature map x final Perform global pooling to compress the feature map into a vector of fixed size and obtain the pooled feature x pooled , the formula is as follows,

[0090] x pooled =GlobalPool(x final ) (15)

[0091] Among them, GlobalPool() represents the global pooling function;

[0092] Through the first fully connected layer fc1, the pooled feature xpooled Map to the category space and finally output the category prediction result;

[0093] x class =fc1(x pooled ) (16)

[0094] Among them, fc1() represents the first fully connected layer function; x class Represents the category prediction result.

[0095] In this embodiment, step S4 uses the UAV hyperspectral image verification set obtained in step S2 to verify the convolutional neural network model in step S3 to ensure that the optimal model parameters are retained. The constructed network model is evaluated in each round using the verification set. If the accuracy of the current model on the verification set exceeds the previous best record, the weight of the current model is saved to a file to ensure that the optimal model parameters are retained.

[0096] In step S5, the test set of image data to be classified is sequentially input into the trained convolutional neural network model to obtain the category prediction result of each pixel point, and all category prediction results are spliced in the order of the original image to predict and classify the category to which the hyperspectral image belongs.

Claims

1. A hyperspectral image classification method for complex environments, characterized by: The following steps are included: S1: Acquire and preprocess UAV hyperspectral remote sensing image data; S2: Label the different types of objects in the preprocessed image data and divide them into training set, validation set and test set; S3: Build a convolutional neural network model, including: The data preprocessing module is used to perform feature extraction and nonlinear transformation on the input UAV hyperspectral images in the training set; The information redistribution module is used to normalize the features output by the data preprocessing module and redistribute the high entropy and low entropy features of the features using scaling parameters and offset parameters; The local attention module is used to extract local spatial information from the features output by the information redistribution module, generate spatial local attention weights, and perform weighted adjustment on the input features to obtain locally weighted features; The global attention module is used to obtain globally weighted features from the features output by the local attention module through global convolution, global average pooling, and fully connected layers; The feature fusion module is used to calculate the residual features and fuse the local weighted features, the global weighted features and the residual features to obtain a fused feature map; The classification module is used to obtain pooled features by global pooling of the fused feature map. The pooled features are then passed through the first fully connected layer to obtain the category prediction results and output them; S4: Validate the convolutional neural network model using the UAV hyperspectral image validation set; S5: Input the test set of drone hyperspectral images to be classified and obtain the classification results of drone hyperspectral images.

2. The hyperspectral image classification method for complex environments according to claim 1, characterized in that: In step S1, image data is acquired through the UAV's onboard hyperspectral camera and visible light camera. Preprocessing includes grayscale conversion, filtering and noise reduction, and image stitching based on feature point matching.

3. The hyperspectral image classification method for complex environments according to claim 1, characterized in that: In step S2, press The labeled pixels are divided into training set, test set and validation set according to the ratio of 4. The hyperspectral image classification method for complex environments according to claim 1, characterized in that: In step S3, the data preprocessing module performs convolution operation on the input UAV hyperspectral image to output the feature map, performs batch normalization on the output feature map, and then applies The activation function obtains features with specific information.

5. The hyperspectral image classification method for complex environments according to claim 4, characterized in that: In step S3, the information redistribution module performs batch normalization on the features output by the data preprocessing module. Processing to obtain normalized features; The information redistribution module continuously adjusts the scaling parameters and offset parameters through training and learning, dynamically adjusts the normalized features, and generates high-entropy features; subtracts the high-entropy features from the normalized features to obtain low-entropy features; and adds the high-entropy features and the low-entropy features and outputs them to the local attention module.

6. The hyperspectral image classification method for complex environments according to claim 5, characterized in that: In step S3, the local attention module obtains a local feature map by performing a convolution operation on the features output by the information redistribution module, and performs batch normalization and Sigmoid The activation function generates spatial local attention weights, which are then multiplied by the features output by the information redistribution module to obtain locally weighted features.

7. The hyperspectral image classification method for complex environments according to claim 6, characterized in that: In step S3, the global attention module obtains a global feature map by performing a convolution operation on the locally weighted features, and performs global pooling on the global feature map to generate a global feature vector. The global attention weight is then obtained after processing by the first fully connected layer and the second fully connected layer of the fully connected layer. The spatial global attention weight is multiplied by the locally weighted features for global weighting to obtain the globally weighted features.

8. The hyperspectral image classification method for complex environments according to claim 7, characterized in that: In step S3, the feature fusion module adjusts the number of channels of the input drone hyperspectral image to be consistent with the number of channels of the output features of the global attention module through convolution operation to obtain the residual features.

9. The hyperspectral image classification method for complex environments according to claim 8, characterized in that: In step S3, the classification module obtains pooled features by globally pooling the feature maps output by the feature fusion module; and then maps the pooled features to the category space through the first fully connected layer to obtain the category prediction result.

10. The hyperspectral image classification method for complex environments according to claim 1, characterized in that: In step S5, the test set of image data to be classified is sequentially input into the trained convolutional neural network model to obtain the category prediction result of each pixel point, and all category prediction results are spliced in the order of the original image to obtain the classification result of the drone hyperspectral image.

Images