Physical information neural network hyperspectral band selection method for target identification

Abstract

The invention discloses a physical information neural network hyperspectral band selection method for target identification, which uses physical information SDI as prior information to be combined with a channel attention mechanism added in a deep neural network to guide band selection, and evaluates the advantages and disadvantages of band weights through a reconstruction module after the weights are generated. Wave band selection is carried out accordingly; according to the hyperspectral image target recognition waveband selection method based on the physical information neural network, compared with a traditional method, the process is simplified and the precision is improved through end-to-end learning, the utilization of spatial context information can be enhanced by adding an attention mechanism, and the model has interpretability by adding prior SDI information.

Description

Technical Field

[0001] This invention relates to the field of hyperspectral target recognition of camouflaged targets, and to a method for selecting hyperspectral bands using a physical information neural network for target recognition. Background Technology

[0002] Remote sensing technology, with its global, wide-area, and continuous observation capabilities, can acquire rich information on Earth's surface targets and is widely used in the field of target identification. Its data has evolved from panchromatic, color, and multispectral images to hyperspectral images. Hyperspectral images contain hundreds of continuous spectral bands, providing detailed physical properties of ground features, enhancing the ability to detect spectral information of ground features, and distinguishing subtle differences between different types of ground features. For camouflaged targets, although their imitation of the texture, structure, and color of the background makes them difficult to identify using traditional detection methods, differences in material and structure may lead to differences in spectral characteristics between camouflaged targets and the background. The high dimensionality and high precision of hyperspectral images in the spectral domain can fully reveal these subtle spectral differences, improving the effectiveness of hyperspectral target identification.

[0003] However, processing hyperspectral images presents numerous challenges. Hundreds of bands result in a massive amount of data, leading to the "curse of dimensionality" and a dramatic increase in time complexity. Furthermore, the strong correlation between adjacent bands causes a significant amount of information redundancy. Therefore, dimensionality reduction has become a crucial step in the practical application of hyperspectral images. Band selection is a commonly used dimensionality reduction technique, the core of which lies in removing redundant bands while selecting bands with significant characteristics.

[0004] Currently, there are numerous methods for selecting hyperspectral bands, including traditional methods based on ranking, search, clustering, and sparse representation, as well as methods based on deep learning. Traditional methods have significant limitations: ranking methods independently rank and select the best bands using single-band evaluation metrics, but ignore band correlation, easily leading to high redundancy in the selected high-information bands; search methods find the optimal subset of bands through combinatorial optimization, divided into global search and heuristic search, but are computationally expensive, time-consuming, and prone to getting trapped in local optima; clustering methods cluster based on band similarity, selecting a representative band for each cluster, but the clustering results are unstable, and the cluster centers may miss edge bands with higher information content; sparse representation methods utilize the property that bands can be sparsely represented by a dictionary, selecting key bands through sparse coefficients, but have high computational complexity, and the relationship between sparse coefficients and band importance is not direct, also relying on empirical thresholds.

[0005] Deep learning methods can effectively utilize the spatial information of hyperspectral images, improving computational efficiency, performance, and generalization ability through end-to-end joint optimization, making them suitable for more complex application scenarios. However, these methods only focus on the mapping relationship between input and output, ignoring the internal structure, algorithm, and data transformation process, requiring visualization of weights to enhance interpretability. Therefore, this invention proposes a Physics-InformedNeural Networks-based Band Selection Method for hyperspectral images (PINN-BS) for target recognition. Summary of the Invention

[0006] To address the problem of purely data-driven methods in deep learning, this invention proposes a hyperspectral band selection method using a physical information neural network for target recognition. A multi-layer neural network is constructed using a DNN network, incorporating channel attention and spatial attention mechanisms. Then, a physical model of Spectral Difference Index (SDI) is proposed and fused with the channel attention mechanism to jointly guide the band selection.

[0007] The technical solution adopted by this invention to solve its technical problem is:

[0008] A hyperspectral band selection method for physical information neural networks used for target recognition, the method comprising the following steps:

[0009] Step (1) Obtain the hyperspectral dataset

[0010] Hyperspectral datasets can be obtained in two ways: one is by downloading public hyperspectral datasets, and the other is by creating your own dataset using a hyperspectral camera.

[0011] Step (2) Data preprocessing and region of interest cropping

[0012] First, save the public or self-made dataset in the standard ENVI format; second, use ENVI software to crop the region of interest in the hyperspectral image; and finally, annotate the spectral image.

[0013] Step (3) Construction of a physical information neural network

[0014] First, a deep neural network is constructed, incorporating channel attention and spatial attention mechanisms. Second, a model based on the spectral dissimilarity index, consisting of spectral gradient angle, Fréchet distance, and Pearson correlation coefficient, is built and combined with the channel attention mechanism in the deep neural network to guide band selection. The combined network consists of three modules: a feature extraction module (FEM), a spectral weighting module (SWM), and an image reconstruction module (IRM).

[0015] Step (4) Compare and analyze experimental results

[0016] Different numbers of band subsets were selected to set up control experiments. Support Vector Machine (SVM) was used as the classification model to compare and analyze the classification effects of each method. Ground Truth (GT) maps and full-band image classification results were also set up for comparison. The classification effect was analyzed by Overall Accuracy (OA) and Kappa coefficient. The larger the OA and Kappa, the better the classification effect.

[0017] Furthermore, the process of step (2) is as follows:

[0018] Step 2.1: After saving the dataset in the standard ENVI format, import the dataset into the ENVI software. In the ENVI tool interface, click "Region of Interest Tool", and in the pop-up window, click "New ROI Region Options". Manually select the region of interest on the image.

[0019] Step 2.2: After selecting the region of interest, click "Classification Image from ROIs" in the toolbar options. In the pop-up window, click "Input File", then select the previously drawn region of interest and save it as label data. Click the "Spatial Subset" option to crop the image, and click "OK". ENVI will then start the image cropping operation and generate the cropped hyperspectral data.

[0020] Furthermore, the process of step (3) is as follows:

[0021] Step 3.1: Construction of the feature extraction module

[0022] FEM directly takes the hyperspectral image as input. First, it uses several 2D convolutional layers to simultaneously extract the spectral and spatial features of the image. Then, a global pooling (GP) layer is used to reduce the spatial size of the resulting feature map. Finally, several fully connected layers are used to obtain band weights ω, which are then used for subsequent band reweighting. If FEM is represented as a function g, with the hyperspectral image x as input, it generates a non-negative band weight tensor.

[0023] ω=g(x;θ b (1)

[0024] In the formula, θ b Let represent the trainable parameters in the FEM, and b be the number of bands in the hyperspectral image x. To ensure that the learned weights are non-negative, the Sigmoid function is used as the activation function of the output layer in the FEM, as shown below:

[0025]

[0026] Where ω represents the band weights obtained above.

[0027] Step 3.2: Calculation of Spectral Dissimilarity Index

[0028] The spectral dissimilarity index I is calculated using the following formula. SD ,

[0029]

[0030] Where x and y represent two spectral vectors, θ SGA (x,y) represents the spectral gradient angle between the two spectral vectors, F(x,y) represents the Fréchet distance between the two spectral vectors, and r(x,y) represents the Pearson correlation coefficient between the two spectral vectors.

[0031] Step 3.3 Establish the spectral weighting module

[0032] SWM performs a band-by-band multiplication operation, which, as an interaction between the original input and the band weights, can be represented as follows:

[0033]

[0034] Where x represents the original hyperspectral data, and ω represents the band weights obtained in step 3.1. This represents the band-by-band multiplication operation between x and ω, while z is the product of reweighting the input x.

[0035] Furthermore, the result I obtained in step 3.2 SD It can be used as a priori parameter for the band weight ω, participating in the band weighting process. In this case, the SWM operation is expressed as...

[0036]

[0037] Step 3.4 Image Reconstruction Module Setup

[0038] Image Reconstruction Mapping (IRM) is a process that reconstructs the original image from the reweighted input x and output z. It is implemented by a convolutional encoder that extracts deep features and a deconvolutional decoder that upsamples the feature map. The reconstruction process is defined as a function f, with z as input, and its predicted output is... for

[0039]

[0040] In the formula, θ c These are trainable parameters during the reconstruction process.

[0041] To measure reconstruction performance, the mean squared error L is used as the cost function, as shown below:

[0042]

[0043] Where S is the number of training samples. Furthermore, to make the band weights as sparse as possible, an L1 norm constraint is applied, resulting in the loss function L(θ). b ,θ c As shown below:

[0044]

[0045] In the formula, λ is the regularization coefficient, used to balance the reconstruction error and the regularization term. Stochastic gradient descent is used here to optimize the loss function.

[0046] By averaging the weights learned from all training samples for different bands, we can evaluate the importance of each band. The average weight of the j-th band... The calculation is as follows:

[0047]

[0048] Among them, bands with higher average weights contribute more to the reconstruction, are considered more important, and are more likely to be selected as representative feature bands. Ultimately, we select the top k bands as the subset of feature bands.

[0049] The design concept of this invention is as follows:

[0050] This invention utilizes physical information SDI as prior information and combines it with the channel attention mechanism added to the deep neural network to guide the selection of bands. After generating weights, a reconstruction module is used to evaluate the quality of the band weights, and band selection is performed accordingly.

[0051] The beneficial effects of this invention are mainly reflected in the following aspects: This invention proposes a hyperspectral image target recognition band selection method based on physical information neural network. Compared with traditional methods, it simplifies the process and improves accuracy through end-to-end learning. The addition of attention mechanism can enhance the utilization of spatial context information, and the addition of prior SDI information makes the model interpretable. Attached Figure Description

[0052] Figure 1 This is a flowchart of the present invention;

[0053] Figure 2 It is a public dataset;

[0054] Figure 3 It represents the overall accuracy of each method on a public dataset with different numbers of bands.

[0055] Figure 4 This is a graph showing the classification results of various methods on a public dataset. Detailed Implementation

[0056] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention.

[0057] Conversely, this invention encompasses any substitutions, modifications, equivalent methods, and solutions made within the spirit and scope of the invention as defined in the claims. Furthermore, to provide a better understanding of the invention, certain specific details are described in detail below. However, those skilled in the art will fully understand the invention even without these detailed descriptions.

[0058] The present invention will be further described with reference to the accompanying drawings. A method for selecting bands for hyperspectral image target recognition based on a physical information neural network includes the following steps:

[0059] (1) Obtain the hyperspectral dataset

[0060] To effectively demonstrate the performance of the PINN-BS model proposed in this invention, a public dataset, the San Diego hyperspectral dataset, was used as input.

[0061] (2) Data preprocessing and region of interest cropping

[0062] Step 2.1: After saving the dataset in the standard ENVI format, import the dataset into the ENVI software. In the ENVI tool interface, click "Region of Interest Tool", and in the pop-up window, click "New ROI Region Options". Manually select the region of interest on the image.

[0063] Step 2.2: After selecting the region of interest, click "Classification Image from ROIs" in the toolbar options. In the pop-up window, click "Input File", then select the previously drawn region of interest and save it as label data. Click the "Spatial Subset" option to crop the image, and click "OK". ENVI will then start the image cropping operation and generate the cropped hyperspectral data.

[0064] (3) Construction of a physical information neural network

[0065] Step 3.1: Construction of the feature extraction module

[0066] FEM directly takes the hyperspectral image as input. First, it uses several 2D convolutional layers to simultaneously extract the spectral and spatial features of the image. Then, a global pooling (GP) layer is used to reduce the spatial size of the resulting feature map. Finally, several fully connected layers are used to obtain band weights ω, which are then used for subsequent band reweighting. If FEM is represented as a function g, with the hyperspectral image x as input, it generates a non-negative band weight tensor.

[0067] ω=g(x;θ b (1)

[0068] In the formula, θ b Let represent the trainable parameters in the FEM, and b be the number of bands in the hyperspectral image x. To ensure that the learned weights are non-negative, the Sigmoid function is used as the activation function of the output layer in the FEM, as shown below:

[0069]

[0070] Where ω represents the band weight obtained above.

[0071] Step 3.2: Calculation of Spectral Dissimilarity Index

[0072] The spectral dissimilarity index I is calculated using the following formula. SD ,

[0073]

[0074] Where x and y represent two spectral vectors, and θ SGA(x,y) represents the spectral gradient angle between the two spectral vectors, F(x,y) represents the Fréchet distance between the two spectral vectors, and r(x,y) represents the Pearson correlation coefficient between the two spectral vectors.

[0075] Step 3.3 Establish the spectral weighting module

[0076] SWM performs a band-by-band multiplication operation, which, as an interaction between the original input and the band weights, can be represented as follows:

[0077]

[0078] Where x represents the original hyperspectral data, and ω represents the band weights obtained in step 3.1. This represents the band-by-band multiplication operation between x and ω, while z is the product of reweighting the input x.

[0079] Furthermore, the result I obtained in step 3.2 SD It can be used as a priori parameter for the band weight ω, participating in the band weighting process. In this case, the SWM operation is expressed as...

[0080]

[0081] Step 3.4 Image Reconstruction Module Setup

[0082] Image Reconstruction Mapping (IRM) is a process that reconstructs the original image from the reweighted input x and output z. It is implemented by a convolutional encoder that extracts deep features and a deconvolutional decoder that upsamples the feature map. The reconstruction process is defined as a function f, with z as input, and its predicted output is... for

[0083]

[0084] In the formula, θ c These are trainable parameters during the reconstruction process.

[0085] To measure reconstruction performance, the mean squared error L is used as the cost function, as shown below:

[0086]

[0087] Where S is the number of training samples. Furthermore, to make the band weights as sparse as possible, an L1 norm constraint is applied, resulting in the loss function L(θ). b ,θ c As shown below:

[0088]

[0089] In the formula, λ is the regularization coefficient, used to balance the reconstruction error and the regularization term. Stochastic gradient descent is used here to optimize the loss function.

[0090] By averaging the weights learned from all training samples for different bands, we can evaluate the importance of each band. The average weight of the j-th band... The calculation is as follows:

[0091]

[0092] Among them, bands with higher average weights contribute more to the reconstruction, are considered more important, and are more likely to be selected as representative feature bands. Ultimately, we select the top k bands as the subset of feature bands.

[0093] (4) Comparison of experimental results and analysis

[0094] Band subsets with 10, 20, 30, 40, 50, and 60 bands were selected as input. Support Vector Machine (SVM) was used as the classification model. The classification performance of the SDI model, Optimal Clustering Framework (OCF), Band Selection Network (BS_Net), Genetic Algorithm (GA), Spectral Angle Mapper (SAM), normalized Fréchet distance (F), and all bands was compared and analyzed. Figure 4 The classification results were compared with those of the Ground Truth (GT) map and the full-band image. The classification effect was analyzed by the Overall Accuracy (OA) and Kappa coefficient. The larger the OA and Kappa, the better the classification effect. Figure 3 Table 1 shows the classification accuracy curves of each method on two datasets, and records the SVM classification results on three datasets when the number of band selections is 50.

[0095] Table 1 Comparison and analysis of the classification effects of various methods

[0096]

[0097] From Table 1 and Figure 3As can be seen, in the public dataset of San Diego airport, the PINN-BS method exhibits the best performance with a Kappa coefficient of 84.64%. Its overall accuracy of 98.35% is also second best, only slightly lower than the SDI method's 98.65%, and the difference between the two is negligible. Both are higher than other methods and the overall accuracy of 98.10% across the entire band. Figure 4 As can be seen in the classification results of the OCF, GA and SAM methods, misclassification occurs. The classification results of the F and BS_Net methods show a large number of noise points, while the PINN-BS method does not show these two issues. The above results indicate that the PINN-BS method can select a higher quality band subset.

[0098] Table 2 Abbreviations, Terms and Explanations:

[0099]

[0100]

[0101] The embodiments described in this specification are merely examples of implementations of the inventive concept. The scope of protection of this invention should not be considered as limited to the specific forms stated in the embodiments. The scope of protection of this invention also extends to equivalent technical means that can be conceived by those skilled in the art based on the inventive concept.

Claims

1. A hyperspectral band selection method for physical information neural networks used in target recognition, characterized in that, Includes the following steps: Step 1) Obtain the hyperspectral dataset: Obtain hyperspectral datasets through public or self-made datasets; Step 2) Data preprocessing and region of interest cropping: Save the acquired hyperspectral dataset in standard ENVI format; The hyperspectral image was cropped using ENVI software, and the spectral image was annotated. Step 3) Construction of a physical information-based neural network: A deep neural network was constructed and channel attention and spatial attention mechanisms were incorporated into it; a model based on the spectral dissimilarity index, consisting of spectral gradient angle, Fréchet distance and Pearson correlation coefficient, was constructed and combined with the channel attention mechanism in the deep neural network to jointly guide the selection of bands; Step 4) Compare and analyze the experimental results: Different numbers of band subsets were selected to set up control experiments. Support Vector Machine (SVM) was used as the classification model to compare and analyze the classification effects of each method. At the same time, ground truth maps and full-band image classification results were set up for comparison. The classification effect was analyzed by overall accuracy (OA) and Kappa coefficient.

2. The method for selecting hyperspectral bands in a physical information neural network for target recognition according to claim 1, characterized in that, The physical information neural network is divided into three modules: feature extraction module (FEM), spectral weighting module (SWM), and image reconstruction module (IRM).

3. The method for selecting hyperspectral bands in a physical information neural network for target recognition according to claim 1, characterized in that, The process of step 3) is as follows: Step 3.1: Construction of the feature extraction module: The FEM takes the hyperspectral image as input directly. First, it uses several 2D convolutional layers to simultaneously extract the spectral and spatial features of the image. Then, a global pooling (GP) layer is used to reduce the spatial size of the resulting feature map. Finally, multiple fully connected layers are used to obtain band weights ω, which are then used for subsequent band reweighting. If the FEM is represented as a function g, with the hyperspectral image x as input, a non-negative band weight tensor is generated. ω=g(x;θ b ) (1) In the formula, θ b This represents the trainable parameters in the FEM, where b is the number of bands in the hyperspectral image x. To ensure that the learned weights are non-negative, the Sigmoid function is used as the activation function of the output layer in FEM, as shown below: Where ω represents the band weights obtained above; Step 3.2: Calculation of Spectral Dissimilarity Index: The spectral dissimilarity index I is calculated using the following formula. SD ; Where, θ SGA (x,y) represents the spectral gradient angle between two spectral vectors, F(x,y) represents the Fréchet distance between the two spectral vectors, and r(x,y) represents the Pearson correlation coefficient between the two spectral vectors. Step 3.3: Establish the spectral weighting module: SWM performs a band-by-band multiplication operation as an interaction between the original input and the band weights, represented as: Where x represents the original hyperspectral data, and ω represents the band weights obtained in step 3.

1. This represents the band-by-band multiplication operation between x and ω, while z is the product of reweighting the input x. The result I obtained in step 3.2 SD As a priori parameter of the band weight ω, it participates in the band weighting process; at this time, the SWM operation is expressed as: Step 3.4 Image Reconstruction Module Setup: IRM is a process that reconstructs the original image from the reweighted output z of the input x. It is implemented by a convolutional encoder that extracts deep features and a deconvolutional decoder that upsamples the feature map. The reconstruction process is defined as a function f, with z as input, and its predicted output is... for In the formula, θ c These are trainable parameters during the reconstruction process; To measure reconstruction performance, the mean squared error L is used as the cost function, as shown below: Where S is the number of training samples, x i and These represent the input and predicted output for the i-th band, respectively. In addition, to make the band weights as sparse as possible, an L1 norm constraint is applied to them, and the loss function L(θ) b ,θ c As shown below: In the formula, λ is the regularization coefficient, used to balance the reconstruction error and the regularization term, and ω i The weights for the i-th band are used, and the loss function is optimized using stochastic gradient descent. The importance of each band is evaluated by averaging the weights learned from all training samples for each band; the average weight of the j-th band... The calculation is as follows: Among them, the bands with larger average weights contribute more to the reconstruction, are considered more important, and are more likely to be selected as representative feature bands. The top k bands are selected as the feature band subset.

Images