A Hyperspectral Camouflage Target Detection Method Based on Spatial-Spectral Feature Fusion
By combining deep learning and spatial spectral feature extraction into a 3D network model, the problems of insufficient model expressive power and redundant information in hyperspectral camouflage target detection are solved, achieving high-precision and efficient camouflage target detection.
Patent Information
- Authority / Receiving Office
- CN · China
- Patent Type
- Patents(China)
- Current Assignee / Owner
- XIAN AERONAUTICAL UNIV
- Filing Date
- 2022-11-16
- Publication Date
- 2026-05-26
AI Technical Summary
Existing hyperspectral image camouflage target detection methods are not effective in tasks involving the detection of unknown target spectral information. Traditional methods suffer from problems such as insufficient model expressive power, insufficient real-time performance, and redundant information affecting detection efficiency.
A hyperspectral camouflage target detection method based on spatial-spectral feature fusion is adopted. A 3D network model is established by combining deep learning and spatial-spectral feature extraction. The 3D convolution and spatial receptive field model are used for feature extraction and target localization. The detection results are optimized by training method to remove false positives/negatives.
It improves the accuracy and efficiency of hyperspectral camouflage target detection, reduces false positives and false negatives, and achieves more accurate camouflage target detection.
Smart Images
Figure CN115731463B_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of hyperspectral image recognition and classification technology, specifically relating to a hyperspectral camouflage target detection method based on spatial-spectral feature fusion. Background Technology
[0002] In recent years, deep learning algorithms have garnered significant attention due to their immense potential in feature extraction and learning, and have been effectively applied in fields such as hyperspectral image recognition and classification. Currently, airborne and spaceborne sensors cover vast areas of the Earth's surface with unprecedented hyperspectral, large spatial, and fine temporal resolution. These characteristics enable numerous applications requiring precise identification of target materials and estimation of physical parameters, making research into related technologies for hyperspectral images a major trend. Hyperspectral images possess both spatial and spectral information. Spectral characteristics are unique to hyperspectral images; under the same spectrometer, each substance exhibits unique spectral properties. This can significantly enhance the information richness of target information in target classification and recognition, thereby improving target detection accuracy. Target detection can also be understood as a binary classification, namely target and background. Deep learning methods have a strong advantage in feature extraction of target information, fully leveraging the spectral and spatial information of hyperspectral images.
[0003] In the existing technology, Wu Yinhua et al. proposed a hyperspectral fast target detection method based on compressed spectral imaging. This method does not require reconstructing the data cube and directly applies the CEM operator to the system, thus avoiding the problem of large data volume. However, since it relies on prior information, it cannot avoid problems in the detection task with unknown target spectral information.
[0004] Wang Yulei et al. proposed a hyperspectral image target detection method based on self-supervised contrastive learning. By maximizing the similarity of positive pairs while minimizing the similarity of negative pairs, the method learns the similarity and dissimilarity of spectra and clusters, thereby increasing the difference between target and background vectors.
[0005] With the widespread application of neural networks, some researchers have used the differences between neighboring pixel values in an image as input to train a network. After multiple layers of convolution and feedback, a trained convolutional neural network (CNN) is obtained, and finally, anomalies are detected by estimating the difference values of test data. However, while simple one-dimensional and two-dimensional convolutional neural networks, such as Super CNN, MAP, SSD, RACDNN, and MSRNet, have solved certain problems in hyperspectral feature extraction, their expressive power is insufficient to handle more complex image data. Furthermore, most of these methods are based on general object detection or salient object detection, with little analysis specifically for camouflaged object detection.
[0006] Zhang Xiuwei et al. proposed a hyperspectral camouflage target detection method based on block diagonal and low-rank representation. The core idea is to use traditional clustering algorithms to refine the background description and then extract the camouflage target. However, this method still has shortcomings: the use of clustering to classify the background leads to insufficient real-time performance; in addition, to achieve a refined description of the background, the extracted spectral features contain a large amount of redundant information, affecting the detection efficiency. Summary of the Invention
[0007] The purpose of this invention is to solve the above-mentioned problems. This application proposes a hyperspectral camouflage target detection method based on spatial-spectral feature fusion. To fully utilize the spectral features in the image, this invention combines deep learning and spatial-spectral feature extraction to establish a 3D network model suitable for spectral feature learning and target spatial localization. Anomaly detection and spatial-spectral constraints are added to the model to improve detection accuracy. Simultaneously, a false positive / negative removal training method based on spatial receptive domain is proposed to further adjust the network and improve its training effect. To achieve the above objectives, this invention provides the following technical solution: a hyperspectral camouflage target detection method based on spatial-spectral feature fusion, the specific detection method being as follows:
[0008] Principal component analysis (PCA) was used to extract bands in the spectral space, and three-dimensional convolution was employed to capture spatial-spectral feature information of multiple bands, including: the hyperspectral data cube representation after principal component decomposition. Where H is the height of the spectral image, W is the width of the spectral image, and D' is the number of spectral bands after principal component decomposition;
[0009] To enhance the feature information in both the spectral and spatial dimensions, the hyperspectral data after PCA decomposition is fused based on RXD and three-dimensional spatial-spectral features. As input to the network framework, the spectral features are extracted by a three-dimensional spatial spectral feature extraction module, which contains four three-dimensional convolutional layers of different sizes. Conv 3 ( k ×3×3), the kernel depths from largest to smallest are: k = (7, 5, 3, 1), and use the output of each convolution as the input of the next layer. During the operation of each convolution, the dilation rate is set to 2 to increase the kernel coverage area. Furthermore, a ReLU nonlinear layer is followed by each convolution to remove negative values from pixels, thus obtaining the spatial spectral feature map. ;
[0010] The processing method is as follows: Dimensional reshaping is performed to restore the spatial dimensions of the feature map. Then, using a spatial receptive field model, initial localization of the camouflaged target is achieved through receptive fields of multiple sizes. Specifically:
[0011] First, the spatial spectral features after deformation A 1×1 convolutional layer is applied to the result for non-linear processing, and four spatial receptive field branches of different sizes are set. k Each branch contains (2) k -1)×1 and 1×(2 k -1) Two symmetrical convolutional layers, followed by a convolutional kernel of 3. 3 and the expansion rate is (2) k -1) dilated convolutions are used to obtain a larger receptive field, and the outputs of the four branches are integrated and added to the original input features. Finally, features with multi-scale range perception capabilities are obtained. This outputs the initial detection results of the camouflaged target. ,
[0012] ;in, This indicates the size of the current convolution kernel. And the expansion rate is b Hollow convolution.
[0013] Furthermore, to improve the accuracy of the 3D spatial-spectral feature fusion algorithm, a false positive / negative removal training method is set up to lock the target. Given the spectral and spatial features, the reinforcement learning stage is used to obtain the features. and initial The feature enhancement of the target region is further achieved by employing pixel-wise multiplication, which reduces intra-class differences and increases inter-class differences. Specifically:
[0014] Will The enhanced camouflage target features are obtained by multiplying them by the initial target segmentation and the initial background segmentation, respectively. and background features The target locking module explores the spatial context information of both the camouflaged target and the background to enrich feature learning and training. Finally, it removes false positive backgrounds and adds false negative target pixels to the camouflage detection results, correcting the blurred edges of the camouflaged target. The locking module comprises four feature layers. n , n =1,2,3,4), iteratively refine the camouflage target outline layer by layer. Specifically, it is expressed as
[0015] ;in, X This is the channel concatenation result of the relevant feature maps obtained through different types of convolution operations. The feature maps of the four branches are as follows:
[0016] ;in, LMg ( g =1,2,3,4) are used for convolution operations, each... LMg It is a sequential operation on the input features, first including (2g-1). (2 g A convolutional layer of -1), each layer ( g Layer, g=1,2,3,4) followed by a dilated convolution with an inflation rate of (2 g -1), the kernel size is 3 A convolutional layer of 3 is used, followed by batch normalization and the ReLU function for further processing.
[0017] Furthermore, to enable the network to dynamically change with the input feature tensor during training, two learning scaling parameters are added during the refinement of detection results. , (If the initial training value is set to 1), then the output feature have,
[0018] Among them, camouflaged target characteristics and background features Specifically, this can be expressed as follows:
[0019] ; where lm( This indicates that the LM module is used to enrich the feature's ability to perceive the scope of the context. The feature represents the first i Layers The convolutional layer is then used with an inflation coefficient of 2. i 3 Three convolutional layers expand the sensing range, and finally a ReLU nonlinear layer is added to output the updated feature tensor.
[0020] Compared with the prior art, the beneficial effects of the present invention are as follows:
[0021] (1) This invention proposes a hyperspectral camouflage target detection algorithm based on RXD (Reed-Xiao Detector) and three-dimensional spatial-spectral feature fusion (RX-3DSSRF) from the spectral dimension. Three-dimensional convolution is used to capture the "spatial-spectral" features of multiple bands in a local range at the same time, so that the network can learn more representative feature information layer by layer in advance; at the same time, from the spatial dimension, a spatial receptive field model (SRF) is proposed, which can enhance the spatial feature receptive range by using multi-scale receptive fields, and then combine the RXD abnormal target detection results to obtain the potential location of the camouflage target. Experimental results show that the RX-3DSSRF algorithm can better make up for the problems of missed detection and false alarm in the traditional RXD algorithm, and can obtain good segmentation results on low-altitude hyperspectral camouflage target detection data. (2) In order to further improve the detection accuracy of the RX-3DSSRF algorithm, a false positive / false negative removal training method is set up to lock the target. By utilizing the complementary logical relationship between the target and the background, the LM module is used to explore the features of both in depth. By removing the false positive features of the target and the false negative features of the background, the final optimized target detection result is obtained. Attached Figure Description
[0022] To more clearly illustrate the technical solutions of the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only for more clearly illustrating the technical solutions in the embodiments of the present invention or the prior art. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0023] Figure 1 This is a schematic diagram of the three-dimensional spatial spectrum feature fusion algorithm framework of the present invention;
[0024] Figure 2 This is a schematic diagram of the LM module structure of the present invention. Detailed Implementation
[0025] To enable those skilled in the art to better understand and implement the technical solutions of the present invention, the present invention will be further described below with reference to specific embodiments. However, the embodiments are only for illustration and are not intended to limit the present invention.
[0026] For an entire hyperspectral image, not all massive amounts of data can be used to enhance the effectiveness of the information. The computational process also becomes more complex with the increase in data volume. Within a certain local space, ground features exhibit spatial continuity, resulting in high similarity in the spectral information of adjacent pixels, thus leading to spatial redundancy. Moreover, hyperspectral images typically contain hundreds of spectral bands. Taking a single pixel as an example... , DThe discrete points of each band can be fitted to form an approximately continuous spectral curve. Therefore, information from some bands can be estimated from other bands, resulting in spectral redundancy. In summary, to detect camouflaged targets in hyperspectral images, Principal Component Analysis (PCA) is used to extract bands in the spectral space. This reduces spectral redundancy while retaining all spatial feature information, providing support for subsequent target edge refinement.
[0027] The hyperspectral data cube after principal component decomposition is represented as: , H The height of the spectral image, W The spectral image is the width. This represents the number of spectral bands after principal component decomposition. Commonly used two-dimensional convolutions can effectively extract features at different levels in the spatial dimension, thereby obtaining effective spatial information from the feature mapping tensor.
[0028] Furthermore, research shows that lower-dimensional spatial features can retain more edge information, thus being more conducive to the identification and refinement of the potential location of camouflaged targets. However, two-dimensional convolution cannot be applied in the spectral dimension. Therefore, a hyperspectral camouflage target detection algorithm based on RXD (Reed-Xiao Detector) and three-dimensional spatial-spectral feature fusion (RX-3 Dimension Spatial Spectralfeature Spatial Receptive Field, RX-3DSSRF) is proposed. Specifically, three-dimensional convolution is used to simultaneously capture "spatial-spectral" feature information of multiple bands. Based on fully utilizing the superior learning ability of three-dimensional convolution on target features in the image, two-dimensional convolution is then used to achieve edge localization of the target, finally obtaining a two-dimensional discriminant map of the target.
[0029] To enhance the representative feature information in both the spectral and spatial dimensions, the hyperspectral data after PCA decomposition... As input to the network framework, the data first passes through a 3D Spatial Spectral Feature (3DSSF) module, the purpose of which is to obtain discriminative spectral feature information. The detailed structure of the 3DSSF module is as follows... Figure 1 As shown in (b), this module contains four three-dimensional convolutional layers of different sizes. Conv 3( k ×3×3), the kernel depths from largest to smallest are: k=(7,5,3,1), and use the output of each convolutional layer as the input of the next layer to gradually extract more representative features. Meanwhile, to broaden the feature acceptance range, the dilation rate of each convolutional layer is set to 2 during computation to increase the kernel coverage area, and a ReLU nonlinear layer is followed by all convolutions to remove negative values from pixels, thus obtaining the spatial spectral feature map. The 3DSSF module aims to combine spatial and spectral correlations to achieve feature extraction of hyperspectral camouflage targets.
[0030] Without considering training batches for the time being It is a four-dimensional tensor. By reshaping its dimensions, the spatial dimensions of the feature map are restored. Then, using a Spatial Receptive Field (SRF) model, the camouflaged target is initially located using receptive fields of multiple sizes. Figure 1 As shown in (a), firstly, the spatial spectral features after deformation... Application 1 A convolutional layer of size 1 is used to perform non-linear processing on the results, and four spatial receptive field branches of different sizes are set. k Each branch contains (2) k -1) 1 and 1 (2 k -1) Two symmetrical convolutional layers, followed by a convolutional kernel of 3. 3 and the expansion rate is (2) k -1) dilated convolutions are used to obtain a larger receptive field, and the outputs of the four branches are integrated and added to the original input features. Finally, features with multi-scale range perception capabilities are obtained. This outputs the initial detection results of the camouflaged target. ,
[0031] ;in, This indicates the size of the current convolution kernel. And the expansion rate is b Hollow convolution.
[0032] Given the spectral and spatial characteristics, the reinforcement learning stage yields the features. and initial camouflaged target detection map This approach employs pixel-by-pixel multiplication to further highlight the features of the target region, thereby reducing intra-class discrepancies and increasing inter-class discrepancies. Specifically, the initial camouflaged target image... It is a blurry, camouflaged target location map. The potential target area it represents, especially near the edges, contains a large amount of false positive and false negative information. The enhanced camouflage target features are obtained by multiplying them by the initial target segmentation and the initial background segmentation, respectively. and background features Combined with the target locking module LM, the LM locking module is as follows: Figure 2 As shown, the spatial context information of the camouflaged target and background is explored to enrich feature learning and training. Finally, false positive backgrounds are removed and false negative target pixels are added to the camouflage detection results, and the blurred edges of the camouflaged target are corrected. Specifically, the Locking Module (LM) combines four feature layers. n , n =1,2,3,4), iteratively refine the camouflage target outline layer by layer. Specifically, it can be expressed as,
[0033] ;in, X This is the channel concatenation result of the relevant feature maps obtained through different types of convolution operations. The feature maps of the four branches are as follows:
[0034] ;in, LMg ( g =1,2,3,4) are used for convolution operations, each... LMg It is a sequential operation on the input features, first including (2) g -1) (2 g A convolutional layer of -1), each layer ( g layer, g =1,2,3,4) followed by a dilated convolution with an inflation rate of (2 g -1), the kernel size is 3 A 3-channel convolutional layer is used, followed by batch normalization and ReLU function processing. Each feature map, after channel reduction, has its receptive field increased by dilation rates at different scales. The four branches are concatenated and global feature information is aggregated. Finally, by combining deeper-level features, the prediction of camouflaged targets is completed, such as... Figure 2 As shown in the diagram. It's worth noting that the input feature layer of each locking module gradually incorporates more edge contour information from deep to shallow. Therefore, the information iteration to obtain the final camouflage target detection result is also dynamic, and as the feature layer becomes shallower, the difference between the target and the background becomes more pronounced, resulting in a more accurate camouflage target detection result.
[0035] To enable the network to dynamically change with the input feature tensor during training, this chapter incorporates two learnable scaling parameters during the refinement of detection results. , (If the initial training value is set to 1), then the output feature have,
[0036] Among them, camouflaged target characteristics and background features Specifically, this can be expressed as follows:
[0037] ; where lm( This indicates that the LM module is used to enrich the feature's ability to perceive the scope of the context. The feature represents the first i Layers The convolutional layer is then used with an inflation coefficient of 2. i 3 Three convolutional layers expand the sensing range, and finally a ReLU nonlinear layer is added to output the updated feature tensor.
[0038] Research has shown that low-dimensional features possess richer detailed features, which can be used for accurate target edge detection. Therefore, this characteristic is utilized in the context feature learning process by adding learnable parameters to the 3rd and 4th layers of the LM module. This method achieves the goal of focusing on learning low-dimensional feature layers during training. Finally, after sampling operations and combining RXD features, it outputs a comprehensive detection result for camouflaged targets. A schematic diagram of the detection result is shown below. Figure 3 As shown.
[0039] This invention fully utilizes the spectral features in images, combining deep learning and spatial spectral feature extraction to establish a 3D network model suitable for spectral feature learning and target spatial localization. Anomaly detection and spatial spectral constraints are incorporated into the model to improve detection accuracy. Simultaneously, a false positive / negative removal training method based on spatial receptive domain is proposed to further adjust the network and improve its training effect.
[0040] All content not described in detail in this invention is prior art.
[0041] The above description is merely a preferred embodiment of the present invention and is not limited to the description in the specification and embodiments. Therefore, all equivalent changes or modifications made to the structure, features, and principles described in the claims of this invention should be included within the scope of this patent application.
Claims
1. A hyperspectral camouflage target detection method based on spatial-spectral feature fusion, characterized in that: The specific testing methods are as follows: Principal component analysis (PCA) was used to extract bands in the spectral space, and three-dimensional convolution was employed to capture spatial-spectral feature information of multiple bands, including: the hyperspectral data cube representation after principal component decomposition. ,in H The height of the spectral image, W The width of the spectral image. The number of spectral bands after principal component decomposition; to enhance the feature information in the spectral and spatial dimensions, the hyperspectral data after PCA decomposition is fused based on RXD and three-dimensional spatial-spectral features. As input to the network framework, the spectral features are extracted by a three-dimensional spatial spectral feature extraction module, which contains four three-dimensional convolutional layers of different sizes. Conv 3( k ×3×3), the kernel depths from largest to smallest are: k =(7,5,3,1), and use the output of each convolutional layer as the input of the next layer. During the computation of each convolutional layer, the dilation rate is set to 2 to increase the kernel coverage area. Furthermore, a ReLU nonlinear layer is followed by each convolution to remove negative values from pixels, thus obtaining the spatial spectral feature map. ; The processing method is as follows: Dimensional reshaping is performed to restore the spatial dimensions of the feature map. Then, using a spatial receptive field model, initial localization of the camouflaged target is achieved through receptive fields of multiple sizes. Specifically: First, the spatial spectral features after deformation Application 1 A convolutional layer of size 1 is used to perform non-linear processing on the results, and four spatial receptive field branches of different sizes are set. k Each branch contains (2) k -1) 1 and 1 (2 k -1) Two symmetrical convolutional layers, followed by a convolutional kernel of 3. 3 and the expansion rate is (2) k -1) dilated convolutions are used to obtain a larger receptive field, and the outputs of the four branches are integrated and added to the original input features. Finally, features with multi-scale range perception capabilities are obtained. This outputs the initial detection results of the camouflaged target. , ;in, This indicates the size of the current convolution kernel. And the expansion rate is b Aperture convolution; To improve the accuracy of the 3D spatial-spectral feature fusion algorithm, a false positive / negative removal training method is set up to lock the target. Given the spectral and spatial features, the reinforcement learning stage is used to obtain the features. and initial The feature enhancement of the target region is further achieved by employing pixel-wise multiplication, which reduces intra-class differences and increases inter-class differences. Specifically: Will The enhanced camouflage target features are obtained by multiplying them by the initial target segmentation and the initial background segmentation, respectively. and background features The target locking module explores the spatial context information of both the camouflaged target and the background to enrich feature learning and training. Finally, it removes false positive backgrounds and adds false negative target pixels to the camouflage detection results, correcting the blurred edges of the camouflaged target. The locking module comprises four feature layers. n , n =1,2,3,4), iteratively refine the camouflage target outline layer by layer. Specifically, it is expressed as ;in, X This is the channel concatenation result of the relevant feature maps obtained through different types of convolution operations. The feature maps of the four branches are as follows: ;in, LMg ( g =1,2,3,4) are used for convolution operations, each... LMg It is a sequential operation on the input features, first including (2) g -1) (2 g A convolutional layer of -1), each layer ( g layer, g =1,2,3,4) followed by a dilated convolution with an inflation rate of (2 g -1), the kernel size is 3 A convolutional layer of 3 is used, followed by batch normalization and the ReLU function for further processing.
2. The hyperspectral camouflage target detection method based on spatial-spectral feature fusion according to claim 1, characterized in that: To enable the network to dynamically change with the input feature tensor during training, two learning scaling parameters are added during the refinement of detection results. , (If the initial training value is set to 1), then the output feature have, Among them, camouflaged target characteristics and background features Specifically, this can be expressed as follows: ; where lm( This indicates that the LM module is used to enrich the feature's ability to perceive the scope of the context. The feature represents the first i Layers The convolutional layer is then used with an inflation coefficient of 2. i 3 Three convolutional layers expand the sensing range, and finally a ReLU nonlinear layer is added to output the updated feature tensor.