Hyperspectral anomaly detection method based on dual spectrum affinity constraint
By using dual spectral affinity constraint method in hyperspectral anomaly detection, the spectral relationship between the central pixel and the surrounding pixel is captured, and the problem of limited detection performance in the prior art is solved, achieving higher detection accuracy and robustness.
Patent Information
- Application Number
- CN202510118945.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-01-24
- Publication Date
- 2025-05-06
AI Technical Summary
The existing hyperspectral anomaly detection methods have shortcomings in capturing the unique spectral characteristics of the central pixel itself and the spectral correlation with surrounding pixels, resulting in limited detection performance.
Using a hyperspectral anomaly detection method based on dual spectral affinity constraints, the unique spectral characteristics of each pixel and their spectral affinity relationship with neighboring pixels are captured by constructing a dual feature encoder and dual spectral constraint module.
It effectively improves the accuracy and robustness of abnormal detection, especially in complex backgrounds and weak abnormal scenarios, and can more accurately distinguish abnormal areas from background areas.
Smart Images
Figure CN119942347A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the technical field of hyperspectral anomaly detection, and in particular relates to a hyperspectral anomaly detection method based on dual spectral affinity constraints. Background Art
[0002] Hyperspectral anomaly detection (HAD) refers to the identification of anomalies that are significantly different from the background without prior knowledge of abnormal spectra. In this field, the use of deep learning networks, especially reconstruction-based methods, has become one of the main technologies for solving HAD tasks. These methods reconstruct the original hyperspectral images and extract the potential key features, thereby improving the accuracy of anomaly detection. However, most of the existing reconstruction-based technologies still have certain shortcomings:
[0003] 1) Existing methods usually use data patches as input, which can provide spatial correlation information between the central pixel and the surrounding pixels, but ignore the unique spectral characteristics of the central pixel itself. This approach may lead to insufficient feature expression of abnormal pixels, thus limiting detection performance.
[0004] 2) Most reconstruction-based methods only focus on the individual characteristics of each pixel, while ignoring the spectral correlation and correlation relationship between the central pixel and its surrounding pixels. This limitation may cause abnormal targets to be incorrectly reconstructed as background distribution, thereby reducing the accuracy of detection.
[0005] Specifically, although the existing reconstruction-based methods can effectively extract potential features and improve the accuracy of anomaly detection, they fail to fully utilize the spectral constraints between patches during the modeling process, resulting in deficiencies in capturing spatial spectral synergy characteristics, limiting reconstruction accuracy and detection effects. There are currently several mainstream hyperspectral anomaly detection methods:
[0006] The DirectNet method establishes an internal window within the network receptive field by erasing the central block information, so that the content of the internal window remains invisible during the reconstruction of the central pixel. The receptive field outside the inner window is considered to be the outer window. However, this method has the defect that completely erasing the central block information may cause the loss of local details, reducing the ability to detect fine abnormal areas. The GT-HAD method can effectively distinguish between background and abnormalities by using the spatial spectral correlation in hyperspectral images. The two branches model the features of the background and abnormalities respectively. However, this method cannot effectively distinguish between background and abnormal features in high-noise or complex scenes.
[0007] Patent CN118334513A (“Hyperspectral Anomaly Detection Method Based on Deep Feature Aggregation”) introduces a deep feature aggregation network model that fully utilizes the spatial-spectral information of hyperspectral images. The multiple aggregation separation loss functions are designed to enable the model to enhance the representation of potential background features and weaken the representation of potential abnormal features, thereby achieving hyperspectral anomaly detection. However, while this method enhances the representation of background features, it excessively weakens the representation of abnormal features, resulting in insufficient detection capabilities for weak anomalies.
[0008] Patent CN118154541A ("A hyperspectral anomaly detection method based on improved dual-layer collaborative structure") proposes a dual-layer collaborative representation structure, which pre-detects most of the abnormal points through the first-layer collaborative representation algorithm, and uses the neighborhood mean vector to fill in the background for background purification, effectively reducing the pollution of the background by the abnormal points. The hyperspectral image after background purification is then reconstructed, and the anomaly detection result can be obtained using the reconstruction error. In this method, the pre-detection performance directly affects the subsequent processing. If there are omissions or false detections in the pre-detection, the overall detection effect will be significantly reduced.
[0009] In the field of hyperspectral anomaly detection, the existing technology uses patches as input. Although this processing method can provide spatial correlation information between the central pixel and the surrounding pixels, it ignores the unique spectral characteristics of the central pixel itself. Specifically, this method uses image patches as input, models the spatial correlation information between the central pixel and its surrounding pixels as the main feature, and emphasizes the contribution of spatial neighborhood to anomaly detection. However, this processing method often mixes the spectral characteristics of the central pixel with those of the surrounding pixels, weakening the focus on the unique spectral characteristics of the central pixel itself. This may cause abnormal pixels to be misjudged as background due to their spectral characteristics being similar to those of the neighborhood, or to be difficult to accurately identify due to the small spatial distribution of abnormal targets. Therefore, in hyperspectral anomaly detection, ignoring the spectral characteristics of the central pixel may have an adverse effect on the robustness and accuracy of the detection results.
[0010] In addition, most current reconstruction-based methods only focus on the individual characteristics of each pixel, treating each pixel as an independent individual for reconstruction, while ignoring the spectral correlation and spatial relationship between the central pixel and its surrounding pixels. This limitation may lead to insufficient performance of the method in capturing spatial and spectral joint features, causing the characteristic information of the abnormal target to be partially or completely reconstructed into a distribution consistent with the background, thereby causing the abnormal target to be ignored or misjudged during the detection process. This situation is particularly prominent when the spectral characteristics of the abnormal target are close to the background, or the spatial distribution is small and insignificant, which significantly reduces the overall performance and robustness of the detection algorithm.
[0011] Therefore, in order to improve the accuracy and robustness of hyperspectral anomaly detection, it is necessary to develop new technical methods that should not only fully exploit the unique spectral characteristics of the central pixel, but also effectively combine its spectral correlation and spatial correlation with the surrounding pixels, so as to more accurately identify abnormal targets and avoid misjudgment. Summary of the invention
[0012] In order to overcome the shortcomings of the above-mentioned prior art, the purpose of the present invention is to provide a hyperspectral anomaly detection method based on dual spectral affinity constraints, which can not only capture the unique spectral characteristics of each pixel at the same time, but also effectively model the spectral affinity relationship between the pixel and its neighboring pixels. Ultimately, the accuracy and robustness of anomaly detection are effectively improved, especially in scenes with complex backgrounds and weak anomalies, and the abnormal area can be more accurately distinguished from the background area.
[0013] In order to achieve the above object, the technical solution adopted by the present invention is:
[0014] The hyperspectral anomaly detection method based on dual spectral affinity constraints includes the following steps:
[0015] Step 1: Preprocess the input hyperspectral image H;
[0016] Step 2: construct a hyperspectral anomaly detection model Ω based on dual spectral affinity constraints; the hyperspectral anomaly detection model Ω takes the preprocessed hyperspectral image H as input and reconstructs Splice them into a reconstructed image H′, calculate the reconstruction error of the hyperspectral image H and the reconstructed image H′, and calculate the Mahalanobis distance to obtain the anomaly detection result;
[0017] Step 3: Define the mean square error loss function Loss of the hyperspectral anomaly detection model Ω;
[0018] Step 4: Train the hyperspectral anomaly detection model Ω;
[0019] Step 5: Use the trained hyperspectral anomaly detection model Ω to perform anomaly detection reconstruction.
[0020] The step 1 is specifically as follows:
[0021] Hyperspectral images Perform normalization and fix the data range to [-1,1], where H represents the number of row pixels, N represents the number of column pixels, and B represents the number of spectral bands of H. Represents the set of real numbers, from H with h i Extract a 3×3 neighborhood for the center pixel h i The pixel is copied to the center of the same 3×3 size and As inputs of the linear layer respectively;
[0022] For each pixel h after processing i It is converted into a neighborhood according to the following formula and Center
[0023]
[0024] Among them, k represents the size of the patch, φ(h i ,H,k) means that h is selected from H i Extract a patch of size k for the center pixel, and Indicates that h i The pixel is replicated multiple times to form a patch of size k. and They are used as inputs of the two branches respectively, and the sizes are the same, and then the three-dimensional image is transformed into a two-dimensional image. As input to the hyperspectral anomaly detection model Ω.
[0025] The step 2 is specifically as follows:
[0026] 2.1) Construct D-dimensional mapping features in feature space and use the linear layer with feature embedding function to and The channel dimension is linearly transformed from the original dimension B to D, and the neighborhood embedding vector is obtained and the center embedding vector Right now
[0027]
[0028] where f(·) represents the forward propagation of the proposed linear layer, θ semb and θ cemb Represent the weights of the linear layer of the neighborhood patch dimension and the linear layer of the center patch dimension respectively;
[0029] 2.2) Constructing a feature extractor in the dual feature encoder;
[0030] In the step 2.2), the feature extractor is composed of five parts: layer normalization LayerNorm, multi-scale visual attention, Dropout layer, residual connection and feedforward network, and the backbone calculation is performed in sequence, and the fusion of input and output is realized through the residual connection;
[0031] Among them, layer normalization is used for model input, standardizing the distribution of input features, alleviating the problem of gradient disappearance or explosion, and improving training stability and model convergence speed;
[0032] The multi-scale visual attention mechanism is the core part of the feature extractor, which is used to capture global and local features at different scales, thereby enhancing the model's ability to express diverse inputs.
[0033] The Dropout layer randomly discards some neurons to prevent overfitting and improve the generalization ability of the model;
[0034] Residual connections directly skip connections to alleviate the gradient vanishing problem of deep networks and promote lossless transmission of information flow;
[0035] The feedforward network further learns the complex relationship between features through nonlinear transformation, increasing the network's expressive power. The step 2.2) is specifically as follows:
[0036] 2.2a) Neighborhood Embedding Vector and the center embedding vector After layer normalization, it enters the multi-scale visual attention module. To simplify the expression, the input of the multi-scale visual attention is defined as x. According to Q = W q x+x′,K=W k x,V=W v x converts x into query Q, key K, and value respectively, where x′ represents a learnable query embedding with the same shape as x, which better adapts to different HSI data input distributions and improves model generalization ability, W q , W k and W v Represent the weights of the corresponding query, key, and value, respectively.
[0037] 2.2b) The query Q and key K generated by the transformation are subjected to dual-path attention graph calculation, with one group of pixel-level attention and one group of semantic-level attention being performed respectively, where the pixel-level attention graph is represented as attn σ =QK+B σ , the semantic level attention map is defined as attn ρ =QK T +B ρ ,Modifying the traditional QKV attention calculation can effectively avoid calculating the correlation between the central pixel and the surrounding pixels. σ and B ρ represents dynamic position deviation, which is a data-driven method different from static relative position deviation. In the pixel-level path, pixels at the edge of the feature map may produce unrealistic similarity values if the boundary is filled with zeros outside the feature map, so the filling mask is set to -∝;
[0038] 2.2c) Couple the attention of the pixel-level and semantic-level dual paths together and connect attn σ and attn ρAnd perform a softmax operation; this enables the attention between the two paths to be prioritized in a competitive manner.
[0039]
[0040] Then according to attn σ and attn ρ The size of A is split and returned to the original path to obtain the pixel-level and semantic-level attention scores A respectively. σ and A ρ , then the pixel-level feature information x σ and semantic-level feature information x ρ It can be obtained by the following formula
[0041] x σ =A σ V
[0042] x ρ =A ρ V
[0043] Using cross attention to query semantic information with pixel-level information can effectively integrate multi-scale features y as the output of the two branches of the dual feature encoder. For each branch, y corresponds to the neighborhood feature vector and the central eigenvector
[0044]
[0045] 2.3) The dual feature encoder consists of a layer of feature extractors designed in step 2.2) to embed the neighborhood into a vector and the center embedding vector The two inputs of the dual relationship are fed into the weight-shared dual feature encoder to obtain the neighborhood feature vector and the central eigenvector
[0046] 2.4) Neighborhood feature vector and the central eigenvector As the dual input of the dual spectral constraint module, the spectral correlation coefficient between the two is r i The spectral correlation of the dual spectral constraint module can be obtained by calculating the following formula
[0047]
[0048] in, represents matrix multiplication, express The jth pixel of Expressing the same thing, and Respectively and In order to show the similarity change process between them in a visible way, the spectral correlation coefficient r is used. i 2 ∈[0,1], then the spectral affinity matrix can be expressed using the formula scc i =softmax(r i )get;
[0049] 2.5) Obtain the reconstructed image and transform the neighborhood feature vector With spectral affinity matrix SCC i Perform matrix multiplication and then send it to the feature decoder composed of two layers of feature extractors to obtain the reconstructed patchh i ′, and after splicing, the reconstructed image H′ can be obtained.
[0050] The hyperspectral anomaly detection model Ω includes a dual feature encoder, a dual spectral constraint module and a feature decoder;
[0051] The dual feature encoder is composed of a layer of feature extractors, and the feature decoder is composed of two layers of feature extractors;
[0052] The feature extractor achieves effective fusion of input and output through residual connection; residual connection is to add two vectors at each pixel position; the neighborhood after the linear layer is embedded in the vector The output neighborhood feature vector of the feature extractor Similarly, when the center embedding vector As input, the corresponding feature extractor outputs the central feature vector Perform addition;
[0053] The dual feature encoder helps extract key information and internal structure in the data through different dimensional transformations, and mines deeper and more diverse features in the data, thereby improving the model's ability to recognize complex pattern data.
[0054] The feature decoder is responsible for decoding the feature information extracted and compressed through the feature encoding process, restoring it into a useful feature representation, and restoring it to a form closer to the original data; thereby helping the model to complete more accurate detection.
[0055] The dual spectral constraint module is used to calculate the spectral correlation of the input branch, and multiply the similarity with the domain branch as a weight, and then input it into the feature decoder as a constraint to optimize the anomaly detection performance;
[0056] The dual feature encoder is preceded by a linear layer, which is used to implement a feature embedding function;
[0057] The dual encoder network includes a feature extractor, which is used to extract effective spatial and spectral features from the input hyperspectral image data. Through multi-level feature extraction and processing, the spatial-spectral characteristics of the hyperspectral image are fully exploited to provide more accurate and effective feature representation for subsequent anomaly detection tasks.
[0058] The feature extractor has two inputs and they are of the same size, i.e. they have the same height, width and number of channels;
[0059] Spectral Affinity Matrix SCC i is the spectral correlation coefficient r i The normalized matrix has the same size as the neighborhood eigenvector Central eigenvector Same, spectral affinity matrix scc i and the neighborhood feature vector Multiplication refers to the spectral affinity matrix scc i As a weight to the neighborhood feature vector The weighted calculation is then input into the feature decoder.
[0060] The feature decoder uses the multiplication result to obtain the reconstructed patchh i ′, after reconstruction is completed, it needs to be reassembled into the original size to obtain the reconstructed image H′.
[0061] The step 3 is specifically as follows:
[0062] Using the reconstructed patchh i ′ and the original neighborhood The mean square error between is used as the loss function for model training. to optimize the model.
[0063] The step 4 is specifically as follows:
[0064] (4a) Iteratively train the hyperspectral anomaly detection model Ω. The network model Ω adopts an encoder and decoder structure. The learning rate is set to 0.0001, the optimizer uses Adam, the batch training data size is set to 1024, the number of training rounds is set to 100, and the input patch size is 3×3.
[0065] (4b) The original image H is input into the model Ω and the Adam optimizer is used to iteratively optimize the loss function Loss, and the model parameters are continuously updated. After 100 rounds of training, the trained hyperspectral anomaly detection model Ω is obtained.
[0066] The step 7 is specifically as follows:
[0067] The reconstructed patchh i′ can be concatenated to obtain the reconstructed image H′, and then the mean square error is calculated with the original image H to obtain the residual result, and then the residual result is calculated using the Mahalanobis distance in the statistical method to obtain the anomaly detection map.
[0068] Beneficial effects of the present invention:
[0069] (1) The hyperspectral anomaly detection model based on dual spectral affinity constraints constructed by the present invention makes full use of the potential multi-scale spectral relationship between the central pixel and its neighboring pixels. The possible abnormal pixels in the patch are identified based on the unique characteristics of each pixel and its spectral correlation with the surrounding pixels, which effectively improves the accuracy of anomaly detection.
[0070] (2) The present invention introduces spectral correlation coefficient into the detection model, fully exploits the unique spectral features of hyperspectral images, and uses spectral correlation to reduce the impact of anomalies on background reconstruction. The constructed dual monitoring network can extract multi-scale information at the pixel level and semantic level, thereby more effectively capturing abnormal features while suppressing background information. BRIEF DESCRIPTION OF THE DRAWINGS
[0071] Figure 1 It is a schematic diagram of the process of the present invention.
[0072] Figure 2 Schematic diagram of the model framework of the present invention.
[0073] Figure 3 This is a comparison chart of the experimental results of this discovery and existing methods. DETAILED DESCRIPTION
[0074] The present invention will be further described in detail below in conjunction with the accompanying drawings.
[0075] like Figure 1 , Figure 2 As shown, the hyperspectral anomaly detection method based on dual spectral affinity constraints includes the following steps:
[0076] (1) Preprocess the input hyperspectral image to construct the encoder input data:
[0077] Hyperspectral images Normalization is performed to fix the data range to [-1,1]. M, N, and B represent the number of row pixels, column pixels, and spectral bands of H, respectively. Represents the set of real numbers, from H with h i Extract a 3×3 neighborhood for the center pixel h i The pixel is copied to the center of the same 3×3 size Both are used as inputs of the linear layer;
[0078] For each pixel h after processing i It is converted into a neighborhood according to the following formula and Center
[0079]
[0080] Among them, k represents the size of the patch, φ(h i ,H,k) means that h is selected from H i A patch of size k is extracted for the center pixel, and Indicates that h i The pixel is replicated multiple times to form a patch of size k. and They are used as inputs of the two branches respectively, and the sizes are the same, and then the three-dimensional image is transformed into a two-dimensional image. As the input of the hyperspectral anomaly detection model Ω, it can significantly reduce the computational burden and improve the efficiency of model training and reasoning. Step (2), construct a hyperspectral anomaly detection model Ω based on dual spectral affinity constraints:
[0081] 2.1) Construct D-dimensional mapping features in feature space and use the linear layer with feature embedding function to and The channel dimension is linearly transformed from the original dimension B to D, and the neighborhood embedding vector is obtained and the center embedding vector Right now
[0082]
[0083] where f(·) represents the forward propagation of the proposed linear layer, θ semb and θ cemb They represent the weights of the linear layer of the neighborhood patch dimension and the linear layer of the center patch dimension respectively.
[0084] 2.2) Construct a feature extractor in the dual feature encoder. The feature extractor module consists of five parts: layer normalization LayerNorm, multi-scale visual attention, Dropout layer, residual connection and feedforward network. The backbone calculation is performed in sequence, and the input and output are fused through residual connection. Among them, layer normalization is usually applied to model input, which can standardize the distribution of input features, alleviate the problem of gradient disappearance or explosion, and improve training stability and model convergence speed. The multi-scale visual attention mechanism is the core part of the feature extractor, which is used to capture global and local features at different scales, thereby enhancing the model's ability to express diverse inputs. The Dropout layer can randomly discard some neurons to prevent overfitting and improve the generalization ability of the model. The residual connection can directly jump to alleviate the gradient disappearance problem of the deep network and promote the lossless transmission of information flow. The feedforward network further learns the complex relationship between features through nonlinear transformation and increases the network's expression ability.
[0085] 2.2a) Neighborhood Embedding Vector and the center embedding vector After layer normalization, it enters the multi-scale visual attention module. To simplify the expression, the input of the multi-scale visual attention is defined as x. According to Q = W q x+x′,K=W k x,V=W v x converts x into query Q, key K, and value V respectively. Where x′ represents a learnable query embedding with the same shape as x, which better adapts to different HSI data input distributions and improves model generalization ability. q , W k and W v Represents the weights of the corresponding query, key, and value, respectively.
[0086] 2.2b) Perform dual-path attention map calculation, and the pixel-level attention map is represented as attn σ =QK+B σ , the semantic level attention map is defined as attn ρ =QK T +B ρ ,Modifying the traditional QKV attention calculation can effectively avoid calculating the correlation between the central pixel and the surrounding pixels. σ and B ρ Denotes dynamic position deviation, which is a data-driven method different from static relative position deviation. In the pixel-level path, pixels at the edge of the feature map may produce unrealistic similar values if the boundary is filled with zeros outside the feature map, so the padding mask is set to -∝.
[0087] 2.2c) Coupling the attention in the two paths, connecting attn σand attn ρ And perform a softmax operation. This enables the attention between the two paths to be prioritized in a competitive manner.
[0088]
[0089] Then according to attn σ and attn ρ The size of A is split and returned to the original path to obtain the pixel-level and semantic-level attention scores A respectively. σ and A ρ , then the pixel-level feature information x σ and semantic-level feature information x ρ It can be obtained by the following formula
[0090] x σ =A σ V
[0091] x ρ =A ρ V
[0092] Using cross attention to query semantic information with pixel-level information can effectively integrate multi-scale features y as the output of the two branches of the dual feature encoder. For each branch, y corresponds to the neighborhood feature vector and the central eigenvector
[0093]
[0094] (2.3) The dual feature encoder consists of a layer of feature extractors designed in step 2.2) to embed the neighborhood into a vector and the center embedding vector The two inputs of the dual relationship are fed into the weight-shared dual feature encoder to obtain the neighborhood feature vector and the central eigenvector
[0095] (2.4) The neighborhood feature vector and the central eigenvector As the double input of the spectral correlation coefficient, the spectral correlation coefficient r between the two i It can be obtained by calculating the spectral correlation in the dual spectral constraint module
[0096]
[0097] in, represents matrix multiplication, express The jth pixel of Express the same. and Respectively and In order to show the similarity change process between them in a visible way, the spectral correlation coefficient r is used. i 2 ∈[0,1], then the spectral affinity matrix can be expressed using the formula scc i =softmax(r i ) calculated.
[0098] (2.5) Obtain the reconstructed image and transform the neighborhood feature vector With Spectral Affinity Matrix using SCC i Multiply them and then send them to the feature decoder composed of two layers of feature extractors to obtain the reconstructed patchh i ′, and after splicing, the reconstructed image H′ can be obtained.
[0099] (3) Define the mean square error loss function Loss of the hyperspectral anomaly detection model Ω;
[0100] Using the reconstructed patchh i ′ and the original neighborhood The mean square error between is used as the loss function for model training. to optimize the model.
[0101] (4) Training the hyperspectral anomaly detection model Ω:
[0102] (4a) Iterative training of the hyperspectral anomaly detection model Ω, the network model Ω uses a one-layer encoder and a two-layer decoder. The learning rate is set to 0.0001, the optimizer uses Adam, the batch training data size is set to 1024, the number of training rounds is set to 100, and the input patch size is 3×3.
[0103] (4b) The original image H is input into the model Ω and the Adam optimizer is used to iteratively optimize the loss function Loss, and the model parameters are continuously updated. After 100 rounds of training, the trained hyperspectral anomaly detection model Ω is obtained.
[0104] (5) Input the test image into the hyperspectral anomaly detection model Ω based on dual spectral affinity constraints containing optimal weights, and reconstruct Splice into the reconstructed image H′.
[0105] (6) Calculate the reconstruction error of the test image, i.e., the original hyperspectral image H and the reconstructed image H′. The reconstruction error can help identify areas with large errors in the reconstruction process, which usually correspond to abnormal pixels that are not easy to reconstruct.
[0106] (7) The Mahalanobis distance of the reconstruction error is calculated to obtain the anomaly detection result, which can more effectively identify the abnormal area on a global scale and improve the accuracy of abnormal point identification. In order to verify the effectiveness of the present invention, the present invention and the prior art are used to perform hyperspectral anomaly detection and identification, and the respective detection results are compared using evaluation indicators, where AUC represents the detection accuracy, and the closer it is to 1, the better the detection performance. Figure 3 As can be seen from Table 1, the present invention has higher detection accuracy and lower false alarm rate. Because the dual network provides richer and more robust feature representation, it effectively reduces the possibility of confusion between abnormal features and background information, and the enhanced feature representation improves the performance of the model.
[0107] Table 1 Comparison of AUC values between the present invention and the existing method
[0108] RX LRX CRD Auto-AD LREN RGAE DirectNet GT-HAD The present invention Salinas 0.8073 0.9817 0.9173 0.9918 0.8160 0.9388 0.9477 0.9468 0.9931 Hydice 0.9763 0.9955 0.9397 0.9965 0.9019 0.6887 0.6819 0.9138 0.9975 Hyperion 0.9978 0.9975 0.9956 0.9982 0.9141 0.9409 0.9811 0.9872 0.9993 Pavia 0.9538 0.9722 0.9407 0.9729 0.8789 0.9042 0.9052 0.9939 0.9953 average value 0.9542 0.9867 0.9411 0.9899 0.8777 0.8682 0.8789 0.9604 0.9963
Claims
1. A hyperspectral anomaly detection method based on dual spectral affinity constraints, characterized in that: The steps include: Step 1: Preprocess the input hyperspectral image H; Step 2, construct a hyperspectral anomaly detection model Ω based on dual spectral affinity constraints; The hyperspectral anomaly detection model Ω takes the preprocessed hyperspectral image H as input and reconstructs Splice them into a reconstructed image H′, calculate the reconstruction error of the hyperspectral image H and the reconstructed image H′, and calculate the Mahalanobis distance to obtain the anomaly detection result; Step 3: Define the mean square error loss function Loss of the hyperspectral anomaly detection model Ω; Step 4: Train the hyperspectral anomaly detection model Ω; Step 5: Use the trained hyperspectral anomaly detection model Ω to perform anomaly detection reconstruction.
2. The hyperspectral anomaly detection method based on dual spectral affinity constraints according to claim 1 is characterized in that: The step 1 is specifically as follows: Hyperspectral images Perform normalization and fix the data range to [-1,1], where H represents the number of row pixels, N represents the number of column pixels, and B represents the number of spectral bands of H. represents the set of real numbers, from H, h i Extract a 3×3 neighborhood for the center pixel Set pixel h i Duplicate the center to the same size of 3×3 and are respectively used as the input of the linear layer.
3. The hyperspectral anomaly detection method based on dual spectral affinity constraints according to claim 2 is characterized in that: For each pixel h after processing i It is converted into a neighborhood patchh according to the following formula i s and Center Among them, k represents the size of the patch, φ(h i ,H,k) means that h is selected from H i A patch of size k is extracted for the center pixel, and Indicates that h i The pixel is replicated multiple times to form a patch of size k. and They are used as inputs of the two branches respectively, and the sizes are the same, and then the three-dimensional image is transformed into a two-dimensional image. As input to the hyperspectral anomaly detection model Ω.
4. The hyperspectral anomaly detection method based on dual spectral affinity constraints according to claim 3 is characterized in that: The step 2 is specifically as follows: 2.1) Construct the D-dimensional mapping features of the feature space and use the linear layer to transform and The channel dimension is linearly transformed from the original dimension B to D, and the neighborhood embedding vector is obtained and the center embedding vector Right now where f(·) represents the forward propagation of the proposed linear layer, θ semb and θ cemb Represent the weights of the linear layer of the neighborhood patch dimension and the linear layer of the center patch dimension respectively; 2.2) Constructing a feature extractor in the dual feature encoder; 2.3) The dual feature encoder consists of a layer of feature extractors designed in step 2.2) to embed the neighborhood into a vector and the center embedding vector As the input of the dual relationship, it is fed into the weight-shared dual feature encoder to obtain the neighborhood feature vector and the central eigenvector 2.4) Neighborhood feature vector and the central eigenvector As the dual input of the dual spectral constraint module, the spectral correlation coefficient between the two is r i The spectral correlation of the dual spectral constraint module is obtained by calculating the following formula: in, represents matrix multiplication, express The jth pixel of Expressing the same thing, and Respectively and The average value of spectral correlation coefficient Spectral affinity matrix using formula scc i =softmax(r i )get; 2.5) Obtain the reconstructed image and transform the neighborhood feature vector Spectral Affinity Matrix with SCC i Multiply them and then send them to the feature decoder composed of two layers of feature extractors to obtain the reconstructed patchh i ′, and after splicing, the reconstructed image H′ can be obtained.
5. The hyperspectral anomaly detection method based on dual spectral affinity constraints according to claim 4 is characterized in that: In the step 2.2), the feature extractor consists of five parts: layer normalization LayerNorm, multi-scale visual attention, Dropout layer, residual connection and feedforward network, and performs backbone calculation in sequence, while realizing the fusion of input and output through residual connection.
6. The hyperspectral anomaly detection method based on dual spectral affinity constraints according to claim 4 is characterized in that: The step 2.2) is specifically as follows: 2.2a) Neighborhood Embedding Vector and the center embedding vector After layer normalization, it enters the multi-scale visual attention module and defines the input of the multi-scale visual attention as x. According to Q = W q x+x′,K=W k x,V=W v x converts x into a query Q, a key K, and a value, respectively, where x′ represents a learnable query embedding of the same shape as x, W q , W k and W v Represent the weights of the corresponding query, key, and value respectively; 2.2b) The query Q and key K generated by the transformation are subjected to dual-path attention graph calculation, with one group of pixel-level attention and one group of semantic-level attention being performed respectively, where the pixel-level attention graph is represented as attn σ =QK+B σ , the semantic level attention map is defined as attn ρ =QK T +B ρ , where B σ and B ρ To represent dynamic position deviation, set the fill mask to -∝; 2.2c) Couple the attention of the pixel-level and semantic-level dual paths together and connect attn σ and attn ρ And perform softmax operation; Then according to attn σ and attn ρ The size of A is split and returned to the original path to obtain the pixel-level and semantic-level attention scores A respectively. σ and A ρ , then the pixel-level feature information x σ and semantic-level feature information x ρ It can be obtained by the following formula x σ =A σ V x ρ =A ρ V Using cross attention to query semantic information with pixel-level information can effectively integrate multi-scale features y as the output of the two branches of the dual feature encoder. For each branch, y corresponds to the neighborhood feature vector and the central eigenvector 7. The hyperspectral anomaly detection method based on dual spectral affinity constraints according to claim 4 is characterized in that: The hyperspectral anomaly detection model Ω includes a dual feature encoder, a dual spectral constraint module and a feature decoder; The dual feature encoder is composed of a layer of feature extractors, and the feature decoder is composed of two layers of feature extractors; The feature extractor achieves effective fusion of input and output through residual connection, which adds two vectors at each pixel position; embedding the neighborhood after the linear layer into the vector The output neighborhood feature vector of the feature extractor Add up; Similarly, when the center embedding vector As input, the corresponding feature extractor outputs the central feature vector Perform addition; The dual feature encoder helps extract key information and internal structure in the data through different dimensional transformations, and mines deeper and more diverse features in the data; The feature decoder is responsible for decoding the feature information extracted and compressed by the feature encoding process, restoring it into a useful feature representation, and restoring it to a form closer to the original data; The dual spectral constraint module is used to calculate the spectral correlation of the input branches, and multiply the similarity with the domain branch as a weight, and then input it into the feature decoder as a constraint to optimize the anomaly detection performance.
8. The hyperspectral anomaly detection method based on dual spectral affinity constraints according to claim 7 is characterized in that: The dual feature encoder is preceded by a linear layer, which is used to implement a feature embedding function; The feature extractor is used to extract effective spatial and spectral features from the input hyperspectral image data. Through multi-level feature extraction and processing, the spatial-spectral characteristics of the hyperspectral image are fully exploited to provide more accurate and effective feature representation for subsequent anomaly detection tasks. The feature extractor has two inputs and are of the same size, that is, they have the same height, width and number of channels.
9. The hyperspectral anomaly detection method based on dual spectral affinity constraints according to claim 4 is characterized in that: The step 3 is specifically as follows: Using the reconstructed patchh i ′ and the original neighborhood The mean square error between is used as the loss function for model training. to optimize the model.
10. The hyperspectral anomaly detection method based on dual spectral affinity constraints according to claim 9 is characterized in that: The step 4 is specifically as follows: (4a) Iteratively train the hyperspectral anomaly detection model Ω. The network model Ω uses one encoder and two decoders. The optimizer uses Adam. (4b) The original image H is input into the model Ω and the Adam optimizer is used to iteratively optimize the loss function Loss, and the model parameters are continuously updated. After 100 rounds of training, the trained hyperspectral anomaly detection model Ω is obtained.