Anomaly detection method based on structural inconsistency and guided augmentation
By combining hyperspectral image patching and multi-scale convolutional networks with a symmetric loss function, an anomaly detection model is constructed, which solves the problem of efficient anomaly detection in hyperspectral images without prior knowledge, and achieves sensitive detection and high-precision recognition of anomalies.
Patent Information
- Application Number
- CN202610089396.4
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2026-01-22
- Publication Date
- 2026-06-09
AI Technical Summary
In hyperspectral image analysis, existing technologies struggle to efficiently detect pixels or regions that are significantly different from their surroundings without prior knowledge, especially in zero-sample detection where high detection accuracy is difficult to achieve.
An anomaly detection model is constructed by combining hyperspectral image patching, structural inconsistency modeling, multi-scale convolutional networks, symmetric loss functions, and anomaly enhancement guided filters. This expands the number of training samples, extracts features, reduces noise, and enhances detection capabilities.
It achieves sensitive detection of weak, small-sized or background-similar anomalies, possesses zero-shot self-supervised learning characteristics, is highly adaptable, and reduces reliance on labeled data and prior knowledge.
Smart Images
Figure CN122175861A_ABST
Abstract
Description
Technical Field
[0001] This invention belongs to the field of hyperspectral image anomaly detection technology, and specifically designs an anomaly detection method based on structural inconsistency and guided enhancement. Background Technology
[0002] Hyperspectral imaging technology acquires spectral and spatial information of ground features simultaneously across hundreds of consecutive narrow bands, forming a three-dimensional data cube containing two-dimensional space and one-dimensional spectrum. Due to the differences in spectral dimensions among different materials, hyperspectral images (HSI) can provide richer ground feature information than RGB images, demonstrating broad application potential in fields such as resource exploration, agricultural monitoring, urban planning, and military reconnaissance. Among numerous hyperspectral image analysis tasks, hyperspectral anomaly detection (HAD) has received continuous attention. Its goal is to automatically identify pixels or regions significantly different from their surroundings from the background without explicit prior class information. Typical anomalous targets include vegetation eroded by pests and diseases, sparsely distributed minerals, and camouflage or man-made structures in battlefield scenarios. However, insufficient prior knowledge of the target and achieving high detection accuracy while using zero-shot detection methods remain core challenges in the field of HAD. Summary of the Invention
[0003] S1: Hyperspectral image patch partitioning: By partitioning the hyperspectral image into patches, multiple hyperspectral image patch sub-images are obtained, thereby increasing the number of training samples; S2: Modeling structural inconsistencies in patch subimages: Each patch subimage is input into a hyperspectral image pair downsampler, and one patch subimage can produce two different downsampled images; S3: Construct a multi-scale convolutional network: By constructing this network, we can adapt the enhanced image obtained after modeling the inconsistent structure, extract the image features, and reduce noise; S4: Construct a symmetric loss function: By constructing this loss function, the model can pay attention to contextual inconsistencies in the image during training, making it easier to detect abnormal targets in the image; S5: Establish a dual-branch training channel, train the multi-scale convolutional network based on the symmetric loss function, and achieve the training objective of the multi-scale convolutional network through the symmetric loss function. S6: Input the image to be detected into the trained multi-scale convolutional network to obtain the detection result, construct an anomaly enhancement guided filter, input the detection structure into the anomaly enhancement guided filter to perform anomaly detection and obtain enhanced detection result; S7: Detection of hyperspectral images: Obtain anomaly detection results by using the trained model and anomaly enhancement guided filtering, and calculate AUC accuracy and anomaly background separability.
[0004] S1 specifically adopts the following method: The entire hyperspectral image is divided into multiple small patch sub-images in a pixel-by-pixel manner with a step size of 1, thereby increasing the number of training samples and ensuring that the training data comes from only one hyperspectral image.
[0005] The specific method used in S2 is as follows: The resulting patch sub-image is input into a hyperspectral image pair downsampler to obtain two different downsampled images. Specifically, this is achieved by applying a 2-dimensional convolution with a stride of 2 and a fixed kernel function to the input image. and What was obtained.
[0006] The specific method used in S3 is as follows: S31: The original multi-channel hyperspectral image is mapped to an intermediate feature space by feature encoding of the input patch through a shared 3×3 initial convolutional layer and ReLU activation function; S32: The encoded features are simultaneously fed into three parallel convolutional branches, corresponding to the original scale, downsampling scale once, and downsampling scale twice, respectively, so that the network can model the structural differences between anomalies and background in images under different receptive fields. S33: Each scale layer contains two simple convolutional modules and one channel attention module. The simple convolutional modules use two standard 3×3 convolutional kernels and the ReLU activation function to extract local feature information layer by layer; the channel attention module assumes that given features... First, calculate the mean pooling vector and the standard pooling vector:
[0007]
[0008] The channel description vector z is obtained by concatenation, and the channel attention w is learned through two fully connected layers:
[0009] in, For ReLU, It is Sigmoid. , , and These represent the weight matrices of the fully connected layer. The scaling factor is used. Finally, the weighted features are obtained:
[0010] in This indicates channel-wise multiplication, and this weighted feature guides the network to pay more attention to feature responses that show obvious anomalies in statistical attributes; S34: Finally, all features are extracted by fusing the features extracted from each scale layer and passing them through a 1×1 convolutional layer.
[0011] The specific method used in S4 is as follows: S41: Construct a symmetric residual loss function: This loss is used to constrain the network to still generate consistent responses to outlier regions when inputting different pairs of downsampled images.
[0012] in and This represents the different downsampled images obtained in S2. This represents the multi-scale convolutional network constructed by S3. Represents the training parameters of the network; S42: Constructing a symmetric consistency loss function: This loss function ensures that the network's response to the same anomalous region remains consistent across images of different resolutions. That is, the result obtained by downsampling before inputting the image into the network is consistent with the result obtained by inputting the image into the network before downsampling. Therefore, this loss function is defined as:
[0013] S43: Construct the total loss function:
[0014] The specific approach used in S5 is as follows: S51: Construct a dual-branch channel training model; S52: The upper branch first models the structural inconsistency of the input patch subgraph (S2), and then extracts features from the input multi-scale convolutional network (S3). S53: The lower branch directly inputs the patch subgraph into the multi-scale convolutional network to extract features (S3), and then performs structural inconsistency modeling on the residuals of the original patch subgraph and the features output by the multi-scale convolutional network (S2). S54: Constrain the obtained features using the symmetric loss function (S4) and fuse the features to obtain the final training model.
[0015] The specific method used in S6 is as follows: S61: During the anomaly detection phase, hyperspectral images Directly input into the trained In this process, the matrix corresponding to outlier scores is obtained directly through the mean operation. :
[0016] in This indicates that the average value is taken based on the channels. S62: Defines the kernel size as (3×3) sliding window Calculate the local mean and local variance :
[0017]
[0018] in Indicates the current center pixel position. Display window pixels within; S63: Based on local variance Construct a nonlinear enhancement factor :
[0019] in It represents the maximum variance of the entire score plot, used to normalize local variance. Will Limited to Within the range, ensure the enhancement factor It won't be too big. It is a small positive number; S64: Apply the enhancement factor to the score plot to obtain the enhancement result:
[0020] S65: Use guided filtering to further detect anomalies and preserve edge information of the image, assuming the output image... With guide map in local window within satisfy:
[0021] in and These are local regression coefficients, and their estimation process involves minimizing the reconstruction error:
[0022] This optimization problem has a closed-form solution, which is:
[0023] in yes and In the window Covariance within, and These are guide maps In the window Mean and variance within, for In the window The mean value within the range. The final calculated abnormal response matrix is obtained. .
[0024] The specific method used in S7 is as follows: By inputting the hyperspectral image to be detected into the trained model, and using anomaly enhancement guided filtering, the anomaly detection results are obtained, and the AUC accuracy and anomaly background separability are calculated.
[0025] Based on the above technical solution, this invention can be widely applied in fields such as agricultural detection and geological exploration. The core idea of structural inconsistency modeling proposed in this invention can keenly capture subtle differences in spectral and spatial structure between the target and the background. This makes the method highly capable of detecting weak, small-sized anomalies or those partially similar to the background. Simultaneously, its zero-shot self-supervised learning characteristic frees it from dependence on large amounts of labeled data or prior knowledge, demonstrating strong adaptability and ease of deployment when facing new, data-scarce detection scenarios. Attached Figure Description
[0026] To more clearly illustrate the technical solutions in the embodiments of the present invention or the prior art, the drawings used in the description of the embodiments or the prior art will be briefly introduced below. Obviously, the drawings described below are some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.
[0027] Figure 1 A flowchart for modeling and guiding the enhancement of structural inconsistencies in this invention.
[0028] Figure 2 This is a schematic diagram of patch segmentation according to the present invention.
[0029] Figure 3 This is a diagram of the multi-scale convolutional network structure of the present invention.
[0030] Figure 4 The schematic diagram illustrates the structural inconsistency modeling and loss function principle of this invention.
[0031] Figure 5 This is a flowchart of the training process of the present invention.
[0032] Figure 6 This is a pseudo-color image of the test dataset for this invention.
[0033] Figure 7 This is a diagram showing the detection results of the present invention.
[0034] Figure 8 The background anomaly can be separated into box plots for the present invention. Detailed Implementation
[0035] To enable those skilled in the art to better understand the present invention, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention. Obviously, the described embodiments are only some embodiments of the present invention, and not all embodiments. Based on the embodiments of the present invention, all other embodiments obtained by those skilled in the art without creative effort should fall within the scope of protection of the present invention.
[0036] To make the technical solutions and advantages of the present invention clearer, the technical solutions of the present invention will be clearly and completely described below with reference to the accompanying drawings of the embodiments of the present invention, as shown in the attached drawings. Figure 1 As shown: S1: The entire hyperspectral image is divided into multiple small patch sub-images in a pixel-by-pixel manner with a step size of 1, thereby increasing the number of training samples and ensuring that the training data comes from only one hyperspectral image, such as... Figure 2 As shown, each image is divided into multiple small patch sub-images according to pixels.
[0037] S2: Input the obtained patch sub-image into the hyperspectral image downsampler to obtain two different downsampled images. That is, by applying a 2D convolution with a stride of 2 and a fixed kernel function to the input image. and Specifically, the resulting image pairs are generated by taking the average of the diagonal pixels of each block in the same band and assigning it to the first low-resolution image, and then taking the average of the anti-diagonal pixels in the same band and assigning it to the second low-resolution image. Figure 4 As shown in the orange box, these are the two downsampled images obtained.
[0038] S3: Constructing a multi-scale convolutional network: This network adapts to the enhanced image obtained after modeling structural inconsistencies, extracts image features, and reduces noise. See [link to network structure] for details. Figure 3 ; S31: The original multi-channel hyperspectral image is mapped to an intermediate feature space by feature encoding of the input patch through a shared 3×3 initial convolutional layer and ReLU activation function; S32: The encoded features are simultaneously fed into three parallel convolutional branches, corresponding to the original scale, downsampling scale once, and downsampling scale twice, respectively, so that the network can model the structural differences between anomalies and background in images under different receptive fields. S33: Each scale layer contains two simple convolutional modules and one channel attention module. The simple convolutional modules use two standard 3×3 convolutional kernels and the ReLU activation function to extract local feature information layer by layer; the channel attention module assumes that given features... First, calculate the mean pooling vector and the standard pooling vector:
[0039]
[0040] The channel description vector z is obtained by concatenation, and the channel attention w is learned through two fully connected layers:
[0041] in, For ReLU, It is Sigmoid. , , and These represent the weight matrices of the fully connected layer. The scaling factor is used. Finally, the weighted features are obtained:
[0042] in This indicates channel-wise multiplication, and this weighted feature guides the network to pay more attention to feature responses that show obvious anomalies in statistical attributes; S34: In the downsampling path, spatial downsampling is performed using a convolution operation with a stride of 2 to expand the network's receptive field and capture contextual information over a wider range. Features extracted from each path have different spatial and semantic representation capabilities at their respective scales, effectively supplementing each other's information blind spots. After completing the three-scale feature extraction, the output of the downsampling path is upsampled using bilinear interpolation to restore it to the original scale, and then fused with the output of the original scale path. The fusion operation employs a simple element-wise averaging strategy to ensure information balance across scales. Finally, the fused features are output after passing through a 1×1 convolutional layer.
[0043] S4: Constructing a Symmetric Loss Function: By constructing this loss function, the model pays attention to contextual inconsistencies in the image during training, making it easier to detect anomalous targets. Based on the characteristics of background and anomalous regions, the two downsampled images are typically consistent in the background region due to smooth spectral changes, while the anomalous region, due to abrupt changes in spatial-spectral structure, exhibits relatively large and unpredictable variations in its values and shapes across patches. Therefore, the network can learn to predict content from the original image that cannot be explained by context, which perfectly matches the characteristics of anomalous regions. Two loss functions are used for this purpose, one for anomalous region extraction and the other for image structure consistency. The principle is as follows: Figure 4 As shown, the input process for generating the image is the same as the calculation process for the loss function; S41: Construct a symmetric residual loss function: This loss is used to constrain the network to still generate consistent responses to outlier regions when inputting different pairs of downsampled images.
[0044] in and This represents the different downsampled images obtained in S2. This represents the multi-scale convolutional network constructed by S3. Represents the training parameters of the network; S42: Constructing a symmetric consistency loss function: This loss function ensures that the network's response to the same anomalous region remains consistent across images of different resolutions. That is, the result obtained by downsampling before inputting the image into the network is consistent with the result obtained by inputting the image into the network before downsampling. Therefore, this loss function is defined as:
[0045] S43: Construct the total loss function:
[0046] S5: Training the Anomaly Detection Model: The model is trained by establishing a dual-branch training channel, combining structural inconsistency modeling and multi-scale convolutional networks in different sequences. A symmetric loss function is used to achieve the model's training objective. See below for the detailed process. Figure 5 ; S51: Construct a dual-branch channel training model; S52: The upper branch first models the structural inconsistency of the input patch subgraph (S2), and then extracts features from the input multi-scale convolutional network (S3). S53: The lower branch directly inputs the patch subgraph into the multi-scale convolutional network to extract features (S3), and then performs structural inconsistency modeling on the residuals of the original patch subgraph and the features output by the multi-scale convolutional network (S2). S54: Constrain the obtained features using the symmetric loss function (S4) and fuse the features to obtain the final training model.
[0047] S6: Constructing anomaly enhancement guided filters: This method enhances the accuracy of detection results after model detection; S61: During the anomaly detection phase, hyperspectral images Directly input into the trained In this process, the matrix corresponding to outlier scores is obtained directly through the mean operation. :
[0048] in This indicates that the average value is taken based on the channels. S62: Defines the kernel size as (3×3) sliding window Calculate the local mean and local variance :
[0049]
[0050] in Indicates the current center pixel position. Display window pixels within; S63: Based on local variance Construct a nonlinear enhancement factor :
[0051] in It represents the maximum variance of the entire score plot, used to normalize local variance. Will Limited to Within the range, ensure the enhancement factor It won't be too big. It is a small positive number; S64: Apply the enhancement factor to the score plot to obtain the enhancement result:
[0052] S65: Use guided filtering to further detect anomalies and preserve edge information of the image, assuming the output image... With guide map in local window within satisfy:
[0053] in and These are local regression coefficients, and their estimation process involves minimizing the reconstruction error:
[0054] This optimization problem has a closed-form solution, which is:
[0055] in yes and In the window Covariance within, and These are guide maps In the window Mean and variance within, for In the window The mean value within the range. The final calculated abnormal response matrix is obtained. .
[0056] S7: Detection of hyperspectral images: Obtain anomaly detection results by using the trained model and anomaly enhancement guided filtering, and calculate AUC accuracy and anomaly background separability.
[0057] Three hyperspectral datasets were selected for testing, such as... Figure 6 As shown, (a) is the Salinas dataset, (b) is the Bay Champagne dataset, and (c) is the Pavia dataset. Detailed information about each dataset is shown in Table 1.
[0058] Table 1. Detailed information on different data.
[0059] The detection results obtained using this method are shown below. Figure 7 The detection effect can be clearly observed in the figure. The bright spots in the figure are abnormal targets. The AUC accuracy of the data is shown in Table 2. The closer the AUC value is to 1, the better the detection effect.
[0060] Table 2 AUC accuracy of test data
[0061] In addition, background anomaly separability visualization was performed on the three datasets, see [link / reference]. Figure 8 As shown in the figure, it can be observed that the values of the background and the anomaly do not overlap, which means that the separability of the background and the anomaly is excellent and the background value is almost 0.
[0062] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some or all of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the scope of the technical solutions of the embodiments of the present invention.
Claims
1. An anomaly detection method based on structural inconsistency and guidance enhancement, characterized in that, Includes the following steps: S1: The hyperspectral image is divided into patches to obtain multiple hyperspectral image patch sub-images; S2: Perform structural inconsistency modeling on the hyperspectral image patch sub-images, and input each hyperspectral image patch sub-image into the hyperspectral image pair downsampler, so that one patch sub-image yields two different downsampled images; S3: Construct multi-scale convolutional networks; S4: Construct a symmetric loss function to enable the multi-scale convolutional network to capture inconsistent contextual information in the image during training; S5: Establish a dual-branch training channel and train the multi-scale convolutional network based on the symmetric loss function; S6: Input the image to be detected into the trained multi-scale convolutional network to obtain the detection result, construct an anomaly enhancement guided filter, input the detection structure into the anomaly enhancement guided filter to perform anomaly detection and obtain enhanced detection result; S7: Analyze the enhanced detection results, calculate the AUC accuracy and separability of the abnormal background, and analyze the advantages of this anomaly detection method.
2. The anomaly detection method based on structural inconsistency and guidance enhancement according to claim 1, characterized in that, The entire hyperspectral image is divided into multiple patch sub-images in a pixel-by-pixel manner with a certain step size.
3. The anomaly detection method based on structural inconsistency and guidance enhancement according to claim 1, characterized in that, The resulting patch sub-image is input into the hyperspectral image pair downsampler to obtain two different downsampled images.
4. The anomaly detection method based on structural inconsistency and guidance enhancement according to claim 1, characterized in that, When constructing multi-scale convolutional networks: S31: The original multi-channel hyperspectral image is mapped to an intermediate feature space by feature encoding of the input patch through a shared 3×3 initial convolutional layer and ReLU activation function; S32: The encoded features are simultaneously input into three parallel convolutional branches, corresponding to the original scale, the downsampling scale once, and the downsampling scale twice, respectively; S33: Each scale layer contains two simple convolutional modules and one channel attention module. The simple convolutional modules use two standard 3×3 convolutional kernels and the ReLU activation function to extract local feature information layer by layer. The channel attention module assumes that given features... Calculate the mean pooling vector and the standard pooling vector: The channel description vector z is obtained by concatenation, and the channel attention w is learned through two fully connected layers: in, For ReLU, It is Sigmoid. , , and These represent the weight matrices of the fully connected layer. Using scaling factors, the final weighted features are obtained: in This indicates channel-wise multiplication, and this weighted feature guides multi-scale convolutional networks to focus on feature responses that exhibit anomalous statistical properties. S34: All features are extracted by fusing the features extracted from each scale layer and then passing them through a 1×1 convolutional layer.
5. The zero-sample direct anomaly detection method based on structural inconsistency modeling and guided enhancement according to claim 1, characterized in that, When constructing a symmetric loss function: S41: Constructing a symmetric residual loss function: This symmetric residual loss function is used to constrain the network to generate consistent responses to outlier regions when inputting different pairs of downsampled images. in and Representing different downsampled images, This represents the multi-scale convolutional network constructed by S3. Represents the training parameters of the network; S42: Construct a symmetric consistency loss function to ensure that the multi-scale convolutional network maintains a consistent response to the same anomalous region in images with different resolutions. Therefore, this loss function is defined as: S43: Construct the total loss function as a symmetric loss function: 。 6. The anomaly detection method based on structural inconsistency and guidance enhancement according to claim 1, characterized in that, The multi-scale convolutional network is trained using a symmetric loss function, resulting in the trained multi-scale convolutional network model: S51: Construct a dual-branch training channel; S52: The upper branch first models the structural inconsistency of the input patch subgraph and extracts features from the input multi-scale convolutional network. S53: The lower branch directly inputs the patch subgraph into the multi-scale convolutional network to extract features, and models the structural inconsistency between the residuals of the original patch subgraph and the features output by the multi-scale convolutional network. S54: Constrain the obtained features using a symmetric loss function and perform feature fusion to obtain the final multi-scale convolutional network.
7. The anomaly detection method based on structural inconsistency and guidance enhancement according to claim 1, characterized in that, The anomaly enhancement guided filter performs anomaly detection in the following manner: S61: During the anomaly detection phase, hyperspectral images Directly input into the trained multi-scale convolutional network In this process, the matrix corresponding to outlier scores is obtained directly through the mean operation. : in This indicates that the average value is taken from each channel. S62: Defines the kernel size as (3×3) sliding window Calculate the local mean and local variance : in Indicates the current center pixel position. Display window pixels within; S63: Based on local variance Construct a nonlinear enhancement factor : in It represents the maximum variance of the entire score plot, used to normalize local variance. Will Limited to Within the range, ensure the enhancement factor It won't be too big. It is a small positive number; S64: Apply the enhancement factor to the score plot to obtain the enhancement result: S65: Use guided filtering to further detect anomalies and preserve edge information of the image, assuming the output image... With guide map in local window within satisfy: in and These are local regression coefficients, and their estimation process involves minimizing the reconstruction error: This optimization problem has a closed-form solution, which is: in yes and In the window Covariance within, and These are guide maps In the window Mean and variance within, for In the window The mean value within the range is used to calculate the processed abnormal response matrix. .