Hyperspectral anomaly detection method based on multi-scale memory network
By constructing a multi-scale memory network MSMNet model, using pseudo-label generation and feature separation technology, the problem of poor background and abnormal separation in hyperspectral anomaly detection is solved, and a more efficient abnormal detection effect is achieved.
Patent Information
- Application Number
- CN202510683976.1
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-05-26
- Publication Date
- 2025-08-19
AI Technical Summary
In the existing hyperspectral anomaly detection methods, the separation effect of background and abnormalities is poor, resulting in limited model detection performance and lack of effective labels, resulting in the model reconstructing abnormal samples, affecting detection accuracy.
A multi-scale memory network MSMNet model was constructed, and a pseudo-label generation module CPLG, a multi-scale memory module MMST and a consistent discriminant feature learning module CDFL were introduced. Through pseudo-label guidance training and feature separation, the background reconstruction ability was enhanced, the exception reconstruction was weakened, and the degree of separation between the exception and the background was improved.
It significantly improves the separation effect of hyperspectral anomaly detection, enhances the model's ability to reconstruct the background, improves the degree of separation between the abnormality and the background, and improves the detection performance.
Smart Images
Figure CN120510447A_ABST
Abstract
Description
Technical Field
[0001] The present invention belongs to the field of computer vision and machine learning technology, and further relates to hyperspectral anomaly detection. Specifically, it is a hyperspectral anomaly detection method based on a multi-scale memory network, which can be used in environmental monitoring, military exploration, geological exploration and other fields. Background Art
[0002] Hyperspectral anomaly detection is a hyperspectral image processing task. It is defined as capturing pixels in a hyperspectral image that exhibit significant spectral differences from surrounding pixels without prior knowledge. This characteristic is widely used in military, agricultural, geological, and marine applications. In recent years, deep learning models have attracted the attention of hyperspectral researchers, who have applied them to hyperspectral anomaly detection, achieving impressive accuracy.
[0003] In the existing technical literature, MSNet [Liu H, Su X, Shen X, et al. MSNet: Self-Supervised Multi-Scale Network with Enhanced Separation Training for Hyperspectral Anomaly Detection [J]. IEEE Transactions on Geoscience and Remote Sensing, 2024, 62: 1-13.], a multi-scale separation training network is designed, using a designed loss function to constrain the separation training of anomalies and background. However, the constraint capability of the loss function is limited, which restricts the effective separation of anomalies and background. In the existing technical literature MSBRNet [Cao W, Zhang H, He W, et al. MSBRNet: Multi-scale background reconstruction network with low-rank embedding for anomaly detection in hyperspectral images [C] / / IGARSS 2022-2022 IEEE International Geoscience and Remote Sensing Symposium. IEEE, 2022: 3720-3723.], this method uses a neural network to extract low-rank background features, fully utilizing the low-rank nature of the background. These features are then input into the multi-scale background reconstruction module to obtain a reconstructed background, thereby separating anomalies from the background. However, in complex scenarios, simple low-rank constraints may not only extract background features but also anomaly features, limiting the separation of anomalies from background.
[0004] Currently, reconstruction-based hyperspectral anomaly detection methods have the following shortcomings: 1) Due to the powerful feature extraction capabilities of deep learning models, some abnormal samples can also be well reconstructed, resulting in missed anomalies; 2) Due to the lack of available labels, most reconstruction-based hyperspectral anomaly detection methods only focus on the quality of sample reconstruction, without considering whether the sample is an anomaly or background. This is not conducive to separating background and anomalies, thus limiting the improvement of model performance. Summary of the Invention
[0005] The purpose of the present invention is to address the deficiencies of the above-mentioned prior art and propose a hyperspectral anomaly detection method based on a multi-scale memory network to solve the problem in the prior art that the background and anomaly separation effect is poor, thereby limiting the model detection performance. In the present invention, a new multi-scale memory network MSMNet model is constructed, which introduces an innovatively designed coarse-to-fine pseudo-label generation CPLG module, a multi-scale memory MMST module based on separation training, and a consistency and discriminative feature learning CDFL module; pseudo-labels are generated by the CPLG module as a guide for the separation training of the MMST module and used as pseudo-supervisory labels for training the CDFL module; the MMST module includes a multi-scale memory module that can weaken the model's ability to reconstruct anomaly samples and improve the degree of separation between anomalies and background; the CDFL module is used to improve the feature consistency of background samples and the feature discriminability of abnormal samples, thereby further enhancing the separation of anomalies and background. The present invention can effectively enhance the model's ability to reconstruct the background, weaken the model's reconstruction of anomalies, and significantly improve the degree of separation between anomalies and background.
[0006] In order to achieve the above object, the technical solution of the present invention includes the following:
[0007] (1) Obtaining the original hyperspectral image Obtaining dimensionality-reduced hyperspectral images through a band selection algorithm based on orthogonal projection Where H and W represent the height and width of the hyperspectral image, and B and C represent the number of bands of the hyperspectral image;
[0008] (2) A hyperspectral anomaly detection model is constructed based on a multi-scale memory network. The model includes an encoder E, an intermediate block, a first decoder D, an upsampling block, a coarse-to-fine pseudo-label generation CPLG module, a multi-scale memory MMST module based on separation training, a consistency and discriminative feature learning CDFL module, and a second decoder M. The implementation steps are as follows:
[0009] (2.1) The encoder E is constructed by cascading two sub-encoders E1 and E2 with the same structure. Both E1 and E2 consist of two continuously stacked 3×3 convolution blocks, which are used to encode X to obtain shallow coding features. and deep encoding features
[0010] (2.2) The intermediate block is constructed by two consecutively stacked 3×3 convolution blocks to encode deep features. Processing to generate transition features
[0011] (2.3) The first decoder D is constructed by cascading two sub-decoders D1 and D2 with the same structure. Both D1 and D2 are composed of stacked deconvolution blocks and convolution blocks. The transition feature F I Input decoder D to generate the first decoding feature and the second decoding feature
[0012] (2.4) The upsampling layer and two consecutively stacked 3×3 convolution blocks form a sub-upsampling block, and the upsampling block is constructed using three sub-upsampling blocks; I 、F D1 and F D2 As the input of the upsampling block, each of them goes through a sub-upsampling block to restore the original resolution, that is, to obtain the upsampling features of the k-th scale branch k=1,2,3;
[0013] (2.5) Build the CDFL module and convert F E1 、F E2 、F I and F D1 As its input, the cosine distance map is obtained by calculating the cosine similarity of the encoding feature, the transition feature and the first decoding feature;
[0014] (2.6) Construct the MMST module and convert Z k As its input, the background samples and abnormal samples are separated and trained to obtain the memory features of the retrieval
[0015] (2.7) Construct decoder M, and use decoder M to remember features Decode to get the reconstructed image The decoder M consists of decoder M1, decoder M2 and decoder M3, and each decoder consists of a 1×1 convolution block and a 1×1 convolution layer;
[0016] (2.8) Construct the CPLG module by calculating the hyperspectral image X and the reconstructed image The reconstruction error between the two generates a pseudo label M f , and use this label to guide the separation training of the MMST module, and at the same time serve as a pseudo-supervisory label for training the CDFL module to supervise the optimization of the cosine distance map;
[0017] (3) Using L2 norm and designed CDFL loss function L CDFL As the joint loss function L, it guides the model training to convergence and obtains the trained final detection model;
[0018] (4) Input the original hyperspectral image X into the final detection model to obtain the reconstructed hyperspectral image according to Calculate with X to obtain the anomaly detection result.
[0019] Compared with the prior art, the present invention has the following advantages:
[0020] First, the present invention designs a multi-scale memory network. Based on the combined error between the hyperspectral image reconstructed from multi-scale features and the original hyperspectral image, a CPLG module is designed to generate pseudo-labels for the hyperspectral image. Furthermore, this module can be extended to a single scale and inserted into other hyperspectral detection methods as a plug-and-play module to generate pseudo-labels and guide model training.
[0021] Second, the present invention designs a MMST module to weaken the model's ability to reconstruct abnormal samples and improve the separation level of anomalies and background.
[0022] Third, the present invention designs a CDFL module to improve the feature consistency of background samples and the feature discriminability of abnormal samples, thereby further enhancing the separation of abnormalities and background. BRIEF DESCRIPTION OF THE DRAWINGS
[0023] Figure 1 Figure 1 is a diagram of the overall implementation architecture of the method of the present invention; (a) is a schematic diagram of the structure of the multi-scale memory network model constructed in the present invention; (b) is a schematic diagram of the structure of the coarse-to-fine pseudo-label generation module in the model; (c) is a schematic diagram of the structure of the multi-scale memory module based on separation training in the model; (d) is a schematic diagram of the structure of the consistency and discriminative feature learning module in the model.
[0024] Figure 2 2 is a comparison chart of the detection effects of the present invention and the existing method. DETAILED DESCRIPTION
[0025] In order to make the purpose and advantages of the present invention more clear, the technical content of the present invention is described in detail below with reference to the accompanying drawings.
[0026] Example 1: Refer to the attached Figure 1 The present invention proposes a hyperspectral anomaly detection method based on a multi-scale memory network, which specifically includes the following steps:
[0027] Step 1. Obtain the original hyperspectral image Obtaining dimensionality-reduced hyperspectral images through a band selection algorithm based on orthogonal projection Where H and W represent the height and width of the hyperspectral image, and B and C represent the number of bands of the hyperspectral image;
[0028] Step 2. Construct a hyperspectral anomaly detection model based on a multi-scale memory network. The model consists of an encoder E, an intermediate block, a first decoder D, an upsampling block, a coarse-to-fine pseudo-label generation CPLG module, a multi-scale memory MMST module based on separation training, a consistency and discriminative feature learning CDFL module, and a second decoder M. The implementation steps are as follows:
[0029] (2.1) The encoder E is constructed by cascading two sub-encoders E1 and E2 with the same structure. Both E1 and E2 consist of two continuously stacked 3×3 convolution blocks, which are used to encode X to obtain shallow coding features. and deep encoding features
[0030] (2.2) The intermediate block is constructed by two consecutively stacked 3×3 convolution blocks to encode deep features. Processing to generate transition features
[0031] (2.3) The first decoder D is constructed by cascading two sub-decoders D1 and D2 with the same structure. Both D1 and D2 are composed of stacked deconvolution blocks and convolution blocks. The transition feature F I Input decoder D to generate the first decoding feature and the second decoding feature
[0032] (2.4) The upsampling layer and two consecutively stacked 3×3 convolution blocks form a sub-upsampling block, and the upsampling block is constructed using three sub-upsampling blocks; I 、F D1 and F D2 As the input of the upsampling block, each of them goes through a sub-upsampling block to restore the original resolution, that is, to obtain the upsampling features of the k-th scale branch k=1,2,3;
[0033] In this embodiment, the convolution blocks used in the above steps (2.1) to (2.4) all include a 3×3 convolution layer, a batch normalization layer BN, and an activation function layer; the deconvolution block used in step (2.3) includes a 3×3 deconvolution, a batch normalization layer BN, and an activation function layer.
[0034] (2.5) Build the CDFL module and convert F E1 、F E2 、F I and F D1As its input, the cosine distance map is obtained by calculating the cosine similarity of the encoding feature, transition feature and the first decoding feature. The specific calculation process is as follows:
[0035] Calculate the encoding feature F E2 and intermediate feature F I Cosine distance graph M between EI :
[0036]
[0037] in, Its range is [0,1];
[0038] Calculate the encoding feature F E1 and decoding features F D1 Cosine distance graph M between ED :
[0039]
[0040] in, Its range is [0,1]. Then, the formula of CDFL loss function is defined as follows:
[0041]
[0042] in, and Indicates that in M f 4x downsampling and 2x downsampling are performed on the above.
[0043] (2.6) Construct the MMST module and convert Z k As its input, the background samples and abnormal samples are separated and trained to obtain the memory features of the retrieval Here are the steps:
[0044] (2.6.1) Based on M f The abnormal characteristics of the k-th branch are calculated according to the following formula:
[0045]
[0046] Among them, ⊙ represents the element-by-element multiplication operation; E(·) represents the expansion operation along the dimension direction, which is used to match Z k Dimensions;
[0047] (2.6.2) Based on M f The background features of the kth branch are calculated according to the following formula:
[0048]
[0049] in, Indicates Mf The negation operation;
[0050] (2.6.3) by flattening Get two-dimensional background features Use it as the input of the multi-scale memory MMST module; calculate the background features of the nth reconstruction
[0051]
[0052] in, Represents the memory matrix The transpose of m i ∈M represents the i-th memory item, express The addressing weight of the nth row in the ni Indicates w n The i-th addressing weight in is calculated as follows:
[0053]
[0054] Where s(·,·) represents the similarity evaluation, and its formula is:
[0055]
[0056] in, express The nth background feature in , ||·||1 represents the L1 norm.
[0057] (2.6.4) Two-dimensional background features Reshape into 3D reconstructed background features The reconstructed sample features are obtained according to the following formula
[0058]
[0059] in, Represents an element-wise addition operation.
[0060] (2.7) Construct decoder M, and use decoder M to remember features Decode to get the reconstructed image The decoder M consists of decoder M1, decoder M2 and decoder M3, and each decoder consists of a 1×1 convolution block and a 1×1 convolution layer;
[0061] (2.8) Construct the CPLG module by calculating the hyperspectral image X and the reconstructed image The reconstruction error between the two generates a pseudo label M f, and use this label to guide the separation training of the MMST module, and at the same time serve as a pseudo-supervisory label for training the CDFL module to supervise the optimization of the cosine distance map;
[0062] In this embodiment, the hyperspectral image X and the reconstructed image are calculated. The reconstruction error between the two generates a pseudo label M f , the specific steps are as follows:
[0063] (2.8.1) Calculate the error value corresponding to the k-th branch at position (i, j)
[0064]
[0065] Among them, ||·||2 represents the L2 norm, x i,j ∈X represents the spectral vector at position (i, j), Represents the reconstructed spectral vector at the position (i, j) corresponding to the kth branch; the error map is obtained by calculating the error value of each position
[0066] (2.8.2) Obtain a comprehensive error map by aggregating k error maps The formula is as follows:
[0067] E m =max(E 1 ,E 2 ,E 3 );
[0068] Among them, max(·) represents the element-by-element maximum value operation;
[0069] (2.8.3) Sort E in descending order m The values in , and the position index of the top t% maximum value is recorded as a set P, and then the rough pseudo label M is obtained by the following definition c :
[0070]
[0071] in, Indicates that the label at position (i, j) belongs to a pseudo-abnormal label, Indicates that the label at position (i, j) belongs to the pseudo background label;
[0072] (2.8.4) Design a pseudo-anomaly label removal strategy PALR based on dual-window spectral similarity, including c Map onto X to get the mapping M X , then in M X A double window is constructed to calculate the double window spectral similarity, that is, the inner window spectral similarity and the outer window spectral similarity, to obtain the fine pseudo label Mf :
[0073] (2.8.4a) Calculate the similarity between the central spectral vector and the inner window region spectral vector to obtain the initial inner window spectral similarity
[0074]
[0075] Among them, P l c Indicates M c The lth spectral vector within the rough pseudo-anomaly label, Indicates P i The mth spectral vector in N i Indicates P i The number of spectral vectors in ; defines the set of inner window spectral similarities Where L represents M c The number of coarse pseudo-anomaly labels in ; the lth final inner window spectral similarity is obtained by normalization operation, and the formula is as follows:
[0076]
[0077] Among them, Max{·} and Min{·} represent maximization and minimization operations, respectively;
[0078] (2.8.4b) Calculate P l c The similarity between the spectral vectors in the outer window area is used to obtain the initial outer window spectral similarity
[0079]
[0080] in, Indicates P o The mth spectral vector in N o Indicates P o The number of spectral vectors in ; definition Collection The lth final external window spectrum similarity is obtained through normalization operation, and the formula is as follows:
[0081]
[0082] If the lth dual-window spectrum similarity satisfies both and Place the position in P l c The pseudo anomaly label on is modified to the pseudo background label, where τ represents the similarity threshold; finally, the pseudo label that meets the above conditions is updated to obtain the updated fine pseudo label M f .
[0083] Step 3. Use the L2 norm and the designed CDFL loss function L CDFL As the joint loss function L, it guides the model training to convergence and obtains the trained final detection model;
[0084] The loss function L used in this embodiment is expressed as follows:
[0085]
[0086] Among them, ||·||2 represents the L2 norm, and λ represents the trade-off parameter used to balance the two losses.
[0087] Step 4. Input the original hyperspectral image X into the final detection model to obtain the reconstructed hyperspectral image according to The anomaly detection result is obtained by calculating X. In this embodiment, the three The sum of the two norms of the difference between the corresponding pixel in X and the pixel in X, and then square it to get:
[0088]
[0089] where r ij Represents the anomaly detection result at position (i, j).
[0090] Example 2: The overall implementation steps of the hyperspectral anomaly detection method proposed in this example are the same as those in Example 1. Figure 1 (a) gives specific parameter settings to further describe the detection model constructed in the present invention in detail:
[0091] Step 1: Reference Figure 1 As shown in part (a), the present invention constructs a multi-scale memory network MSMNet (Multi-scale Memory Network) model based on separation training as a hyperspectral anomaly detection model, including an encoder E, an intermediate block, a first decoder D, an upsampling block, a coarse-to-fine pseudo-label generation CPLG module, a multi-scale memory MMST module based on separation training, a consistency and discriminative feature learning CDFL module, and a second decoder M; the processing process is as follows:
[0092] a) Given an original hyperspectral image The dimensionality-reduced hyperspectral image is obtained by using the Orthogonal Projection-based Band Selection (OPBS) algorithm [Zhang W, Li X, Dou Y, et al. A geometry-based band selection approach for hyperspectral image analysis [J]. IEEE transactions on geoscience and remote sensing, 2018, 56(8): 4318-4333.] Where 100 and 100 represent the height and width of the hyperspectral image, and 191 and 64 represent the number of bands of the hyperspectral image;
[0093] b) Input X into the encoder E to obtain encoding features of different scales and
[0094] c) F E2 Generate transition features through intermediate blocks
[0095] d) F I Input decoder D to generate decoding features and
[0096] e) F I , F D1 and F D2 Input their corresponding upsampling blocks respectively to restore the original resolution, that is, obtain the upsampling features of the k-th scale branch where k = {1, 2, 3};
[0097] f) Z k Input MMST to obtain retrieved memory features
[0098] g) Input decoder M to obtain the reconstructed hyperspectral image Then X and Input CPLG module to generate pseudo label M f Finally, F E1 , F E2 , F I and F D1 Input the CDFL module to calculate the cosine distance map and use M f Serves as pseudo-supervisory labels for training the CDFL module.
[0099] For model training, this embodiment uses the L2 norm and the designed CDFL loss function as the loss function L to guide model training, as follows:
[0100]
[0101] Among them, 0.5 indicates that the trade-off parameter is used to balance the losses of the two.
[0102] After the model training is completed, X is input into the trained model to obtain the reconstructed hyperspectral image. Then, the test results are obtained by the following formula:
[0103]
[0104] where r ij Represents the anomaly detection result at position (i, j).
[0105] Example 3: The overall implementation steps of the hyperspectral anomaly detection method proposed in this example are the same as those in Example 1. Figure 1 (b)-(d) in the figure further explain the innovations in the model, including the coarse-to-fine pseudo-label generation CPLG module, the multi-scale memory MMST module based on separation training, and the consistency and discriminative feature learning CDFL module:
[0106] (1) Coarse-to-fine pseudo labels generation CPLG (Coarse-to-fine pseudo labels generation) module, such as Figure 1 As shown in (b):
[0107] The CPLG module is designed to generate pseudo labels M f , which serves as a guide for the separate training of the MMST module and is used as a pseudo-supervisory label for training the CDFL module; M f The generation process includes the following two steps:
[0108] 1) Rough pseudo-label generation:
[0109] 1a) Calculate the error value corresponding to the k-th branch at position (i, j) using the following formula
[0110]
[0111] Among them, ||·||2 represents the L2 norm, x i,j ∈X represents the spectral vector at position (i, j), Represents the spectrum vector reconstructed at the position (i, j) corresponding to the kth branch
[0112] 1b) First, we calculate the error value at each position (i, j) to obtain the error map Then, a comprehensive error map is obtained by aggregating k error maps. The formula is as follows:
[0113] E m =max(E 1 ,E 2 ,E 3 ) (2)
[0114] Among them, max(·) represents the element-by-element maximum value operation.
[0115] Finally, sort E in descending order m The values in , and the position index of the top 1% maximum value is recorded as set P. Then, the coarse pseudo label is obtained by the following definition:
[0116]
[0117] in, Indicates that the label at position (i, j) belongs to a pseudo-abnormal label, Indicates that it belongs to the pseudo background label.
[0118] 2) Fine pseudo-label generation: rough pseudo-label M c In order to obtain fine pseudo labels, this paper proposes a pseudo anomaly label removal (PALR) strategy based on dual-window spectral similarity, which is to remove the M c Map onto X to get the mapping M X , then in M X A double window is constructed to calculate the double window spectral similarity (inner window spectral similarity and outer window spectral similarity), as follows:
[0119] 2a) Initial inner window spectrum similarity It is obtained by calculating the similarity between the central spectral vector and the spectral vector of the inner window area. The formula is as follows:
[0120]
[0121] Among them, P l c Indicates M c The lth spectral vector within the rough pseudo-anomaly label, Indicates P i The mth spectral vector in . 8 represents P i Then, define the set of inner window spectral similarities Where L represents Mc Finally, the lth final inner window spectral similarity is obtained through normalization operation, and the formula is as follows:
[0122]
[0123] Among them, Max{·} and Min{·} represent maximization and minimization operations respectively.
[0124] 2b) Initial external window spectrum similarity By calculating P l c The similarity between the spectral vectors in the outer window area is obtained using the following formula:
[0125]
[0126] in, Indicates P o The mth spectral vector in , 40 represents P o Then, define Collection The lth final external window spectrum similarity is obtained through normalization operation, and the formula is as follows:
[0127]
[0128] If the lth dual-window spectrum similarity satisfies both and Place the position in P l c The pseudo anomaly labels on are modified to pseudo background labels, where 0.5 represents the similarity threshold. Finally, the pseudo labels that meet the above conditions are updated to obtain the updated fine pseudo labels M f .
[0129] (2) Multi-scale Memory with Separation Training (MMST) module:
[0130] The design purpose of the MMST module is to weaken the model's ability to reconstruct abnormal samples and improve the separation between abnormalities and background. Figure 1 As shown in (c) in .
[0131] A) Based on M f , the abnormal characteristics of the k-th branch The calculation formula is as follows:
[0132]
[0133] Where ⊙ represents the element-wise multiplication operation, E(·) represents the expansion operation along the dimension direction to match Z k Similarly, the background features of the k-th branch The calculation formula is as follows:
[0134]
[0135] in, Indicates M f The negation of is defined as
[0136] B) By flattening Get two-dimensional background features As the input of the memory module. Then, the n-th reconstructed background feature The calculation is as follows:
[0137]
[0138] in, Represents the memory matrix The transpose of m i ∈M represents the i-th memory item, express The addressing weight of the nth row in the ni Indicates w n The i-th addressing weight in is calculated as follows:
[0139]
[0140] Where s(·,·) represents the similarity evaluation, and its formula is:
[0141]
[0142] in, express The nth background feature in , ||·||1 represents the L1 norm.
[0143] C) Two-dimensional background features Reshape into 3D reconstructed background features Finally, the reconstructed sample features are expressed as follows:
[0144]
[0145] in Represents an element-wise addition operation.
[0146] (3) Consistent and discriminative feature learning CDFL (Consistent and discriminative feature learning) module, such as Figure 1 As shown in (d), the processing process is as follows:
[0147] Calculate the encoding feature F E2 and intermediate feature F I Cosine distance graph M between EI as follows:
[0148]
[0149] in, Its range is [0,1].
[0150] Similarly, the encoding feature F E1 and decoding features F D1 Cosine distance graph M between ED The calculation is as follows:
[0151]
[0152] in, Its range is [0,1]. Then, the formula of CDFL loss function is defined as follows:
[0153]
[0154] in, and Indicates that in M f 4x downsampling and 2x downsampling are performed on the above.
[0155] The effects of the present invention will be further described below with reference to experiments.
[0156] 1. Experimental conditions:
[0157] The experiments of the present invention were carried out in a hardware environment with a CPU main frequency of 3.00 GHz, a memory of 48 GB, a Windows 10 operating system, and a software environment of Python 3.7.
[0158] 2. Experimental content:
[0159] The proposed method is qualitatively and quantitatively compared with seven popular algorithms on three public hyperspectral anomaly detection datasets. The public hyperspectral anomaly detection datasets used in the experiment include the Texas Coast, Salinas, and San Diego datasets; the seven popular algorithms compared are: RX[ISReed,and X.Yu,“Adaptive multiple-bandCFAR detection of an optical pattern with unknown spectral distribution,”IEEETrans.Acoust.Speech Signal Process.,vol.38,no.10,pp.1760-1770.], CRD[W.Li andQ.Du,“Collaborative representation for hyperspectral anomaly detection,”IEEETransactions on Geoscience and Remote Sensing,vol.53,no.3,pp.1463-1474.], RGAE[Fan G,Ma Y,Mei X,et al.Hyperspectral anomaly detection with robust graphautoencoders[J].IEEE Transactions on Geoscience and Remote Sensing,2021,60:1-14.], GAED[P.Xiang,S.Ali,SKJung et al., "Hyperspectral anomaly detection with guided autoencoder," IEEE Transactions on Geoscience and Remote Sensing, vol.60, pp.1-18, Sep., 2022. Art no.5538818.], MTVLRR [Li L, Wu Z, WangB. Hyperspectral anomaly detection via merging total variation into low-rankrepresentation[J].IEEE Journal of Selected Topics in Applied EarthObservations and Remote Sensing,2024,17:14894-14907.],GT-HAD[Lian J,Wang L,Sun H,et al. Systems, 2024, 36(2): 3631-3645.], MSNet [Liu H, Su .
[0160] This paper first conducts a qualitative comparison with seven popular comparison algorithms on three public hyperspectral anomaly detection datasets. Figure 2 It can be seen that the detection results of the present invention are better than those of the comparison algorithms. At the same time, the present invention is quantitatively compared with seven popular comparison algorithms on three data sets, and the evaluation index used is the detection rate AUC (P d ,P f ). Among them, the detection rate AUC (P d ,P f ) is larger, the better the performance of detecting abnormal pixels is. As shown in Table 2, the present invention has the best detection rate.
[0161] Table 2: AUC (P d ,P f )Indicator comparison
[0162]
[0163] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, stored data, displayed data, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of relevant data must comply with the laws, regulations and standards of relevant countries and regions, and provide corresponding operation entrances for users to choose to authorize or refuse.
[0164] The above simulation analysis proves the correctness and effectiveness of the method proposed in the present invention.
[0165] Parts of the present invention that are not described in detail belong to common knowledge among those skilled in the art.
[0166] The above description is only a preferred embodiment of the present invention and is not intended to limit the present invention. Obviously, for professionals in this field, after understanding the content and principles of the present invention, they may make various modifications and changes in form and details without departing from the principles and structure of the present invention. However, these modifications and changes based on the ideas of the present invention are still within the scope of protection of the claims of the present invention.
Claims
1. A hyperspectral anomaly detection method based on a multi-scale memory network, characterized in that: The steps include: (1) Obtaining the original hyperspectral image Obtaining dimensionality-reduced hyperspectral images through a band selection algorithm based on orthogonal projection Where H and W represent the height and width of the hyperspectral image, and B and C represent the number of bands of the hyperspectral image; (2) A hyperspectral anomaly detection model is constructed based on a multi-scale memory network. The model includes an encoder E, an intermediate block, a first decoder D, an upsampling block, a coarse-to-fine pseudo-label generation CPLG module, a multi-scale memory MMST module based on separation training, a consistency and discriminative feature learning CDFL module, and a second decoder M. The implementation steps are as follows: (2.1) The encoder E is constructed by cascading two sub-encoders E1 and E2 with the same structure. Both E1 and E2 consist of two continuously stacked 3×3 convolution blocks, which are used to encode X to obtain shallow coding features. and deep encoding features (2.2) The intermediate block is constructed by two consecutively stacked 3×3 convolution blocks to encode deep features. Processing to generate transition features (2.3) The first decoder D is constructed by cascading two sub-decoders D1 and D2 with the same structure. Both D1 and D2 are composed of stacked deconvolution blocks and convolution blocks. The transition feature F I Input decoder D to generate the first decoding feature and the second decoding feature (2.4) The upsampling layer and two consecutively stacked 3×3 convolution blocks form a sub-upsampling block, and the upsampling block is constructed using three sub-upsampling blocks; I 、F D1 and F D2 As the input of the upsampling block, each of them goes through a sub-upsampling block to restore the original resolution, that is, to obtain the upsampling features of the k-th scale branch (2.5) Build the CDFL module and convert F E1 、F E2 、F I and F D1 As its input, the cosine distance map is obtained by calculating the cosine similarity of the encoding feature, the transition feature and the first decoding feature; (2.6) Construct the MMST module and convert Z k As its input, the background samples and abnormal samples are separated and trained to obtain the memory features of the retrieval (2.7) Construct decoder M, and use decoder M to remember features Decode to get the reconstructed image The decoder M consists of decoder M1, decoder M2 and decoder M3, and each decoder consists of a 1×1 convolution block and a 1×1 convolution layer; (2.8) Construct the CPLG module by calculating the hyperspectral image X and the reconstructed image The reconstruction error between the two generates a pseudo label M f , and use this label to guide the separation training of the MMST module, and at the same time serve as a pseudo-supervisory label for training the CDFL module to supervise the optimization of the cosine distance map; (3) Using L2 norm and designed CDFL loss function L CDFL As the joint loss function L, it guides the model training to convergence and obtains the trained final detection model; (4) Input the original hyperspectral image X into the final detection model to obtain the reconstructed hyperspectral image according to Calculate with X to obtain the anomaly detection result.
2. The method according to claim 1, wherein: The convolution blocks in steps (2.1)-(2.4) all contain a 3×3 convolution layer, a batch normalization layer BN, and an activation function layer; the deconvolution block in step (2.3) contains a 3×3 deconvolution, a batch normalization layer BN, and an activation function layer.
3. The method according to claim 1, wherein: In (2.5), the cosine distance graph is obtained by calculating the cosine similarity of the encoding feature, the transition feature, and the first decoding feature, specifically: Calculate the encoding feature F E2 and intermediate feature F I Cosine distance graph M between EI : in, Its range is [0,1]; Calculate the encoding feature F E1 and decoding features F D1 Cosine distance graph M between ED : in, Its range is [0,1]. Then, the formula of CDFL loss function is defined as follows: in, and Indicates that in M f 4x downsampling and 2x downsampling are performed on the above.
4. The method according to claim 1, wherein: The MMST module in (2.6) obtains the memory features according to the following steps (2.6.1) Based on M f The abnormal characteristics of the k-th branch are calculated according to the following formula: Among them, ⊙ represents the element-by-element multiplication operation; E(·) represents the expansion operation along the dimension direction, which is used to match Z k Dimensions; (2.6.2) Based on M f The background features of the kth branch are calculated according to the following formula: in, Indicates M f The negation operation; (2.6.3) by flattening Get two-dimensional background features Use it as the input of the multi-scale memory MMST module; calculate the background features of the nth reconstruction in, Represents the memory matrix The transpose of m i ∈M represents the i-th memory item, express The addressing weight of the nth row in the ni represents w n The i-th addressing weight in is calculated as follows: Where s(·,·) represents the similarity evaluation, and its formula is: in, express The nth background feature in , ||·||1 represents the L1 norm. (2.6.4) Two-dimensional background features Reshape into 3D reconstructed background features The reconstructed sample features are obtained according to the following formula in, Represents an element-wise addition operation.
5. The method according to claim 1, wherein: In step (2.8), the CPLG module calculates the hyperspectral image X and the reconstructed image The reconstruction error between the two generates a pseudo label M f , the implementation steps are as follows: (2.8.1) Calculate the error value corresponding to the k-th branch at position (i, j) Among them, ||·||2 represents the L2 norm, x i,j ∈X represents the spectral vector at position (i, j), Represents the reconstructed spectral vector at the position (i, j) corresponding to the kth branch; the error map is obtained by calculating the error value of each position (2.8.2) Obtain a comprehensive error map by aggregating k error maps The formula is as follows: AND m =max(E 1 ,AND 2 ,AND 3 ); Among them, max(·) represents the element-by-element maximum value operation; (2.8.3) Sort E in descending order m The values in , and the position index of the top t% maximum value is recorded as a set P, and then the rough pseudo label M is obtained by the following definition c : in, Indicates that the label at position (i, j) belongs to a pseudo-abnormal label, Indicates that the label at position (i, j) belongs to the pseudo background label; (2.8.4) Design a pseudo-anomaly label removal strategy PALR based on dual-window spectral similarity, including c Map onto X to get the mapping M X , then in M X A double window is constructed to calculate the double window spectral similarity, that is, the inner window spectral similarity and the outer window spectral similarity, to obtain the fine pseudo label M f .
6. The method according to claim 5, characterized in that: The fine pseudo label M f , specifically according to the following steps: (2.8.4a) Calculate the similarity between the central spectral vector and the inner window region spectral vector to obtain the initial inner window spectral similarity Among them, P l c Indicates M c The lth spectral vector within the rough pseudo-anomaly label, Indicates P i The mth spectral vector in N i Indicates P i The number of spectral vectors in ; defines the set of inner window spectral similarities Where L represents M c The number of coarse pseudo-anomaly labels in ; the lth final inner window spectral similarity is obtained by normalization operation, and the formula is as follows: Among them, Max{·} and Min{·} represent maximization and minimization operations, respectively; (2.8.4b) Calculate P l c The similarity between the spectral vectors in the outer window area is used to obtain the initial outer window spectral similarity in, Indicates P o The mth spectral vector in N o Indicates P o The number of spectral vectors in ;;Define Collection The lth final external window spectrum similarity is obtained through normalization operation, and the formula is as follows: If the lth dual-window spectrum similarity satisfies both and Place the position in P l c The pseudo anomaly label on is modified to the pseudo background label, where τ represents the similarity threshold; finally, the pseudo label that meets the above conditions is updated to obtain the updated fine pseudo label M f .
7. The method according to claim 1, wherein: The loss function L in step (3) is expressed as follows: Among them, ||·||2 represents the L2 norm, and λ represents the trade-off parameter used to balance the two losses.
8. The method according to claim 1, wherein: The abnormality detection result in step (4) is specifically obtained by calculating three The sum of the two norms of the difference between the corresponding pixel in X and the pixel in X, and then square it to get: where r ij Represents the anomaly detection result at position (i, j).