Spatial-spectral feature enhancement and SAM (Spatial Amplitude Modulation) guided hyperspectral multi-class change detection method
Through the hyperspectral multi-class variation detection method of null spectral feature enhancement and SAM-guided, combined with multi-scale encoder and wavelet transformation, the problem of spatial features being ignored in hyperspectral change detection is solved, and efficient and accurate multi-class variation detection is achieved.
Patent Information
- Application Number
- CN202510545972.7
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2025-04-27
- Publication Date
- 2025-08-08
AI Technical Summary
The existing hyperspectral change detection methods ignore spatial characteristics, are susceptible to noise interference, have low accuracy, and are inefficient in hyperspectral data processing and are difficult to batch detection.
The hyperspectral multi-class variation detection method with null spectral feature enhancement and SAM-guided is adopted to extract shallow features through 3×3 convolution, and the feature map is processed using a multi-scale encoder and a differential module, combining wavelet transform to fusion spatial domain and frequency domain features to enhance the detection of the change region.
It improves the accuracy and efficiency of multi-class variation detection of hyperspectral images, can accurately detect small area changes, reduce error detection, and is suitable for batch detection of hyperspectral images.
Smart Images
Figure CN120451788A_ABST
Abstract
Description
Technical Field
[0001] The present invention specifically relates to a hyperspectral multi-class change detection method based on spatial spectrum feature enhancement and SAM guidance. Background Art
[0002] Hyperspectral remote sensing images have extensive application value in environmental monitoring, agricultural assessment, urban change detection and other fields due to their rich spectral information and spatial resolution. Among them, hyperspectral multi-class change detection aims to identify and classify various types of changes that occur in the same area when imaging at different time points, such as vegetation degradation, land development, and natural disaster impacts. Accurately and efficiently detecting and classifying these changes is crucial for land resource management, ecological environment monitoring and disaster assessment.
[0003] However, traditional hyperspectral change detection methods mainly rely on statistical analysis methods such as spectral difference, principal component analysis, and change vector analysis. Although these methods can effectively extract some change information, they often ignore the role of spatial features, are easily affected by noise, and have low accuracy.
[0004] In recent years, models such as convolutional neural networks and Transformer have been able to learn deep spectral-spatial features and improve the accuracy of hyperspectral multi-class change detection. However, convolutional neural networks are limited by their local receptive fields and have difficulty capturing long-range spectral dependencies. Transformer processing of hyperspectral data has high computational complexity, making it difficult to batch detect hyperspectral remote sensing images and inefficient.
[0005] It can be seen that how to fully exploit the spatial-spectral characteristics of hyperspectral images and improve the accuracy and efficiency of hyperspectral multi-class change detection is an important direction of current research.
[0006] Therefore, it is necessary to invent a hyperspectral multi-class change detection method with spatial spectrum feature enhancement and SAM guidance to solve the above problems. Summary of the Invention
[0007] The purpose of the present invention is to provide a hyperspectral multi-class change detection method with spatial spectrum feature enhancement and SAM guidance to address the above-mentioned deficiencies in the technology.
[0008] To achieve the above objectives, the present invention provides the following technical solution: a hyperspectral multi-class change detection method with spatial spectrum feature enhancement and SAM guidance, comprising the following steps:
[0009] Step 1: Given a pair of dual-time images T1 and T2, perform the difference to obtain the difference image T D , use 3×3 convolution to extract the shallow features f of T1 pre , shallow features f of T2 pos 、T DThe shallow features f sub ;
[0010] Step 2: The shallow feature f pre , shallow features f pos , shallow features f sub Input into the multi-scale encoder SGSEM for processing to obtain the intermediate layer feature F 1 pre / pos / sub , intermediate layer features F 2 pre / pos / sub , intermediate layer features F 3 pre / pos / sub ;
[0011] Step 3: The intermediate layer feature F 1 pre / pos / sub , the intermediate layer feature F 2 pre / pos / sub , the intermediate layer feature F 3 pre / pos / sub Input into the differential module DM to obtain the fine feature E i ;
[0012] Step 4: Use the multi-scale feature fusion decoder WMAM to process the fine feature map E i , get the intermediate layer feature G 1 i ;
[0013] Step 5: Fusion feature G 1 i After two fully connected layers, the final multi-class change detection result graph T is obtained map .
[0014] The aforementioned spatial spectrum feature enhancement and SAM-guided hyperspectral multi-class change detection method, in step 1, given a pair of dual-time images T1 and T2, the difference image T is obtained by subtraction. D , use 3×3 convolution to extract the shallow features f of T1 pre , shallow features f of T2 pos 、T D The shallow features f sub The specific process is as follows:
[0015] The first time image T1 uses 3×3 convolution to obtain the shallow feature f pre ;
[0016] The second time image T2 uses 3×3 convolution to obtain the shallow feature f pos ;
[0017] The difference image T is obtained by subtracting the dual time image pair. D , difference graph T D Use 3×3 convolution to get shallow features fsub
[0018] Among them, the shallow feature f pre / pos / sub = shallow feature f pre , shallow features f pos , shallow features f sub .
[0019] The aforementioned spatial spectrum feature enhancement and SAM-guided hyperspectral multi-class change detection method, in step 2, the shallow feature f pre , shallow features f pos , shallow features f sub Input into the multi-scale encoder SGSEM for processing to obtain the intermediate layer feature F 1 pre / pos / sub , intermediate layer features F 2 pre / pos / sub , intermediate layer features F 3 pre / pos / sub The specific steps are as follows:
[0020] 2.1、The shallow feature f pre , shallow features f pos , shallow features f sub Input it into the first SGSEB module of the multi-scale encoder SGSEM, and model it along the three dimensions of H, W, and C through three state space models to obtain the spectral feature f pre_H , spectral characteristics f pre_W , spatial features f pre_C , spectral characteristics f pos_H , spectral characteristics f pos_W , spatial features f pos_C , spectral characteristics f sub_H , spectral characteristics f sub_W , spatial features f sub_C ;
[0021] 2.2、Spectral feature f pre_H , spectral characteristics f pos_H , spectral characteristics f sub_H With the spectral characteristics f pre_W , spectral characteristics f pos_W , spectral characteristics f sub_W One-to-one correspondence and weighted fusion, and then respectively combined with the spatial feature f pre_C , spatial features f pos_C , spatial features f sub_C Multiply to get the intermediate layer feature F 1 pre , intermediate layer features F 1 pos , intermediate layer features F 1 sub, the specific formula is as follows:
[0022] F 1 pre =f pre_C ×(ω 11 *f pre_H +ω 21 *f pre_W )
[0023] F 1 pos =f pos_C ×(ω 12 *f pos_H +ω 22 *f pos_W )
[0024] F 1 sub =f sub_C ×(ω 13 *f sub_H +ω 23 *f sub_W )
[0025] Among them, ω 11 f pre_H The weight value of
[0026] ω 12 f pos_H The weight value of
[0027] ω 13 f sub_H The weight value of
[0028] ω 21 f pre_W The weight value of
[0029] ω 22 f pos_W The weight value of
[0030] ω 23 f sub_W The weight value of
[0031] Intermediate layer feature F 1 pre / pos / sub =Intermediate layer feature F 1 pre , intermediate layer features F 1 pos , intermediate layer features F 1 sub ;
[0032] 2.3. The intermediate layer feature F 1 pre / pos / subInput into the second SGSEB module to obtain the intermediate layer feature F 2 pre / pos / sub , the specific formula is:
[0033] F 2 pre / pos / sub =SGSEB(F 1 pre / pos / sub );
[0034] 2.4. The intermediate layer feature F 2 pre / pos / sub Input into the third SGSEB module to obtain the intermediate layer feature F 3 pre / pos / sub , the specific formula is:
[0035] F 3 pre / pos / sub =SGSEB(Upsampling(F 2 pre / pos / sub ))
[0036] Here, Upsampling(·) indicates double upsampling.
[0037] The aforementioned spatial spectrum feature enhancement and SAM-guided hyperspectral multi-class change detection method, in step 2.1, the three state space models are SSM _H 、SSM _W 、SSM _C , specifically:
[0038] SSM _H The WC plane is scanned from the upper left to the lower right, and the state space model is built. The specific formula is:
[0039] f pre_H 、f pos_H 、f sub_H =SSM _H (f pre / pos / sub );
[0040] SSM _W The HC plane is scanned from the upper left to the lower right, and the state space model is built. The specific formula is:
[0041] f pre_W 、f pos_W 、f sub_W =SSM _W (f pre / pos / sub );
[0042] SSM _C Scan the HW plane from the upper left to the lower right and build a state space model. The specific formula is:
[0043] f pre_C 、f pos_C 、f sub_C =SSM _C (f pre / pos / sub );
[0044] H is the shallow feature f pre / pos / sub height;
[0045] W is the shallow feature f pre / pos / sub width;
[0046] C is the shallow feature f pre / pos / sub Channel;
[0047] The multi-scale encoder SGSEM is composed of three SGSEB modules connected in series.
[0048] The aforementioned spatial spectrum feature enhancement and SAM-guided hyperspectral multi-class change detection method, in step 3, the intermediate layer feature F 1 pre / pos / sub , the intermediate layer feature F 2 pre / pos / sub , the intermediate layer feature F 3 pre / pos / sub Input into the differential module DM to obtain the fine feature E i The specific steps are as follows:
[0049] 3.1. Processing the difference graph T through unsupervised methods D , get a rough two-class change mask f i seg , the specific formula is:
[0050] f i seg =pre-model(T D )
[0051] Among them, pre-model(·) is an unsupervised method for pre-training;
[0052] 3.2. The rough two-class change mask f i seg As a hint for the SAM model, the difference graph T D The segmentation map is obtained by channel segmentation, and the segmentation map is selectively fused by band selection to obtain an accurate two-category mask map F i seg , the specific formula is:
[0053] F i seg =Bandselect(SAM(T D , f iseg ))
[0054] Among them, SAM(·) is the segmentation operation of all models.
[0055] Bandselect(·) is the process of fusing the channel-by-channel segmentation masks;
[0056] 3.3, the middle layer feature F 1 pre / pos / sub , intermediate layer features F 2 pre / pos / sub , and the intermediate layer features F 3 pre / pos / sub All are fused through 1×1 convolution to obtain a rough differential feature map e i ,
[0057] The specific formula is as follows:
[0058] e i =Conv(Cat(F i pre -F i pos , F i sub ))
[0059] Among them, Conv(·) is a 1×1 convolution operation;
[0060] Cat(·) is the concatenation operation along the channel dimension;
[0061] 3.4. Use F i seg The coarse features are analyzed by SAM-KGFEM module. i Enhance the change area features to obtain the fine feature map E i , the specific formula is as follows:
[0062] E i =Conv(Cat(e i ×sigmoid(F i seg )+e i , e i ×sigmoid(1-F i seg )+e i ))
[0063] Among them, sigmoid(·) is a normalization operation.
[0064] The aforementioned spatial spectrum feature enhancement and SAM-guided hyperspectral multi-class change detection method uses a multi-scale feature fusion decoder WMAM to process the fine feature map E in step 4.i , get the intermediate layer feature G 1 i The specific steps are as follows:
[0065] 4.1. The fine feature map E is transformed into i Wavelet decomposition is into frequency domain features, the specific formula is as follows:
[0066] E i_H , E i_L =DWT(E i )
[0067] Among them, the frequency domain features include high frequency features E i_H , and the low-frequency feature E i_L ;
[0068] DWT(·) is the process of wavelet decomposition;
[0069] And high-frequency features contain texture information;
[0070] Low-frequency features contain color information, category information, and semantic information;
[0071] The multi-scale feature fusion decoder WMAM is composed of multiple WMAB modules in series;
[0072] 4.2. Low-frequency feature E i_L Perform state space modeling to extract the original color, category, and semantic information in the image and obtain E i_Lssm , the specific formula is as follows:
[0073] E i_Lssm =SSM(E i_L )
[0074] Among them, SSM(·) is the process of modeling the state space model;
[0075] 4.3. High-frequency features E i_H As q, k, low-frequency features E i_L As v input to the self-attention mechanism for high-frequency features E i_H Enhance and get E i_Henhanced , the specific formula is as follows:
[0076] E i_Henhanced =Attention(E i_H , E i_L )
[0077] Among them, Attention(·) represents the self-attention mechanism;
[0078] 4.4. Processing E by inverse wavelet decomposition i_Lssm and Ei_Henhanced , and obtain the spatial domain feature E 1 i , the specific formula is as follows:
[0079] E 1 i =IWT(E i_Henhanced , E i_Lssm )
[0080] Among them, IWT(·) is the process of inverse operation of wavelet decomposition;
[0081] 4.5 Spatial Domain Features E 1 i Through 1×1 convolution, the fusion feature G is obtained after fusion. 1 i , the process formula is as follows:
[0082] G 1 i =Conv(Cat(E 1 i ,Downsampling(E 1 i )))
[0083] Here, Downsampling(·) represents the process of downsampling by a factor of two.
[0084] Compared with the prior art, the present invention has the following beneficial effects:
[0085] The present invention enhances spatial information through rich spectral information, mines the spatial spectrum characteristics of hyperspectral images, improves the accuracy of change detection in small areas, and enhances the rough differential feature map to increase attention to the change area, reduce false detection, and combines spatial domain features with frequency domain features through wavelet transform. The texture information in the high-frequency features makes the edges of the change area more accurate, and the color information, category information, and semantic information in the low-frequency features can better detect the change areas of different categories, thereby further improving the change detection accuracy. In summary, when processing hyperspectral image pairs, the present invention can fully extract the spatial features, spectral features, semantic information and texture information of hyperspectral images, thereby improving the accuracy of multi-category change detection in hyperspectral images, without the need for complex calculations, and can be applied to batch detection of hyperspectral images, with fast detection efficiency and high accuracy. BRIEF DESCRIPTION OF THE DRAWINGS
[0086] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the following briefly introduces the drawings required for use in the embodiments. Obviously, the drawings described below are only some embodiments described in the present invention. For ordinary technicians in this field, other drawings can also be obtained based on these drawings.
[0087] Figure 1 is a flow chart of the present invention;
[0088] Figure 2 The hyperspectral pre-phase image and the hyperspectral post-phase image given in the present invention;
[0089] Figure 3 This is a graph of multi-class change detection results obtained by the present invention;
[0090] Figure 4 Figure 2 is the actual change detection result of a given dual-phase image. DETAILED DESCRIPTION
[0091] In order to enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.
[0092] The present invention provides Figure 1-4 The method for hyperspectral multi-class change detection using spatial spectrum feature enhancement and SAM guidance is shown, comprising the following steps:
[0093] Step 1: Given a pair of dual-time images T1 and T2, perform the difference to obtain the difference image T D , use 3×3 convolution to extract the shallow features f of T1 pre , shallow features f of T2 pos 、T D The shallow features f sub The specific process is as follows:
[0094] The first time image T1 uses 3×3 convolution to obtain the shallow feature f pre ;
[0095] The second time image T2 uses 3×3 convolution to obtain the shallow feature f pos ;
[0096] The difference image T is obtained by subtracting the dual time image pair. D , difference graph T D Use 3×3 convolution to get shallow features f sub ;
[0097] Among them, the shallow feature f pre / pos / sub = shallow feature f pre , shallow features f pos , shallow features f sub;
[0098] In this step 1, by obtaining the difference map T D The shallow features f sub As input, it enhances the attention to the changing regions during feature extraction.
[0099] Step 2: The shallow feature f pre , shallow features f pos , shallow features f sub Input into the multi-scale encoder SGSEM for processing to obtain the intermediate layer feature F 1 pre / pos / sub , intermediate layer features F 2 pre / pos / sub , intermediate layer features F 3 pre / pos / sub , the specific steps are as follows:
[0100] 2.1、The shallow feature f pre , shallow features f pos , shallow features f sub Input it into the first SGSEB module of the multi-scale encoder SGSEM, and model it along the three dimensions of H, W, and C through three state space models to obtain the spectral feature f pre_H , spectral characteristics f pre_W , spatial features f pre_C , spectral characteristics f pos_H , spectral characteristics f pos_W , spatial features f pos_C , spectral characteristics f sub_H , spectral characteristics f sub_W , spatial features f sub_C ;
[0101] Among them, the three state space models are SSM _H 、SSM _W 、SSM _C ;
[0102] SSM _H The WC plane is scanned from the upper left to the lower right, and the state space model is built. The specific formula is:
[0103] f pre_H 、f pos_H 、f sub_H =SSM _ H(f pre / pos / sub );
[0104] SSM _W The HC plane is scanned from the upper left to the lower right, and the state space model is built. The specific formula is:
[0105] f pre_W 、f pos_W 、f sub_W =SSM _W (f pre / pos / sub );
[0106] SSM _C Scan the HW plane from the upper left to the lower right and build a state space model. The specific formula is:
[0107] f pre_C 、f pos_C 、f sub_C =SSM _C (f pre / pos / sub );
[0108] H is the shallow feature f pre / pos / sub height;
[0109] W is the shallow feature f pre / pos / sub width;
[0110] C is the shallow feature f pre / pos / sub Channel;
[0111] Among them, the multi-scale encoder SGSEM is composed of three SGSEB modules in series;
[0112] 2.2、Spectral feature f pre_H , spectral characteristics f pos_H , spectral characteristics f sub_H With the spectral characteristics f pre_W , spectral characteristics f pos_W , spectral characteristics f sub_W One-to-one correspondence and weighted fusion, and then respectively combined with the spatial feature f pre_C , spatial features f pos_C , spatial features f sub_C Multiply to get the intermediate layer feature F 1 pre , intermediate layer features F 1 pos , intermediate layer features F 1 sub , the specific formula is as follows:
[0113] F 1 pre =f pre_C ×(ω 11 *f pre_H +ω 21 *f pre_W )
[0114] F 1 pos =f pos_C×(ω 12 *f pos_H +ω 22 *f pos_W )
[0115] F 1 sub =f sub_C ×(ω 13 *f sub_H +ω 23 *f sub_W )
[0116] Among them, ω 11 f pre_H The weight value of
[0117] ω 12 f pos_H The weight value of
[0118] ω 13 f sub_H The weight value of
[0119] ω 21 f pre_W The weight value of
[0120] ω 22 f pos_W The weight value of
[0121] ω 23 f sub_W The weight value of
[0122] Among them, the intermediate layer feature F 1 pre / pos / sub =Intermediate layer feature F 1 pre , intermediate layer features F 1 pos , intermediate layer features F 1 sub ;
[0123] In step 2.2, the spatial feature f pre_C , spatial features f pos_C , spatial features f sub_C Scanning and modeling are performed on the HW plane to obtain richer spatial information;
[0124] Spectral characteristics f pre_H , spectral characteristics f pos_H , spectral characteristics f sub_H , and spectral characteristics f pre_W , spectral characteristics f pos_W , spectral characteristics f sub_WThese two groups are respectively modeling the WC and HC scans, and the process involves the channel dimension C, which is the feature with richer spectral information. Therefore, the spectral feature f is enriched by spectral information. pre_H , spectral characteristics f pos_H , spectral characteristics f sub_H , and spectral characteristics f pre_W , spectral characteristics f pos_W , spectral characteristics f sub_W , which can enhance the spatial feature f pre_C , spatial features f pos_C , spatial features f sub_C The spatial features of the enhanced intermediate layer feature F 1 pre / pos / sub ;
[0125] 2.3. The intermediate layer feature F 1 pre / pos / sub Input into the second SGSEB module to obtain the intermediate layer feature F 2 pre / pos / sub , the specific formula is:
[0126] F 2 pre / pos / sub =SGSEB(F 1 pre / pos / sub );
[0127] 2.4. The intermediate layer feature F 2 pre / pos / sub Input into the third SGSEB module to obtain the intermediate layer feature F 3 pre / pos / sub , the specific formula is:
[0128] F 3 pre / pos / sub =SGSEB(Upsampling(F 2 pre / pos / sub ))
[0129] Among them, Upsampling(·) means double upsampling;
[0130] In this step 2, the SGSEB module processes the shallow features f pre , shallow features f pos , shallow features f sub It can not only extract the spatial information of the hyperspectral image, but also enhance the spatial information through rich spectral information, so that even small change areas can be accurately detected, thereby accurately extracting the features of different scales of the hyperspectral image pair, which is convenient for subsequent fusion and detection.
[0131] Step 3: The intermediate layer feature F 1pre / pos / sub , the intermediate layer feature F 2 pre / pos / sub , the intermediate layer feature F 3 pre / pos / sub Input into the differential module DM to obtain the fine feature E i , the specific steps are as follows:
[0132] 3.1. Processing the difference graph T through unsupervised methods D , get a rough two-class change mask f i seg , the specific formula is:
[0133] f i seg =pre-model(T D )
[0134] Among them, pre-model(·) is an unsupervised method for pre-training;
[0135] i = 1, 2, 3;
[0136] 3.2. The rough two-class change mask f i seg As a hint for the SAM model, the difference graph T D The segmentation map is obtained by channel segmentation, and the segmentation map is selectively fused by band selection to obtain an accurate two-category mask map F i seg , the specific formula is:
[0137] F i seg =Bandselect(SAM(T D , f i seg ))
[0138] Among them, SAM(·) is the segmentation operation of all models.
[0139] Bandselect(·) is the process of fusing the channel-by-channel segmentation masks;
[0140] 3.3, the middle layer feature F 1 pre / pos / sub , intermediate layer features F 2 pre / pos / sub , and the intermediate layer features F 3 pre / pos / sub All are fused through 1×1 convolution to obtain a rough differential feature map e i ,
[0141] The specific formula is as follows:
[0142] ei =Conv(Cat(F i pre -F i pos , F i sub ))
[0143] Among them, Conv(·) is a 1×1 convolution operation;
[0144] Cat(·) is the concatenation operation along the channel dimension;
[0145] 3.4. Use F i seg The coarse features are analyzed by SAM-KGFEM module. i Enhance the change area features to obtain the fine feature map E i , the specific formula is as follows:
[0146] E i =Conv(Cat(e i ×sigmoid(F i seg )+e i , e i ×sigmoid(1-F i seg )+e i ))
[0147] Among them, sigmoid(·) is a normalization operation;
[0148] In this step 3, the features of images at different scales are fused to obtain a rough differential feature map e i , then through the accurate two-class mask map F i seg For the rough differential feature map e i Enhancement is performed so that the changed areas can receive more attention, which is beneficial to the subsequent multi-category change detection tasks.
[0149] Step 4: Use the multi-scale feature fusion decoder WMAM to process the fine feature map E i , get the intermediate layer feature G 1 i , the specific steps are as follows:
[0150] 4.1. The fine feature map E is transformed into i Wavelet decomposition is into frequency domain features, the specific formula is as follows:
[0151] E i_H , E i_L =DWT(E i )
[0152] Among them, the frequency domain features include high frequency features E i_H , and the low-frequency feature E i_L ;
[0153] DWT(·) is the process of wavelet decomposition;
[0154] Wavelet decomposition is a method of decomposing an image into two parts: high-frequency features and low-frequency features. High-frequency features contain texture information, while low-frequency features contain color information, category information, and semantic information.
[0155] The multi-scale feature fusion decoder WMAM is composed of multiple WMAB modules in series;
[0156] 4.2. Low-frequency feature E i_L Perform state space modeling to extract the original color, category, and semantic information in the image and obtain E i_Lssm , the specific formula is as follows:
[0157] E i_Lssm =SSM(E i_L )
[0158] Among them, SSM(·) is the process of modeling the state space model;
[0159] 4.3. High-frequency features E i_H As q, k, low-frequency features E i_L As v input to the self-attention mechanism for high-frequency features E i_H Enhance and get E i_Henhanced , the specific formula is as follows:
[0160] E i_Henhanced =Attention(E i_H , E i_L )
[0161] Among them, Attention(·) represents the self-attention mechanism;
[0162] In this step 4.3, the low frequency E i_L Information can enhance the edge information of the changing area contained in the high-frequency information.
[0163] 4.4. Processing E by inverse wavelet decomposition i_Lssm and E i_Henhanced , and obtain the spatial domain feature E 1 i , the specific formula is as follows:
[0164] E 1 i =IWT(E i_Henhanced, E i_Lssm )
[0165] Among them, IWT(·) is the process of inverse operation of wavelet decomposition;
[0166] 4.5 Spatial Domain Features E 1 i Through 1×1 convolution, the fusion feature G is obtained after fusion. 1 i , the process formula is as follows:
[0167] G 1 i =Conv(Cat(E 1 i ,Downsampling(E 1 i )))
[0168] Wherein, Downsampling(·) represents the process of downsampling by two times;
[0169] In step 4.5, the multi-scale feature maps are fused to extract comprehensive contextual information of the dual-time images T1 and T2;
[0170] In this step 4, the spatial domain feature E i In the WMAB module, it is converted into frequency domain features. In the low-frequency domain, the model can pay more attention to semantic and color features, and in the high-frequency domain, the model can pay more attention to texture features, so that the subsequent multi-class detection effect is better. At the same time, the feature learning effect under different network depths is also different. Through the fusion operation of step 4.5, each layer can learn information that is beneficial to the multi-class change detection effect, that is, it can select locally more favorable features from multiple spatial domain features, so that the multi-class change detection effect is further enhanced.
[0171] Step 5: Fusion feature G 1 i After two fully connected layers, the final multi-class change detection result graph T is obtained map .
[0172] Verification experiment
[0173] like Figure 2 As shown in FIG, it is a pair of dual-phase hyperspectral images. After being processed by the present invention, the change detection result diagram is obtained, that is, Figure 3 As shown, at this time, it can be seen visually that Figure 3 A good detection effect is obtained, which not only detects the change information of large connected areas, but also obtains good detection effect for changes in small areas, which is consistent with the actual change detection map, that is, Figure 4, and there is no significant difference. The present invention can be used to obtain reliable change detection maps.
[0174] In summary, the present invention enhances spatial information through rich spectral information, mines the spatial spectrum characteristics of hyperspectral images, improves the accuracy of change detection in small areas, and enhances the rough differential feature map to increase attention to the change area, which is beneficial to the subsequent multi-class change detection task, thereby reducing false detection, and further improving the accuracy of hyperspectral multi-class change detection. The spatial domain features and frequency domain features are combined through wavelet transform, and the texture information in the high-frequency features makes the edges of the change area more accurate. The color information, category information, and semantic information in the low-frequency features can better detect the change areas of different categories, thereby further improving the change detection accuracy. That is, when processing the hyperspectral image pair, the present invention can fully extract the spatial features, spectral features, semantic information and texture information of the hyperspectral image, so that the accuracy of multi-class change detection of the hyperspectral image is improved. The present invention does not require complex calculations and can be applied to batch detection of hyperspectral images, with fast detection efficiency and high accuracy.
[0175] The above description is merely illustrative of certain exemplary embodiments of the present invention. It goes without saying that those skilled in the art will be able to modify the described embodiments in various ways without departing from the spirit and scope of the present invention. Therefore, the above drawings and description are illustrative in nature and should not be construed as limiting the scope of protection of the claims.
Claims
1. A hyperspectral multi-class change detection method based on spatial spectrum feature enhancement and SAM guidance, characterized by: The following steps are involved: Step 1: Given a pair of dual-time images T1 and T2, perform the difference to obtain the difference image T D , use 3×3 convolution to extract the shallow features f of T1 pre , shallow features f of T2 pos 、T D The shallow features f sub ; Step 2: The shallow feature f pre , shallow features f pos , shallow features f sub Input into the multi-scale encoder SGSEM for processing to obtain the intermediate layer feature F 1 pre / pos / sub , intermediate layer features F 2 pre / pos / sub , intermediate layer features F 3 pre / pos / sub ; Step 3: The intermediate layer feature F 1 pre / pos / sub , the intermediate layer feature F 2 pre / pos / sub , the intermediate layer feature F 3 pre / pos / sub Input into the differential module DM to obtain the fine feature E i ; Step 4: Use the multi-scale feature fusion decoder WMAM to process the fine feature map E i , get the fusion feature G 1 i ; Step 5: Fusion feature G 1 i After two fully connected layers, the final multi-class change detection result graph T is obtained map .
2. The method for hyperspectral multi-class change detection based on spatial spectrum feature enhancement and SAM guidance according to claim 1 is characterized by: In step 1, given a pair of dual-time images T1 and T2, the difference is obtained by subtracting the difference image T D , use 3×3 convolution to extract the shallow features f of T1 pre , shallow features f of T2 pos 、T D The shallow features f sub The specific process is as follows: The first time image T1 uses 3×3 convolution to obtain the shallow feature f pre ; The second time image T2 uses 3×3 convolution to obtain the shallow feature f pos ; The difference image T is obtained by subtracting the dual time image pair. D , difference graph T D Use 3×3 convolution to get shallow features f sub Among them, the shallow feature f pre / pos / sub = shallow feature f Pre , shallow features f pos , shallow features f sub .
3. The method for hyperspectral multi-class change detection based on spatial spectrum feature enhancement and SAM guidance according to claim 1 is characterized by: In step 2, the shallow feature f Pre , shallow features f pos , shallow features f sub Input into the multi-scale encoder SGSEM for processing to obtain the intermediate layer feature F 1 pre / pos / sub , intermediate layer features F 2 pre / pos / sub , intermediate layer features F 3 pre / pos / sub The specific steps are as follows: 2.1、The shallow feature f pre , shallow features f pos , shallow features f sub Input it into the first SGSEB module of the multi-scale encoder SGSEM, and model it along the three dimensions of H, W, and C through three state space models to obtain the spectral feature f pre_H , spectral characteristics f pre_W , spatial features f pre_C , spectral characteristics f pos_H , spectral characteristics f pos_W , spatial features f pos_C , spectral characteristics f sub_H , spectral characteristics f sub_W , spatial features f sub_C ; 2.2、Spectral feature f pre_H , spectral characteristics f pos_H , spectral characteristics f sub_H With the spectral characteristics f pre_W , spectral characteristics f pos_W , spectral characteristics f sub_W One-to-one correspondence and weighted fusion, and then respectively combined with the spatial feature f pre_C , spatial features f pos_C , spatial features f sub_C Multiply to get the intermediate layer feature F 1 pre , intermediate layer features F 1 pos , intermediate layer features F 1 sub , the specific formula is as follows: F 1 pre =f pre_C ×(ω 11 *f pre_H +oh 21 *f pre_W ) F 1 pos =f pos_C ×(ω 12 *f pos_H +oh 22 *f pos_W ) F 1 sub =f sub_C ×(ω 13 *f sub_H +oh 23 *f sub_W ) Among them, ω 11 f pre_H The weight value of ω 12 f pos_H The weight value of ω 13 f sub_H The weight value of ω 21 f pre_W The weight value of ω 22 f pos_W The weight value of ω 23 f sub_W The weight value of Intermediate layer feature F 1 pre / pos / sub =Intermediate layer feature F 1 pre , intermediate layer features F 1 pos , intermediate layer features F 1 sub ; 2.
3. The intermediate layer feature F 1 pre / pos / sub Input into the second SGSEB module to obtain the intermediate layer feature F 2 pre / pos / sub , the specific formula is: F 2 pre / pos / sub =SGSEB(F 1 pre / pos / sub ); 2.
4. The intermediate layer feature F 2 pre / pos / sub Input into the third SGSEB module to obtain the intermediate layer feature F 3 pre / pos / sub , the specific formula is: F 3 pre / pos / sub =SGSEB(Upsampling(F 2 pre / pos / sub )) Here, Upsampling(·) indicates double upsampling.
4. The method for hyperspectral multi-class change detection based on spatial spectrum feature enhancement and SAM guidance according to claim 1 is characterized by: In step 2.1, the three state space models are SSM _H 、SSM _W 、SSM _C , specifically: SSM H The WC plane is scanned from the upper left to the lower right, and the state space model is built. The specific formula is: f pre_H 、f pos_H 、f sub_H =SSM _H (f pre / pos / sub ); SSM _W The HC plane is scanned from the upper left to the lower right, and the state space model is built. The specific formula is: f pre_W 、f pos_W 、f sub_W =SSM _W (f pre / pos / sub ); SSM _C Scan the HW plane from the upper left to the lower right and build a state space model. The specific formula is: f pre_C 、f pos_C 、f sub_C =SSM _C (f pre / pos / sub ); H is the shallow feature f pre / pos / sub height; W is the shallow feature f pre / pos / sub Width; C is the shallow feature f pre / pos / sub Channel; The multi-scale encoder SGSEM is composed of three SGSEB modules connected in series.
5. The method for hyperspectral multi-class change detection based on spatial spectrum feature enhancement and SAM guidance according to claim 1 is characterized by: In step 3, the intermediate layer features F 1 pre / pos / sub , the intermediate layer feature F 2 pre / pos / sub , the intermediate layer feature F 3 pre / pos / sub Input into the differential module DM to obtain the fine feature E i The specific steps are as follows: 3.
1. Processing the difference graph T through unsupervised methods D , get a rough two-class change mask f i seg , the specific formula is: f i seg =pre-model(T D ) Among them, pre-model(·) is an unsupervised method for pre-training; 3.
2. The rough two-class change mask f i seg As a hint for the SAM model, the difference graph T D The segmentation map is obtained by channel segmentation, and the segmentation map is selectively fused by band selection to obtain an accurate two-category mask map F i seg , the specific formula is: F i seg =Bandselect(SAM(T D ,f i seg )) Among them, SAM(·) is the segmentation operation of all models. Bandselect(·) is the process of fusing the channel-by-channel segmentation masks; 3.3、The middle layer feature F 1 pre / pos / sub , intermediate layer features F 2 pre / pos / sub , and the intermediate layer features F 3 pre / pos / sub All are fused through 1×1 convolution to obtain a rough differential feature map e i , The specific formula is as follows: e i =Conv(Cat(F i pre -F i pos ,F i sub )) Among them, Conv(·) is a 1×1 convolution operation; Cat(·) is the concatenation operation along the channel dimension; 3.
4. Use F i seg The coarse features are analyzed by SAM-KGFEM module. i Enhance the change area features to obtain the fine feature map E i , the specific formula is as follows: AND i =Conv(Cat(e i ×sigmoid(F i seg )+e i ,and i ×sigmoid(1-F i seg )+e i )) Among them, sigmoid(·) is a normalization operation.
6. The method for hyperspectral multi-class change detection based on spatial spectrum feature enhancement and SAM guidance according to claim 1, characterized in that: In step 4, the multi-scale feature fusion decoder WMAM is used to process the fine feature map E i , get the fusion feature G 1 i The specific steps are as follows: 4.
1. The fine feature map E is transformed into i Wavelet decomposition is into frequency domain features, the specific formula is as follows: E i_H ,E i_L =DWT(E i ) Among them, the frequency domain features include high frequency features E i_H , and the low-frequency feature E i_L ; DWT(·) is the process of wavelet decomposition; And high-frequency features contain texture information; Low-frequency features contain color information, category information, and semantic information; The multi-scale feature fusion decoder WMAM is composed of multiple WMAB modules in series; 4.
2. Low-frequency feature E i_L Perform state space modeling to extract the original color, category, and semantic information in the image and obtain E i_Lssm , the specific formula is as follows: AND i_Lssm =SSM(E i_L ) Among them, SSM(·) is the process of modeling the state space model; 4.
3. High-frequency features E i_H As q, k, low-frequency features E i_L As v input to the self-attention mechanism for high-frequency features E i_H Enhance and get E i_Henhanced , the specific formula is as follows: AND i_Henhanced =Attention(E i_H ,AND i_L ) Among them, Attention(·) represents the self-attention mechanism; 4.
4. Processing E by inverse wavelet decomposition i_Lssm and E i_Henhanced , and obtain the spatial domain feature E 1 i , the specific formula is as follows: E 1 i =IWT(E i_Henhanced ,E i_Lssm ) Among them, IWT(·) is the process of inverse operation of wavelet decomposition; 4.5 Spatial Domain Features E 1 i Through 1×1 convolution, the fusion feature G is obtained after fusion. 1 i , the process formula is as follows: G 1 i =Conv(Cat(E 1 i ,Downsampling(E 1 i ))) Here, Downsampling(·) represents the process of downsampling by a factor of two.