Hyperspectral image restoration method

Abstract

The invention discloses a hyperspectral image restoration method, which belongs to the technical field of hyperspectral imaging, and comprises the following steps: inputting a single hyperspectral image to be restored into a hyperspectral image restoration network for denoising / super-resolution reconstruction, the features are respectively input into a multi-scale space-spectrum fusion module to obtain branch features, local multi-scale space-spectrum features after dimension raising are obtained through splicing and dimension raising, and then the local multi-scale space-spectrum features are input into a self-adaptive space feature aggregation group to obtain global space-spectrum features; and after the noisy hyperspectral image passes through a convolutional layer, fusing the noisy hyperspectral image with the global space-spectral features to obtain a final de-noised image. The extended denoising method can obtain a super-resolution reconstruction method, and the method comprises the steps: carrying out the up-sampling after the global space-spectrum features are obtained; a low-resolution hyperspectral image to be restored is up-sampled before passing through a convolutional layer. According to the invention, the problems of noise introduction and spatial resolution reduction in the hyperspectral imaging process in the prior art are solved.

Description

Technical Field

[0001] The present invention relates to a hyperspectral image restoration method, belonging to the technical field of hyperspectral imaging. Background Art

[0002] In recent years, with the continuous in-depth research of remote sensing technology, the application of hyperspectral imaging has also developed vigorously. Hyperspectral imaging obtains rich spectral information by capturing hundreds of continuous bands from the visible spectrum, near-infrared to far-infrared. Benefiting from the relatively high spectral resolution, hyperspectral images produce more discriminative feature representations than traditional RGB images. Hyperspectral image restoration covers two aspects: super-resolution reconstruction (HSI-SR) and denoising (HSID), aiming to improve the quality of hyperspectral images (HSI). Due to physical characteristics and cost limitations, hyperspectral imaging devices cannot simultaneously obtain images with high spectral resolution and high spatial resolution, and affected by physical limitations and environmental factors, images are often accompanied by atmospheric scattering and sensor noise, etc., which affect the accuracy of subsequent analysis. Therefore, researching hyperspectral image restoration technology can make up for equipment limitations, improve the signal-to-noise ratio of images, reduce the difficulty of data processing, and provide a high-quality data basis for tasks such as ground object classification and abnormal target detection.

[0003] Convolutional neural networks have been widely used in hyperspectral image restoration due to their automatic feature learning ability and the ability to process high-dimensional spatial-spectral information. However, problems such as dependence on a large amount of labeled data, emphasis on spatial features while ignoring spectral information, and the need for high-resolution auxiliary images still pose challenges to its processing tasks. Therefore, the focus of the research field lies in how to effectively extract spatial-spectral information without auxiliary images, utilize global and local spatial-spectral correlations, and improve the quality of hyperspectral image restoration.

[0004] In summary, the existing technologies have obvious deficiencies in hyperspectral image restoration. During the hyperspectral imaging process, noise is often introduced and the spatial resolution decreases. In addition, the problems of insufficient and unbalanced samples commonly existing in hyperspectral images make it difficult for the existing technologies to stably complete the task of restoring the quality of degraded hyperspectral images. These problems jointly limit the accuracy and effect of hyperspectral image restoration. Summary of the Invention

[0005] The purpose of the present invention is to provide a hyperspectral image restoration method, which solves the problems of noise introduced and spatial resolution decrease during the hyperspectral imaging process by improving the accuracy of hyperspectral image restoration, and at the same time, stably completes the task of image quality restoration for degraded hyperspectral images in the case of insufficient and unbalanced samples commonly existing in hyperspectral images.

[0006] To solve the above technical problems, the present invention is implemented by adopting the following technical solutions:

[0007] The present invention provides a hyperspectral image restoration method, including:

[0008] Obtain a single hyperspectral image HSI to be restored;

[0009] Input the single hyperspectral image HSI to be restored into the hyperspectral image restoration network SSFSA for denoising / super-resolution reconstruction to obtain a restored hyperspectral image;

[0010] The restored hyperspectral image includes the denoised hyperspectral image / the hyperspectral image after super-resolution reconstruction;

[0011] Among them, the method for the hyperspectral image restoration network SSFSA to perform denoising includes:

[0012] Using the spectral overlap grouping and parameter sharing strategy, set the number of overlapping spectral channels, overlap and group the single hyperspectral image HSI along the spectral dimension, input each grouped hyperspectral image HSI into the corresponding multi-scale spatial-spectral fusion module MSSF respectively, extract the mapping relationship in the multi-scale space to obtain all branch features, concatenate all branch features, and perform dimension elevation through a 1×1 convolutional layer to obtain the elevated local multi-scale space-spectral features; among them, all groups share the network parameters of each multi-scale spatial-spectral fusion module MSSF;

[0013] Input the elevated local multi-scale space-spectral features into the adaptive spatial feature aggregation group ASAG to obtain global space-spectral features;

[0014] After passing the single hyperspectral image through a 1×1 convolutional layer, fuse it with the global space-spectral features, and then pass through a 1×1 convolutional layer again to obtain the denoised hyperspectral image;

[0015] Among them, the method for the extended hyperspectral image restoration network SSFSA to perform denoising to obtain the method for the hyperspectral image restoration network SSFSA to perform super-resolution reconstruction includes:

[0016] After obtaining the global space-spectral features, perform an upsampling operation;

[0017] Before passing the single hyperspectral image through a 1×1 convolutional layer, perform a bicubic upsampling operation.

[0018] Furthermore, grouping the single hyperspectral image along the spectral dimension, inputting it into the multi-scale spatial-spectral fusion module MSSF, extracting the mapping relationship in the multi-scale space to obtain all branch features, concatenating all branch features, and performing dimension elevation through a 1×1 convolutional layer to obtain the elevated local multi-scale space-spectral features, including:

[0019] After grouping a single hyperspectral image along the spectral dimension, it is respectively input into two half-channel branches and one full-channel branch of the multi-scale spatial-spectral fusion module MSSF to obtain two half-channel branch features and one full-channel branch feature;

[0020] After concatenating the two half-channel branch features, passing them through the ReLU activation function, and then performing pixel-wise addition with the full-channel branch feature, the fused multi-scale feature is obtained;

[0021] After passing the fused multi-scale feature through the spatial residual module, the multi-scale spatial feature is obtained;

[0022] The multi-scale spatial feature is fed into the channel attention module CAB with residual connection to obtain the multi-scale spatial-spectral feature;

[0023] After performing pixel-wise addition of the multi-scale spatial-spectral feature and the multi-scale spatial feature, the branch feature is obtained;

[0024] All the branch features are concatenated and upsampled through a 1×1 convolutional layer to obtain the upsampled local multi-scale spatial-spectral feature.

[0025] Furthermore, the local multi-scale spatial-spectral feature is expressed as:

[0026] ;

[0027] In the formula, represents the upsampled local multi-scale spatial-spectral feature, represents the number of upsampled spectral channels, represents the height in the image spatial resolution, represents the width in the image spatial resolution, represents the 1×1 convolutional operation, represents the concatenation operation, represents the local multi-scale spatial-spectral feature obtained after concatenating all the branch features, represents the first branch feature, represents the second branch feature, represents the th branch feature, represents the th branch feature, represents the number of spectral channels of the single hyperspectral image HSI to be restored.

[0028] Furthermore, after grouping the single hyperspectral image along the spectral dimension and respectively inputting it into two half-channel branches and one full-channel branch of the multi-scale spatial-spectral fusion module MSSF to obtain two half-channel branch features and one full-channel branch feature, it includes:

[0029] After grouping a single hyperspectral image along the spectral dimension, the spectral channel numbers of each group are divided into two and respectively input into two half-channel branches of the multi-scale spatial-spectral fusion module MSSF. After two 3×3 convolution operations, each 3×3 convolution operation is followed by a ReLU activation function, then another 3×3 convolution operation, and finally a dilated convolution operation to obtain the features of the two half-channel branches;

[0030] After grouping a single hyperspectral image along the spectral dimension, all spectral channels of each group are input into the full-channel branch of the multi-scale spatial-spectral fusion module MSSF. After three cascaded 3×3 convolutions plus ReLU activation functions, and then a dilated convolution operation, the features of the full-channel branch are obtained.

[0031] Furthermore, the multi-scale spatial feature is expressed as:

[0032] ;

[0033] wherein, represents the multi-scale spatial feature, represents the multi-scale feature after fusing the output features of the half-channel branch and the full-channel branch, represents the th group after grouping a single hyperspectral image along the spectral dimension, represents the 3×3 convolution operation.

[0034] Furthermore, feeding the multi-scale spatial feature into the channel attention module CAB with residual connection to obtain the multi-scale spatial-spectral feature includes:

[0035] After passing the multi-scale spatial feature through two 3×3 convolution operations, the first channel attention feature is obtained, where a GELU activation function is used after the first 3×3 convolution operation;

[0036] Passing the first channel attention feature through a global average pooling layer and then through two 3×3 convolution operations to obtain the second channel attention feature, where a ReLU activation function is used after the first 3×3 convolution operation;

[0037] After passing the second channel attention feature through the Sigmoid function, it is multiplied pixel by pixel with the first channel attention feature to obtain the multi-scale spatial-spectral feature.

[0038] Furthermore, the adaptive spatial feature aggregation group ASAG includes cascaded adaptive spatial aggregation modules SAB, and the cascaded adaptive spatial aggregation modules form a residual through long and short skip connections, where, .

[0039] Furthermore, the upsampled local multi-scale spatio-spectral features are input into the Adaptive Spatial Feature Aggregation Group (ASAG) to obtain global spatio-spectral features. Each adaptive spatial aggregation module (SAB) cascaded in the ASAG includes:

[0040] For each SAB, after layer normalization, it passes through a parallel three-branch structure including a channel attention module (CAB), a 3×3 depthwise convolution (DWConv), and a learnable convolutional kernel attention (LCK) to obtain the output features of the three branches;

[0041] The output features of the three branches are used for spatio-spectral feature aggregation by pixel-wise multiplication, then passed through a 1×1 convolutional layer, and then through a residual connection to obtain the global deep features output by the SAB;

[0042] Among them, the learnable convolutional kernel attention (LCK) is used to adaptively process the spatial features of the local multi-scale spatio-spectral features after layer normalization; the 3×3 depthwise convolution (DWConv) is used to capture the spatial invariant features of the local multi-scale spatio-spectral features after layer normalization; the channel attention module (CAB) is used to further extract and refine the spectral features of the local multi-scale spatio-spectral features after layer normalization.

[0043] Furthermore, the upsampled local multi-scale spatio-spectral features pass through N cascaded SABs to obtain global deep features, and the global deep features are expressed as:

[0044] ;

[0045] In the formula, represents the th global deep feature, represents the th SAB module, represents the th global deep feature, represents the th SAB module, represents the first SAB module, represents the upsampled local multi-scale spatio-spectral features;

[0046] The global spatio-spectral features are expressed as:

[0047] ;

[0048] In the formula, represents the global spatio-spectral features, " " represents per-pixel addition.

[0049] Furthermore, the output features of the three branches include LCK output features, and the method for obtaining the LCK output features includes:

[0050] Learning a set of convolutional kernels shared at all spatial positions and all images using the locally multi-scale spatio-spectral features after layer normalization;

[0051] Using a lightweight convolutional layer branch to predict each spatial position of the locally multi-scale spatio-spectral features after prediction layer normalization to obtain a fusion coefficient map, where the lightweight convolutional branch includes two 3×3 grouped convolutional layers and a SimpleGate activation function;

[0052] Linearly fusing the fusion coefficient feature map and the convolutional kernels, and calculating the fusion weights of each spatial position of the locally multi-scale spatio-spectral features after layer normalization through the linear combination of the convolutional kernels;

[0053] Transforming the locally multi-scale spatio-spectral features after layer normalization through a 1×1 convolution to obtain an enhanced feature map;

[0054] Performing a grouped convolution operation on the enhanced feature map and the fusion weights of each spatial position of the locally multi-scale spatio-spectral features after layer normalization, and calculating through the grouped convolution to obtain the LCK output features at each spatial position.

[0055] Compared with the prior art, the beneficial effects achieved by the present invention:

[0056] 1. By adopting a spectral grouping and parameter sharing strategy, the present invention makes full use of the strong correlation between adjacent bands, effectively improving the computational efficiency and the accuracy of feature extraction in the restoration process. At the same time, combined with the multi-scale spatio-spectral fusion module MSSF, it captures the spatial and spectral information of the image at different scale features, realizing a more detailed and refined image restoration effect. In addition, the adaptive spatial aggregation group ASAG is also applied, which adaptively optimizes the aggregation process of spatial information through learnable convolutional kernels and automatically adjusts the aggregation strategy according to the image content, thereby significantly restoring the image details and structure. In summary, the hyperspectral image restoration method provided by the present invention can significantly improve the restoration accuracy and effectively cope with the noise and spatial resolution degradation problems introduced in the hyperspectral imaging process. Especially in the case of insufficient and unbalanced samples, it can also achieve stable image quality restoration.

[0057] 2. The present invention designs a multi-scale spatial-spectral fusion module MSSF. Firstly, by using a parallel-branch group convolution structure including a semi-channel branch and a full-channel branch, it effectively fuses spatial and spectral information at different scales, can parallelly enhance the internal and external relationships between different spectral bands, and is conducive to capturing smooth image features related to low-frequency information. Among them, the semi-channel branch reduces the computational amount while maintaining the richness of features, and the full-channel branch further deeply extracts global features. Secondly, each branch uses a layer of dilated convolution respectively, which can increase the receptive field without increasing network parameters and extract multi-scale spatial information from different receptive fields. Finally, a channel attention module with a residual connection is added to exploit the spectral similarity and complementarity to mine the correlation of spectral features. This not only improves the computational efficiency but also significantly enhances the fineness and accuracy of image restoration, especially when dealing with complex hyperspectral images.

[0058] 3. The adaptive spatial feature aggregation group ASAG provided by the present invention realizes the dynamic adjustment and adaptive aggregation of local multi-scale spatial-spectral features through cascaded adaptive spatial aggregation modules SAB, combined with long and short skip connections to form residuals. In particular, the three-branch structure within the adaptive spatial aggregation module SAB can finely extract and fuse spatial and spectral features, further enhancing the detail restoration ability and structure preservation of image restoration. In addition, the learnable convolutional kernel attention LCK realizes the adaptive selection and fusion of learnable convolutional kernels at each position by predicting the fusion coefficient map, thereby enhancing the expression ability of global spatial-spectral features. BRIEF DESCRIPTION OF THE DRAWINGS

[0059] Figure 1 FIG. is a schematic structural diagram of a hyperspectral image restoration network SSFSA provided by an embodiment of the present invention;

[0060] Figure 2 FIG. is a schematic structural diagram of a multi-scale spatial-spectral fusion module MSSF provided by an embodiment of the present invention;

[0061] Figure 3 FIG. is a schematic structural diagram of an adaptive spatial aggregation module SAB provided by an embodiment of the present invention. DETAILED DESCRIPTION OF THE INVENTION

[0062] The technical solutions of the present invention will be described in detail below with reference to the drawings and specific embodiments. It should be understood that the specific features in the embodiments of the present invention and the embodiments are detailed descriptions of the technical solutions of the present invention, rather than limitations on the technical solutions of the present invention. Without conflict, the technical features in the embodiments of the present invention and the embodiments can be combined with each other.

[0063] The term "and / or" is merely a description of the association relationship between associated objects, indicating that three relationships can exist. For example, A and / or B can represent three situations: A exists alone, A and B exist simultaneously, and B exists alone. Additionally, the character " / " generally indicates that the associated objects before and after are in an "or" relationship.

[0064] Embodiment 1

[0065] As Figure 1 shown, this embodiment introduces a hyperspectral image restoration method, including:

[0066] Obtain a single hyperspectral image HSI to be restored;

[0067] Hyperspectral images contain rich spectral information. The hyperspectral images to be restored have poor quality due to problems such as noise and low resolution. The purpose of this embodiment is to solve the problems of noise and decreased spatial resolution introduced in the prior art during hyperspectral imaging by improving the accuracy of hyperspectral image restoration. At the same time, in the case of insufficient and unbalanced samples commonly existing in hyperspectral images, stably complete the task of image quality restoration for degraded hyperspectral images.

[0068] Input the single hyperspectral image to be restored into the hyperspectral image restoration network SSFSA for denoising / super-resolution reconstruction to obtain a restored hyperspectral image.

[0069] The hyperspectral image restoration network SSFSA framework provided in this embodiment combines spectral and spatial information and restores a single hyperspectral image through a specific joint coding method, which can more effectively utilize the information redundancy and differences in hyperspectral images.

[0070] Among them, the method for the hyperspectral image restoration network SSFSA to perform denoising includes:

[0071] Step 1: Use the spectral overlapping grouping and parameter sharing strategy, set the number of overlapping spectral channels, overlap and group the single hyperspectral image HSI along the spectral dimension, input each grouped HSI into the multi-scale spatial-spectral fusion module MSSF respectively, extract the mapping relationship in the multi-scale space to obtain all branch features, concatenate all branch features, and perform dimension elevation through a 1×1 convolutional layer to obtain the dimension-elevated local multi-scale space-spectral features.

[0072] In the present invention, all groups share the network parameters of each multi-scale spatial-spectral fusion module MSSF, avoiding high computational costs and complex optimization processes. Among them, the structure of the multi-scale spatial-spectral fusion module MSSF is as Figure 2 shown.

[0073] By adopting the spectral grouping and parameter sharing strategies, the present invention aims to reduce the difficulty of feature extraction through spectral grouping, while greatly reducing the parameters of the model by using the parameter sharing strategy. Setting the number of overlapping spectra can ensure the information continuity between adjacent spectral bands and avoid information loss. Therefore, through spectral grouping and parameter sharing, the present invention reduces the computational cost while maintaining performance, and improves the training efficiency of the hyperspectral image restoration network SSFSA.

[0074] The present invention groups a single hyperspectral image along the spectral dimension and inputs it into the multi-scale spatial-spectral fusion module MSSF. By grouping along the spectral dimension, it helps the hyperspectral image restoration network SSFSA to independently process different spectral bands. The multi-scale spatial-spectral fusion module MSSF can also extract the mapping relationships in the multi-scale space, capture the spatial features of different scales, and provide strong support for subsequent feature fusion and denoising.

[0075] The present invention extracts the mapping relationships in the multi-scale space to obtain all branch features, cascades all branch features, and raises the dimension through a 1×1 convolutional layer, aiming to fuse the multi-scale spatial features extracted by the multi-scale spatial-spectral fusion module MSSF. Secondly, the cascading operation can splice the features of different branches together to form a more comprehensive feature representation, and the 1×1 convolutional layer is used to raise the dimension of the fused features. Through the cascading and dimension raising processing of the 1×1 convolutional layer, the hyperspectral image restoration network SSFSA can generate local multi-scale space-spectral features, which contain both spatial information and spectral information, providing key information for subsequent denoising operations.

[0076] Step 2: Input the dimension-raised local multi-scale space-spectral features into the adaptive spatial feature aggregation group ASAG to obtain global space-spectral features.

[0077] The present invention inputs the dimension-raised local multi-scale space-spectral features into the adaptive spatial feature aggregation group ASAG to obtain global space-spectral features, which can adaptively aggregate the local multi-scale space-spectral features to generate global space-spectral features. The global space-spectral features can reflect the overall structure and spectral characteristics of the hyperspectral image. Through the processing of the adaptive spatial feature aggregation group ASAG, the hyperspectral image restoration network SSFSA can capture the global information of the hyperspectral image and provide a more comprehensive feature representation for subsequent denoising and reconstruction operations.

[0078] Step 3: After passing the single hyperspectral image through a 1×1 convolutional layer, fuse it with the global space-spectral features, and then pass through a 1×1 convolutional layer again to obtain the denoised hyperspectral image.

[0079] After passing the single hyperspectral image through a 1×1 convolutional layer, the present invention fuses it with the global spatio-spectral features, and then passes it through a 1×1 convolutional layer again to obtain the denoised hyperspectral image. The aim is to fuse the original hyperspectral image with the global spatio-spectral features, and through the processing of the 1×1 convolutional layer for final integration and processing to generate the final denoising result. By fusing the original image and the global features and combining the processing of the 1×1 convolutional layer, the hyperspectral image restoration network SSFSA of the present invention can generate a denoised hyperspectral image. While maintaining the original spectral information, these hyperspectral images effectively reduce noise interference.

[0080] Among them, for the method of denoising by the extended hyperspectral image restoration network SSFSA, and for the method of super-resolution reconstruction by the hyperspectral image restoration network SSFSA, it includes:

[0081] After obtaining the global spatio-spectral features, an upsampling operation is performed.

[0082] The present invention magnifies the low-resolution hyperspectral image to high resolution through an upsampling operation. This is one of the key steps in the super-resolution reconstruction task. Through the upsampling operation, the hyperspectral image restoration network SSFSA can generate a hyperspectral image with a higher spatial resolution, thus meeting the requirements for high-resolution images in practical applications.

[0083] Before passing the single hyperspectral image through a 1×1 convolutional layer, a bicubic upsampling operation is performed.

[0084] Bicubic upsampling estimates the value of the target pixel using the information of 16 surrounding pixel points, and can generate relatively smooth image edges and details. In the super-resolution reconstruction task of the present invention, the bicubic upsampling operation serves as a preprocessing step to provide a higher-resolution input image for subsequent network processing.

[0085] Embodiment 2

[0086] Based on the same inventive concept as Embodiment 1, this embodiment introduces the specific implementation steps of a hyperspectral image restoration method, including:

[0087] Step 1: Obtain a single hyperspectral image HSI to be restored.

[0088] Step 2: Input the single hyperspectral image to be restored into the hyperspectral image restoration network SSFSA for denoising / super-resolution reconstruction to obtain a restored hyperspectral image;

[0089] In this embodiment, the restored hyperspectral image includes a denoised hyperspectral image / a hyperspectral image after super-resolution reconstruction.

[0090] In some embodiments, the training parameters of the hyperspectral image restoration network SSFSA are set as follows:

[0091] The number of spectra in each group is set to 8, the number of overlapping spectra between adjacent groups is set to 2, and the dilation rates of the dilated convolutions in the multi-scale spatial-spectral fusion module MSSF are set to 1, 3, and 5 respectively. Among them, the hyperspectral image restoration network SSFSA is implemented based on the Pytorch framework, and the Adam optimizer is used to optimize the network parameters. The initial learning rate is set to 5×10 -5 , and the learning rate decays to 1 / 10 of the original value after 30 iterations. The batch size is set to 16, and the number of training iterations is set to 100.

[0092] Among them, the method for the hyperspectral image restoration network SSFSA to perform denoising includes:

[0093] Step 2.1: Set the number of overlapping spectra based on the spectral grouping and parameter sharing strategy.

[0094] In this embodiment, the number of overlapping spectra of the spectral grouping and parameter sharing strategy is set to , that is, it is ensured that there are identical spectra between two adjacent groups after grouping.

[0095] Step 2.2: Group a single hyperspectral image along the spectral dimension, input it into the multi-scale spatial-spectral fusion module MSSF, extract the mapping relationships in the multi-scale space to obtain all branch features, concatenate all branch features, and perform dimension elevation through a 1×1 convolutional layer to obtain the dimension-elevated local multi-scale space-spectral features.

[0096] In this embodiment, using the overlapping grouping method, according to the number of overlapping spectra , the input hyperspectral image to be restored is divided into G groups along the spectral dimension, denoted as , where the th group is denoted as . The hyperspectral image to be restored has spectral channels and spatial resolution, represents the height in the image spatial resolution, represents the width in the image spatial resolution. The th hyperspectral image group after grouping has spectral channels, where represents rounding up. Each group of hyperspectral images is respectively input into the branch network MSSF for encoding.

[0097] In this embodiment, by using a spectral grouping strategy, each hyperspectral image group is added to the branch network as input to effectively utilize the correlation of adjacent spectra. Through the parameter sharing strategy of the branch network, the parameters of the hyperspectral image restoration network SSFSA and the difficulty of feature extraction are reduced.

[0098] In this embodiment, a single hyperspectral image is grouped along the spectral dimension and input into the multi-scale spatial-spectral fusion module MSSF. By utilizing the information redundancy and spectral difference between adjacent spectra, the mapping relationship in the multi-scale space is extracted to obtain all branch features. All branch features are concatenated and dimensionally increased through a 1×1 convolutional layer to obtain the dimensionally increased local multi-scale space-spectral features, including:

[0099] Step 2.2.1: After grouping a single hyperspectral image along the spectral dimension, it is respectively input into two half-channel branches and one full-channel branch of the multi-scale spatial-spectral fusion module MSSF to obtain two half-channel branch features and one full-channel branch feature.

[0100] In this embodiment, the output features of the first branch and the second branch of the two half-channel branches HCB are respectively expressed as:

[0101] ;

[0102] In the formula, respectively represent the output features of the first branch and the second branch of the half-channel branch HCB, represents a 3×3 convolution operation, represents a 3×3 convolution plus ReLU operation, represents a dilated convolution plus ReLU operation with a dilation coefficient of where the input features of the first branch and the second branch of the two half-channel branches HCB are respectively .

[0103] In this embodiment, the output feature of the full-channel branch is expressed as:

[0104] ;

[0105] In the formula, represents the output feature of the full-channel branch, represents the th group input feature after grouping a single hyperspectral image along the spectral dimension, where the spectral channel number of the th group input feature is , while the spectral channel numbers of the half-channel input features are both , and the two half-channel input features together are the full-channel input feature .

[0106] Step 2.2.2: After cascading the two half-channel branch features, passing them through the ReLU activation function, and then performing pixel-wise addition with the full-channel branch features, the fused multi-scale features are obtained;

[0107] In this embodiment, the output feature of the half-channel branch is expressed as:

[0108] ;

[0109] In the formula, represents the output feature of the half-channel branch, represents the ReLU activation function, represents the output feature of the first branch for splicing the two half-channel branches HCB and the output feature of the second branch .

[0110] The fused multi-scale features are expressed as:

[0111]

[0112] In the formula, represents the multi-scale feature after fusing the output feature of the half-channel branch and the output feature of the full-channel branch.

[0113] Step 2.2.3: After passing the fused multi-scale features through the spatial residual module, multi-scale spatial features are obtained;

[0114] In this embodiment, the spatial residual module consists of two 3×3 convolution functions:

[0115] ;

[0116] In the formula, represents the multi-scale spatial feature after passing through the spatial residual module.

[0117] Step 2.2.4: Feed the multi-scale spatial features into the channel attention module CAB with residual connection to obtain multi-scale spatial-spectral features, and after performing pixel-wise addition of the multi-scale spatial-spectral features and the multi-scale spatial features, branch features are obtained;

[0118] In this embodiment, the output feature of the th branch is expressed as:

[0119] ;

[0120] In the formula, represents the output feature of the th branch, Represents the spectral attention module CAB.

[0121] Step 2.2.5: Concatenate all branch features and increase the dimension through a 1×1 convolutional layer to obtain the upsampled local multi-scale spatial-spectral features.

[0122] In this embodiment, the local multi-scale spatial-spectral features are represented as:

[0123] ;

[0124] where, represents the upsampled local multi-scale spatial-spectral features, represents the number of upsampled spectral channels, which is set to 240 in this embodiment, represents the 1×1 convolution operation, represents the concatenation operation, represents the local multi-scale spatial-spectral features obtained after concatenating all branch features, represents the first branch feature, represents the second branch feature, represents the th branch feature, represents the th branch feature. When the output features of G branches of MSSF are concatenated, due to the overlap between adjacent groups, the feature values in the overlapping bands need to be averaged according to their original spectral band positions to generate a comprehensive feature map with the number of spectral channels being .

[0125] In this embodiment, after the single hyperspectral image is grouped along the spectral dimension, it is respectively input into two half-channel branches and one full-channel branch of the multi-scale spatial-spectral fusion module MSSF to obtain two half-channel branch features and one full-channel branch feature, including:

[0126] After the single hyperspectral image is grouped along the spectral dimension, it is input into two half-channel branches of the multi-scale spatial-spectral fusion module MSSF, and after two 3×3 convolutional operations, each 3×3 convolutional operation is followed by a ReLU activation function, and then after another 3×3 convolutional operation, and finally through a layer of dilated convolution, two half-channel branch features are obtained;

[0127] After the single hyperspectral image is grouped along the spectral dimension, it is input into the full-channel branch of the multi-scale spatial-spectral fusion module MSSF, and after three cascaded 3×3 convolutions plus ReLU activation function processing, and then through a layer of dilated convolution, a full-channel branch feature is obtained.

[0128] In this embodiment, feeding the multi-scale spatial features into the channel attention module CAB of the residual connection to obtain the multi-scale spatial-spectral features includes:

[0129] After the multi-scale spatial features go through two 3×3 convolution operations, the first channel attention feature is obtained, where after the first 3×3 convolution operation, it is processed by the GELU activation function;

[0130] Pass the first channel attention feature through the global average pooling layer, and after two 3×3 convolution operations, the second channel attention feature is obtained, where after the first 3×3 convolution operation, it is processed by the ReLU activation function;

[0131] After passing the second channel attention feature through the Sigmoid function, perform pixel-wise multiplication with the first channel attention feature to obtain the multi-scale spatial-spectral features.

[0132] Step 2.3: Input the upsampled local multi-scale spatial-spectral features into the adaptive spatial feature aggregation group ASAG to obtain the global spatial-spectral features.

[0133] In some embodiments, the adaptive spatial feature aggregation group ASAG includes cascaded adaptive spatial aggregation modules SAB, and the cascaded adaptive spatial aggregation modules form a residual through long and short skip connections, where . Among them, the structure of the adaptive spatial aggregation module SAB is as Figure 3 shown.

[0134] In some embodiments, input the upsampled local multi-scale spatial-spectral features into the adaptive spatial feature aggregation group ASAG to obtain the global spatial-spectral features, where each cascaded adaptive spatial aggregation module SAB in the adaptive spatial feature aggregation group ASAG includes:

[0135] For each adaptive spatial aggregation module SAB, after layer normalization, it goes through a parallel three-branch structure including a channel attention module CAB, a 3×3 depthwise convolution DWConv, and a learnable convolution kernel attention LCK to obtain the output features of the three branches;

[0136] Use pixel-wise multiplication to aggregate the spatial features and spectral features of the output features of the three branches, then pass through a 1×1 convolution layer, and then through a residual connection to obtain the global deep features output by the adaptive spatial aggregation module SAB.

[0137] Among them, the learnable convolutional kernel attention LCK is used to adaptively process the spatial features of the local multi-scale spatial-spectral features after layer normalization; the 3×3 depthwise convolution DWConv is used to capture the spatial invariant features of the local multi-scale spatial-spectral features after layer normalization; the channel attention module CAB is used to further extract and refine the spectral features of the local multi-scale spatial-spectral features after layer normalization.

[0138] In this embodiment, the layer normalization LN operation is expressed as:

[0139] ;

[0140] In the formula, represents the layer normalization LN operation, represents the input feature map of the th spatial aggregation module, represents the output feature after passing through the LN layer in the th spatial aggregation module.

[0141] After the layer normalization LN operation, a parallel three-branch structure is passed through, and each branch is executed using different operators, including the 3×3 depthwise convolution DWConv, the channel attention module CAB, and the learnable convolutional kernel attention LCK operation, which is expressed as:

[0142] ;

[0143] In the formula, represents the depthwise convolution operation, represents the channel attention module CAB, represents the convolutional kernel attention module.

[0144] In some embodiments, the channel attention module CAB includes two standard convolutional layers, a GELU activation function, and a spectral attention mechanism, where the spectral attention mechanism includes a global average pooling layer, a dimensionality reduction layer, a dimensionality increase layer, and a Sigmoid function.

[0145] In this embodiment, the output feature map formed by the global deep features output by the th SAB module is expressed as:

[0146] ;

[0147] Among them, is the output feature map of the th SAB module, represents pixel-wise multiplication.

[0148] In this embodiment, the global deep features are expressed as:

[0149] ;

[0150] In the formula, represents the th global deep feature, represents the th SAB module, represents the th global deep feature, represents the th SAB module, represents the first SAB module, represents the local multi-scale spatio-spectral feature.

[0151] The global spatio-spectral feature is expressed as:

[0152] ;

[0153] In the formula, represents the global spatio-spectral feature, " " represents pixel-wise addition.

[0154] In some embodiments, the output features of the three branches include the LCK output feature, wherein the method for obtaining the LCK output feature includes:

[0155] Learning a set of convolutional kernels shared at all spatial positions and all images using the layer-normalized local multi-scale spatio-spectral feature;

[0156] Using a lightweight convolutional layer branch to predict each spatial position of the layer-normalized local multi-scale spatio-spectral feature to obtain a fusion coefficient map, wherein the lightweight convolutional branch includes two 3×3 grouped convolutional layers and a SimpleGate activation function;

[0157] Linearly fusing the fusion coefficient feature map and the convolutional kernel, and calculating the fusion weight of each spatial position of the layer-normalized local multi-scale spatio-spectral feature through the linear combination of the convolutional kernels.

[0158] In this embodiment, the fusion weight at the spatial position located at is expressed as:

[0159] ;

[0160] In the formula, represents the fusion weight at the spatial position located at , represents the fusion coefficient of the th learnable convolutional kernel, Denote the th learnable convolutional kernel.

[0161] Transform the locally multi-scale spatial-spectral features after layer normalization through 1×1 convolution to obtain an enhanced feature map;

[0162] Perform grouped convolution operation on the fusion weights at each spatial position of the enhanced feature map and the locally multi-scale spatial-spectral features after layer normalization. Through grouped convolution calculation, obtain the fusion feature map at each spatial position as the LCK output feature.

[0163] In this embodiment, the LCK output feature map is expressed as:

[0164] ;

[0165] In the formula, represents the LCK output feature at the spatial position , represents the grouped convolution calculation, represents the enhanced feature map at the spatial position .

[0166] In this embodiment, the enhanced feature map is expressed as:

[0167] ;

[0168] In the formula, represents the enhanced feature map.

[0169] Step 2.4: After passing the single hyperspectral image through a 1×1 convolutional layer, fuse it with the global spatial-spectral features, and then pass through a 1×1 convolutional layer again to obtain a denoised hyperspectral image.

[0170] Among them, the method for denoising the hyperspectral image restoration network SSFSA, and the method for super-resolution reconstruction of the hyperspectral image restoration network SSFSA includes:

[0171] After obtaining the global spatial-spectral features, perform an upsampling operation, which is expressed as:

[0172] ;

[0173] In the formula, represents the feature map after upsampling the global spatial-spectral features, represents the upsampling function performed by the PixelShuffle operator.

[0174] Before passing the single hyperspectral image through a 1×1 convolutional layer, perform bicubic upsampling operation.

[0175] In this embodiment, the super-resolution restored hyperspectral image is expressed as:

[0176] ;

[0177] In the formula, represents the restored hyperspectral image, that is, the hyperspectral image after super-resolution reconstruction, represents the restored feature obtained by fusing the upsampled single hyperspectral image after passing through the 1×1 convolutional layer with the upsampled global spatial-spectral feature, represents the single hyperspectral image through bicubic upsampling operation.

[0178] The upsampling operation is only for the hyperspectral super-resolution reconstruction task. For the hyperspectral denoising task, no upsampling operation is required. At this time, the restored hyperspectral image is expressed as:

[0179] ;

[0180] In the formula, represents the restored hyperspectral image, that is, the denoised hyperspectral image.

[0181] In some embodiments, the weighted average method is used to weight the loss function and the spatial-spectral total variation SSTV loss function to obtain a loss function. The loss function is used to measure the difference between the restored hyperspectral image and the clear ground truth image of the single hyperspectral image. In this embodiment, the loss function is expressed as:

[0182] ;

[0183] In the formula, represents the overall loss, represents the parameter set of the hyperspectral image restoration network SSFSA, represents the loss function, represents the spatial-spectral total variation SSTV loss function, represents the weight coefficient used to balance the two losses, where, .

[0184] Among them, the loss function is expressed as:

[0185] ;

[0186] In the formula, represents the loss, is the number of images in a training batch, and respectively represent the clear ground truth image and the corresponding restored hyperspectral image of the th single-frame hyperspectral image. Among them, the spatio-spectral total variation (SSTV) loss function is expressed as:

[0187] ;

[0188] In the formula, represents the spatio-spectral total variation (SSTV) loss, , and respectively represent the functions for calculating the horizontal, vertical, and spectral gradients.

[0189] Embodiment 3

[0190] Based on the same inventive concept as other embodiments, this embodiment introduces a computer-readable storage medium storing computer instructions, characterized in that when the computer instructions are executed by a processor, the steps of the method in the above Embodiment 1 or 2 are implemented.

[0191] Embodiment 4

[0192] Based on the same inventive concept as other embodiments, the present invention further provides a computer program product including computer instructions, characterized in that when the computer instructions are executed by a processor, the steps of the method in the above Embodiment 1 or 2 are implemented.

[0193] In summary of the above embodiments, by adopting the spectral grouping and parameter sharing strategies, the present invention makes full use of the strong correlation between adjacent bands, effectively improves the calculation efficiency during the restoration process and the accuracy of feature extraction. At the same time, combined with the multi-scale spatio-spectral fusion module (MSSF), it captures the spatial and spectral information of the image at different scale features, achieving a more detailed and refined image restoration effect. In addition, the adaptive spatial aggregation group (ASAG) is also applied, which adaptively optimizes the aggregation process of spatial information through a learnable convolution kernel and automatically adjusts the aggregation strategy according to the image content, thereby significantly restoring the image details and structure. In summary, the hyperspectral image restoration method provided by the present invention can significantly improve the restoration accuracy, effectively cope with the noise introduced during the hyperspectral imaging process and the problem of reduced spatial resolution. Especially in the case of insufficient and unbalanced samples, it can also achieve stable image quality restoration.

[0194] The present invention designs a multi-scale spatio-spectral fusion module MSSF. Firstly, by using a parallel-branch group convolution structure including a semi-channel branch and a full-channel branch, it effectively fuses spatial and spectral information at different scales, can enhance the internal and external relationships between different spectral bands in parallel, and is conducive to capturing smooth image features related to low-frequency information. Among them, the semi-channel branch reduces the computational amount while maintaining the richness of features, and the full-channel branch further deeply extracts global features. Secondly, each branch uses a layer of dilated convolution respectively, which can increase the receptive field without increasing network parameters and extract multi-scale spatial information from different receptive fields. Finally, a channel attention module with a residual connection is added to exploit the spectral similarity and complementarity to mine the correlation of spectral features. This not only improves the computational efficiency but also significantly enhances the fineness and accuracy of image restoration, especially when dealing with complex hyperspectral images.

[0195] The adaptive spatio-feature aggregation group ASAG provided by the present invention realizes the dynamic adjustment and adaptive aggregation of local multi-scale spatio-spectral features through cascaded adaptive spatio-aggregation modules SAB, combined with long and short skip connections to form residuals. In particular, the three-branch structure within the adaptive spatio-aggregation module SAB includes a channel attention module CAB, a 3×3 depthwise convolution DWConv, and a learnable convolution kernel attention LCK, which can finely extract and fuse spatial and spectral features, further enhancing the detail recovery ability and structure preservation of image restoration. In addition, the learnable convolution kernel attention LCK realizes the adaptive selection and fusion of learnable convolution kernels at each position by predicting a fusion coefficient map, thereby enhancing the expression ability of global spatio-spectral features.

[0196] Those skilled in the art should understand that the embodiments of the present invention can be provided as a method, a system, or a computer program product. Therefore, the present invention can take the form of a completely hardware embodiment, a completely software embodiment, or an embodiment combining software and hardware aspects. Moreover, the present invention can take the form of a computer program product implemented on one or more computer-usable storage media (including but not limited to disk storage, CD-ROM, optical storage, etc.) containing computer-usable program code.

[0197] The present invention is described with reference to the flowcharts and / or block diagrams of methods, devices (systems), and computer program products according to the embodiments of the present invention. It should be understood that each process and / or block in the flowchart and / or block diagram, and the combination of processes and / or blocks in the flowchart and / or block diagram, can be realized by computer program instructions. These computer program instructions can be provided to the processor of a general-purpose computer, a special-purpose computer, an embedded processor, or other programmable data processing devices to generate a machine, so that the instructions executed by the processor of the computer or other programmable data processing devices generate for realizing the processFigure 1 one process or multiple processes and / or blocks Figure 1 means for the functions specified in one block or multiple blocks.

[0198] These computer program instructions can also be stored in a computer-readable memory that can direct a computer or other programmable data processing device to work in a specific manner, such that the instructions stored in the computer-readable memory produce a manufacture including an instruction means that implements the functions in the process Figure 1 one process or multiple processes and / or blocks Figure 1 specified in one block or multiple blocks.

[0199] These computer program instructions can also be loaded onto a computer or other programmable data processing device, such that a series of operation steps are performed on the computer or other programmable device to produce a computer-implemented process, so that the instructions executed on the computer or other programmable device provide steps for implementing the functions specified in the process Figure 1 one process or multiple processes and / or blocks Figure 1 specified in one block or multiple blocks.

[0200] The embodiments of the present invention have been described above in conjunction with the accompanying drawings. However, the present invention is not limited to the above specific embodiments. The above specific embodiments are merely illustrative and not restrictive. Under the inspiration of the present invention, those of ordinary skill in the art can also make many forms without departing from the spirit of the present invention and the scope protected by the claims. All of these fall within the protection scope of the present invention.

Claims

1. A hyperspectral image restoration method, characterized in that: include: Obtain a single hyperspectral image HSI to be restored; Inputting the single hyperspectral image HSI to be restored into the hyperspectral image restoration network SSFSA to perform denoising / super-resolution reconstruction to obtain a restored hyperspectral image; The restored hyperspectral image includes a denoised hyperspectral image / a hyperspectral image reconstructed with super resolution; The method for denoising the hyperspectral image restoration network SSFSA comprises: Using the spectral overlapping grouping and parameter sharing strategy, the number of overlapping spectral channels is set, and the single hyperspectral image HSI is overlapped and grouped along the spectral dimension. Each grouped hyperspectral image HSI is input into the corresponding multi-scale spatial-spectral fusion module MSSF, and the mapping relationship in the multi-scale space is extracted to obtain all branch features. All branch features are cascaded and dimensionally upgraded through a 1×1 convolutional layer to obtain the local multi-scale spatial-spectral features after dimension upgrade; among them, all groups share the network parameters of each multi-scale spatial-spectral fusion module MSSF; The local multi-scale spatial-spectral features after dimension upgrading are input into the adaptive spatial feature aggregation group ASAG to obtain the global spatial-spectral features; After passing the single hyperspectral image through a 1×1 convolution layer, it is fused with the global spatial-spectral features and passed through a 1×1 convolution layer again to obtain a denoised hyperspectral image; Among them, the method of extending the hyperspectral image restoration network SSFSA for denoising and obtaining the method of super-resolution reconstruction using the hyperspectral image restoration network SSFSA includes: After obtaining the global spatial-spectral features, an upsampling operation is performed; Before passing the single hyperspectral image through a 1×1 convolutional layer, a bicubic upsampling operation is performed.

2. The hyperspectral image restoration method according to claim 1, characterized in that: The single hyperspectral image is grouped along the spectral dimension and input into the multi-scale spatial-spectral fusion module MSSF to extract the mapping relationship in the multi-scale space and obtain all branch features. All branch features are cascaded and dimensionally upgraded through a 1×1 convolutional layer to obtain the local multi-scale spatial-spectral features after dimension upgrade, including: After grouping the single hyperspectral image along the spectral dimension, they are input into two half-channel branches and one full-channel branch of the multi-scale spatial-spectral fusion module MSSF respectively to obtain two half-channel branch features and one full-channel branch feature; After the two half-channel branch features are cascaded, the ReLU activation function is used, and then pixel-by-pixel addition is performed with the full-channel branch features to obtain the fused multi-scale features; After the fused multi-scale features are passed through the spatial residual module, multi-scale spatial features are obtained; The multi-scale spatial features are fed into the residual connected channel attention module CAB to obtain multi-scale spatial-spectral features; After pixel-by-pixel addition of the multi-scale spatial-spectral features and the multi-scale spatial features, the branch features are obtained; All branch features are cascaded and dimensionally upgraded through a 1×1 convolutional layer to obtain the local multi-scale spatial-spectral features after dimension upgrade.

3. The hyperspectral image restoration method according to claim 1 or 2, characterized in that: The local multi-scale spatial-spectral feature is expressed as: ； In the formula, represents the local multi-scale spatial-spectral characteristics after dimensionality increase, represents the number of spectral channels after dimension increase, represents the height in the image spatial resolution, represents the width in the image spatial resolution, represents a 1×1 convolution operation, Indicates cascade operation, represents the local multi-scale spatial-spectral features obtained by cascading all branch features, represents the first branch feature, represents the second branch feature, Indicates Branch features, Indicates Branch features, Indicates the number of spectral channels of the single hyperspectral image HSI to be restored.

4. The hyperspectral image restoration method according to claim 2, characterized in that: After the single hyperspectral image is grouped along the spectral dimension, it is respectively input into two half-channel branches and one full-channel branch of the multi-scale spatial spectrum fusion module MSSF to obtain two half-channel branch features and one full-channel branch feature, including: After grouping a single hyperspectral image along the spectral dimension, the number of spectral channels of each group is divided into two and input into the two half-channel branches of the multi-scale spatial-spectral fusion module MSSF respectively. After two 3×3 convolution operations, each 3×3 convolution operation is processed by the ReLU activation function, and then another 3×3 convolution operation is performed, and finally a layer of dilated convolution is processed to obtain two half-channel branch features; After grouping a single hyperspectral image along the spectral dimension, all spectral channels of each group are input into the full-channel branch of the multi-scale spatial-spectral fusion module MSSF, processed by three cascaded 3×3 convolutions plus ReLU activation functions, and then processed by a layer of dilated convolution to obtain the full-channel branch features.

5. The hyperspectral image restoration method according to claim 2, characterized in that: The multi-scale spatial feature is expressed as: ； In the formula, Represents multi-scale spatial features, Represents the multi-scale features after the fusion of the half-channel branch output features and the full-channel branch output features, Represents the first grouping of a single hyperspectral image along the spectral dimension Groups, Represents a 3×3 convolution operation.

6. The hyperspectral image restoration method according to claim 2, characterized in that: The multi-scale spatial features are sent to the residual connected channel attention module CAB to obtain multi-scale spatial-spectral features, including: The multi-scale spatial features are subjected to two 3×3 convolution operations to obtain the first channel attention features, where the GELU activation function is used after the first 3×3 convolution operation; The first channel attention feature is passed through the global average pooling layer and then through two 3×3 convolution operations to obtain the second channel attention feature, where the ReLU activation function is used after the first 3×3 convolution operation; After the attention feature of the second channel passes through the Sigmoid function, it is multiplied pixel by pixel with the attention feature of the first channel to obtain the multi-scale spatial-spectral feature.

7. The hyperspectral image restoration method according to claim 1, characterized in that: The adaptive spatial feature aggregation group ASAG includes cascaded adaptive spatial aggregation modules SAB, the The cascaded adaptive spatial aggregation modules form the residual through long and short skip connections, where .

8. The hyperspectral image restoration method according to claim 7, characterized in that: The local multi-scale spatial-spectral features after dimensionality increase are input into the adaptive spatial feature aggregation group ASAG to obtain the global spatial-spectral features, wherein each adaptive spatial aggregation module SAB cascaded in the adaptive spatial feature aggregation group ASAG includes: For each adaptive spatial aggregation module SAB, after layer normalization, it passes through a parallel three-branch structure including a channel attention module CAB, a 3×3 deep convolution DWConv, and a learnable convolution kernel attention LCK to obtain the output features of the three branches; The output features of the three branches are aggregated with spatial and spectral features using pixel-by-pixel multiplication, and then passed through a 1×1 convolutional layer and a residual connection to obtain the global deep features output by the adaptive spatial aggregation module SAB. Among them, the learnable convolution kernel attention LCK is used to adaptively process the spatial features of the local multi-scale spatial-spectral features after layer normalization; the 3×3 deep convolution DWConv is used to capture the spatial invariant features of the local multi-scale spatial-spectral features after layer normalization; the channel attention module CAB is used to further extract and refine the spectral features of the local multi-scale spatial-spectral features after layer normalization.

9. The hyperspectral image restoration method according to claim 1 or 8, characterized in that: The local multi-scale spatial-spectral features after dimensionality increase are subjected to N cascaded SABs to obtain global deep features, which are expressed as: ； In the formula, Indicates A global deep feature, Indicates SAB modules, Indicates A global deep feature, Indicates SAB modules, Indicates the first SAB module, Represents the local multi-scale spatial-spectral characteristics after dimensionality increase; The global spatial-spectral signature is expressed as: ； In the formula, represents the global spatial-spectral characteristics, " ” indicates pixel-by-pixel addition.

10. The hyperspectral image restoration method according to claim 8, characterized in that: The output features of the three branches include LCK output features, wherein the method for obtaining the LCK output features includes: We use layer-normalized local multi-scale spatial-spectral features to learn a set of convolutional kernels that are shared across all spatial locations and all images. A lightweight convolutional layer branch is used to predict each spatial position of the local multi-scale spatial-spectral features after normalization of the prediction layer to obtain a fusion coefficient map, wherein the lightweight convolutional branch includes two 3×3 grouped convolutional layers and a SimpleGate activation function; Linearly fuse the fusion coefficient feature map and the convolution kernel, and calculate the fusion weight of each spatial position of the local multi-scale spatial-spectral feature after layer normalization through the linear combination of the convolution kernel; The local multi-scale spatial-spectral features after layer normalization are transformed through 1×1 convolution to obtain enhanced feature maps; The enhanced feature map and the fusion weight of each spatial position of the layer-normalized local multi-scale spatial-spectral feature are subjected to a group convolution operation, and the LCK output feature at each spatial position is obtained through group convolution calculation.

Images