A hyperspectral compressive sensing reconstruction method based on scale adaptive local-global collaborative recovery

Abstract

The application provides a hyperspectral compression sensing reconstruction method based on scale adaptive local-global collaborative recovery, which comprises: intermediate hyperspectral estimation and spectral change guide information fusion are performed to execute feature embedding; an encoder executes at least one level of difference window attention block operation and down-sampling operation on the embedded features and outputs low-resolution encoded features; at least one level of difference random reconstruction block operation and down-sampling operation is performed on the low-resolution encoded features and bottleneck features are output; global long-range information aggregation is performed on the bottleneck features through the difference random reconstruction block; a decoder gradually recovers the feature space resolution through multi-level up-sampling, and the difference random reconstruction block and the difference window attention block corresponding to the encoder are symmetrically arranged at each level; and at each level of the reconstruction stage, the same scale encoded features of the encoder are fused through cross-layer jump connection; the high-resolution features finally output by the decoder are sent to a mapping layer to complete dimension conversion, and a complete three-dimensional hyperspectral reconstruction image is output.

Description

Technical Field

[0001] This invention relates to the fields of computer vision and deep learning technology, and in particular to a hyperspectral compressed sensing reconstruction method based on scale-adaptive local-global collaborative recovery. Background Technology

[0002] Hyperspectral images simultaneously record the spatial structure and spectral response of a scene across a continuous narrow band, enabling them to describe the spectral characteristics of different materials, targets, or tissues. They have significant application value in fields such as remote sensing, agricultural monitoring, target recognition, medical image analysis, and industrial visual inspection.

[0003] Traditional hyperspectral imaging typically relies on point-by-point, line-by-line, or band-by-band scanning, resulting in long acquisition times and making it unsuitable for dynamic scenes. Compressed sensing imaging methods, such as coded aperture snapshot spectral imaging, improve imaging efficiency by compressing three-dimensional hyperspectral data into a single-frame two-dimensional measurement through coded modulation, dispersion shifting, and detector integration. However, the superposition of different spatial locations and different bands on the two-dimensional detector makes hyperspectral image reconstruction a severely ill-conditioned inverse problem.

[0004] In compressed measurements, local textures, target edges, and fine structures are easily aliased; meanwhile, structural connections between distant spatial regions are difficult to recover directly from single-frame measurements. Local modeling is beneficial for preserving textures and boundaries, but it is limited by the receptive field and struggles to recover long-range spatial dependencies; global modeling can promote information exchange between non-neighboring regions, but if applied directly to high-resolution features, it incurs high computational and memory overhead and may weaken fine-grained structures during large-scale feature aggregation.

[0005] Existing hyperspectral reconstruction networks typically reuse the same convolutional blocks or attention blocks across different resolution layers, or simply increase representational power by increasing network depth and attention range. This unified modeling approach fails to fully utilize the differences between multi-scale features: high-resolution features contain richer local textures and subtle spectral transitions, while low-resolution features have fewer spatial tokens and are better suited for long-range interactions and global structure modeling. Therefore, it is necessary to design a reconstruction method that adaptively allocates local and global recovery tasks based on feature scale. Summary of the Invention

[0006] This invention proposes a hyperspectral compressed sensing reconstruction method based on scale-adaptive local-global collaborative recovery to solve or partially solve the above-mentioned problems.

[0007] This invention provides a hyperspectral compressed sensing reconstruction method based on scale-adaptive local-global collaborative recovery, the method comprising: Obtain intermediate hyperspectral estimates and spectral change guidance information that matches the spatial dimension of the intermediate hyperspectral estimates; fuse the intermediate hyperspectral estimates and spectral change guidance information, perform feature embedding, obtain embedded features, and send them to the encoder; The encoder performs at least one level of differential window attention block operation and downsampling operation on the embedded features to combine spectral change guidance information to complete local correlation estimation and local detail recovery, and outputs low-resolution encoded features; it performs at least one level of differential random reconstruction block operation and downsampling operation on the low-resolution encoded features, selectively injects spectral change guidance information to complete global spectral and spatial structure correction, and outputs bottleneck features with the smallest size. The bottleneck features are aggregated globally using differential random reconstruction blocks to perform long-range information aggregation. After completing the global structural consistency modeling of the entire graph, the data is sent to the decoder. The decoder recovers the feature space resolution step by step through multi-level upsampling. It uses hierarchically symmetrical configuration of differential random reconstruction blocks and differential window attention blocks corresponding to the encoder to achieve global structure optimization and local detail refinement on the decoding side, respectively. In each reconstruction stage, the decoder fuses the encoder's encoded features at the same scale through cross-layer skip connections to make up for the multi-scale spatial and spectral detail information lost during the upsampling process. The high-resolution features output by the decoder are fed into the mapping layer to complete the dimensionality transformation, and the output is a complete three-dimensional hyperspectral reconstructed image that matches the original input size and number of spectral channels.

[0008] Further, the acquisition of intermediate hyperspectral estimates and spectral change guidance information matching the spatial dimension of the intermediate hyperspectral estimates includes: Initial spectral reconstruction is performed based on the two-dimensional compressed measurement, system coding parameters, and dispersion parameters output by the hyperspectral compressed sensing imaging system to obtain an intermediate hyperspectral estimate with complete spectral channel dimensions. Feature extraction is performed on the band variation law and spatial spectral type difference of the intermediate hyperspectral estimate to generate spectral change guidance information that corresponds one-to-one with the spatial location of the intermediate hyperspectral estimate.

[0009] Furthermore, the difference window attention block performs the following operations: The original features input to the differential window attention block are normalized and then fed into the differential prior guided window attention block for computation, outputting high-resolution attention features; The high-resolution attention features and the original features input to the differential window attention block are fused and then fed into the dual-branch detail recovery block operation to compensate for image texture and edge details, thereby obtaining high-resolution detail recovery features; After normalizing the high-resolution detail recovery features again, they are fed into a gated feedforward network for computation. The output features of the gated feedforward network are then residually fused with the high-resolution detail recovery features to obtain the output features of the differential window attention block.

[0010] Furthermore, the differential prior guidance window attention block performs the following operations: The spectral change guidance information is projected onto a feature space compatible with the current input features, and the projected spectral change guidance information is fused with the current input features to obtain the guidance enhancement representation; The guided enhancement representation generates the first query feature and the first key feature, and generates the first value feature only from the current input feature, so that the spectral change guidance information participates in the local correlation estimation of the feature; Divide the first query feature, the first key feature, and the first value feature into multiple local windows, perform window attention interaction, and output the local attention features; Based on the local attention features and the spectral change guidance information after projection, a priori perception gating is generated. The priori perception gating is used to selectively modulate the local attention output position by position, channel by channel, or group by group, thereby enhancing the attention response in reliable spectral change regions and suppressing the response in unreliable regions, and finally outputting high-resolution attention features.

[0011] Furthermore, the dual-branch detail recovery block performs the following operations: The current input features of the dual-branch detail recovery block are fed into the parallel local refinement branch and the extended neighborhood branch in two paths. After fusing the output features of the local refinement branch and the extended neighborhood branch with the current input features of the dual-branch detail recovery block, a channel blending transformation is performed. The channel-transformed features are fused with the current input feature residual of the dual-branch detail recovery block to output high-resolution detail recovery features.

[0012] Furthermore, the differential random reconstruction block is used to perform the following operations: The current input features of the differential random reconstruction block are normalized, the spectral change guidance information is projected onto a feature space compatible with the current input features, and simultaneously sent to the feature block routing module. The feature block routing module converts the normalized backbone features into a sequence of basic feature block vectors and converts the synchronously input spectral change guidance information into a sequence of differential feature block vectors. At the same time, it traverses each spatial feature block and generates a corresponding feature block-level injection mask based on the spectral statistical features such as the spectral amplitude and average fluctuation within each spatial feature block. The output features of the feature block routing module are input into the differential grouped spectral attention module. The injection intensity of the differential feature block vector sequence is controlled by the feature block-level injection mask differentiation to obtain the grouped spectral attention features. The grouped spectral attention features are fed into the random global block fusion module to complete the cross-block fusion of long-distance global features and obtain global interactive enhancement features; The global interactive enhancement features are fused with the current input features of the differential random reconstruction block and then fed into the dual-branch detail recovery block operation to obtain low-resolution detail recovery features; After normalizing the low-resolution detail recovery features again, they are fed into the gated feedforward network for computation to complete the channel feature transformation. Finally, the output features of the gated feedforward network are residually fused with the low-resolution detail recovery features to obtain the output features of the differential random reconstruction block.

[0013] Furthermore, the differential grouped spectral attention module performs the following operations: The basic feature block vector sequence and the difference feature block vector sequence are fused and then sent to the grouping module, which divides them into multiple grouped feature block sequences according to the spectral channels. The grouped feature block sequences are then transformed by a linear transformation layer to generate the second query feature, the second key feature, and the second value feature, respectively. The original attention weights are calculated based on the second query features and the second key features. The original attention weights are then weighted and fused with the second value features to obtain the fused attention features. The fused attention features and the differential feature block vector sequence are input into the gated fusion unit to complete the adaptive weighted fusion of the two features, and the fused spectral correlation features are obtained. Then, a feature block-level injection mask is introduced to perform differential scaling for each feature block. The features after feature block-level injection mask modulation are residually fused with the basic feature block vector sequence to output grouped spectral attention features.

[0014] Furthermore, the random global block mixing module performs the following operations.

[0015] The feature block vector sequence of the grouped spectral attention features is added and fused with the pre-generated position encoding features element by element, and the original spatial coordinate information of each feature block is embedded in the grouped spectral attention features; The feature block vector sequence with fused location information is fed into the random grouping module, which generates a global random rearrangement index and an inverse rearrangement index corresponding to the global random rearrangement index based on a fixed random seed. The feature block vector sequence with fused location information is shuffled and rearranged according to the global random rearrangement index to generate a global hybrid feature block vector sequence. The global hybrid feature block vector sequence is subjected to layer normalization, multi-head self-attention operation, second layer normalization, and feedforward network channel nonlinear transformation in sequence to obtain the global interactive enhancement hybrid feature sequence. Based on the inverse rearrangement index, the global interactive enhancement feature sequence is restored to the original feature block arrangement order, and the feature block vector sequence after restoration of the arrangement order is remapped into a spatial feature map to obtain the global interactive enhancement feature.

[0016] Furthermore, in each reconstruction stage of the decoder, the same-scale encoded features of the encoder are fused through cross-layer skip connections, including: Obtain the encoder's output at the same scale level, and the encoded features after processing by differential window attention blocks or differential random reconstruction blocks; Obtain the decoding features output from the decoder's local upsampling module and after resolution upsampling; The encoder's same-scale encoded features and the decoder's local upsampled decoded features are input into the channel splicing unit, and the two features are spliced ​​along the channel dimension; The hybrid features obtained by channel splicing are fed into the fusion module, and convolution operation is used to complete channel mixing, thereby realizing the fusion of shallow spatial detail features of the encoder and deep reconstruction features of the decoder. The output features of the fusion module are fed into the differential window attention block or differential random reconstruction block corresponding to the current level of the decoder to complete the feature reconstruction of this level.

[0017] This invention proposes a hyperspectral compressed sensing reconstruction method based on scale-adaptive local-global collaborative restoration. According to the spatial resolution and information distribution characteristics of multi-scale features, fine-grained local restoration tasks and long-range global modeling tasks are assigned to different scales. At the high-resolution layer, spectral variations are used to guide information to participate in local correlation estimation. At the low-resolution layer, feature block-level reliable routing, grouped spectral interaction, and reversible random global mixing are used to improve long-range structural modeling capabilities. Ultimately, with limited model parameters and computational resources, the spatial texture, boundary continuity, spectral fidelity, and global structural consistency of hyperspectral images are improved.

[0018] The above description is merely an overview of the technical solution of the present invention. In order to better understand the technical means of the present invention and to implement it in accordance with the contents of the specification, and to make the above and other objects, features and advantages of the present invention more apparent and understandable, specific embodiments of the present invention are described below. Attached Figure Description

[0019] Various other advantages and benefits will become apparent to those skilled in the art upon reading the following detailed description of preferred embodiments. The accompanying drawings are for illustrative purposes only and are not intended to limit the invention. In the drawings: Figure 1 This is a diagram of the overall network architecture of the hyperspectral compressed sensing reconstruction method based on scale-adaptive local-global collaborative recovery according to an embodiment of the present invention.

[0020] Figure 2 This is a network architecture diagram of each sub-module in the overall network architecture diagram of an embodiment of the present invention, wherein: Figure 2 (a) is a network architecture diagram of the differential window attention block; Figure 2 (b) is a network architecture diagram of the differential random reconstruction block; Figure 2 (c) is the network architecture diagram of the gated feedforward network; Figure 2 (d) is the network architecture diagram of the differential prior guidance window attention block; Figure 2 (e) is the network architecture diagram of the differential grouped spectral attention module; Figure 2 (f) is the network architecture diagram of the dual-branch detail recovery block; Figure 2 (g) is the network architecture diagram of the random global block hybrid module. Detailed Implementation

[0021] Exemplary embodiments of the present disclosure will now be described in more detail with reference to the accompanying drawings. While exemplary embodiments of the present disclosure are shown in the drawings, it should be understood that the present disclosure may be implemented in various forms and should not be limited to the embodiments set forth herein. Rather, these embodiments are provided so that this disclosure will be thorough and complete, and will fully convey the scope of the disclosure to those skilled in the art.

[0022] This invention addresses the challenges in hyperspectral compressed sensing reconstruction, including the difficulty in simultaneously preserving local details and global structure, the repeated use of a unified recovery operator across different scales, the easy indiscriminate injection of spectral change-guided information, and the high computational cost of high-resolution global modeling. It proposes a hyperspectral compressed sensing reconstruction method based on scale-adaptive local-global collaborative recovery. (Refer to...) Figure 1 The hyperspectral compressed sensing reconstruction method based on scale-adaptive local-global collaborative recovery provided by this invention specifically includes the following steps: S1. Obtain the intermediate hyperspectral estimate (Original Input) and the spectral change guidance information (Spectral Prior) that matches the spatial dimension of the intermediate hyperspectral estimate; after fusing the intermediate hyperspectral estimate and the spectral change guidance information, perform feature embedding to obtain embedded features and send them to the encoder; S2. The encoder performs at least one level of Differential Window Attention Block (DWB) operation and downsampling operation on the embedded features to combine spectral change guidance information to complete local correlation estimation and local detail recovery, and outputs low-resolution encoded features; it performs at least one level of Differential Random Reconstruction Block (DRB) operation and downsampling operation on the low-resolution encoded features to selectively inject spectral change guidance information to complete global spectral and spatial structure correction, and outputs bottleneck features with the smallest size. S3. Perform global long-range information aggregation on bottleneck features through differential random reconstruction blocks (DRB), and after completing the global structural consistency modeling of the whole graph, send it to the decoder. S4. The decoder restores the feature space resolution step by step through multi-level upsampling. It uses a hierarchically symmetrical configuration of differential random reconstruction blocks and differential window attention blocks corresponding to the encoder to achieve global structure optimization and local detail refinement on the decoding side, respectively. In each reconstruction stage, the decoder fuses the encoder's encoded features at the same scale through cross-layer skip connections to make up for the multi-scale spatial and spectral detail information lost during the upsampling process. S5. The high-resolution features output by the decoder are fed into the mapping layer to complete the dimensionality transformation and output a complete three-dimensional hyperspectral reconstructed image that matches the original input size and number of spectral channels (Denoising Output).

[0023] Based on the spatial resolution and information distribution characteristics of multi-scale features, this invention allocates fine-grained local restoration tasks and long-range global modeling tasks to different scales; at the high-resolution layer, spectral variations are used to guide information to participate in local correlation estimation; at the low-resolution layer, feature block-level reliable routing, grouped spectral interaction, and reversible random global mixing are used to improve long-range structural modeling capabilities; ultimately, with limited model parameters and computational resources, the spatial texture, boundary continuity, spectral fidelity, and global structural consistency of hyperspectral images are improved.

[0024] Specifically, such as Figure 1 As shown, the encoding path of this invention sequentially completes high-resolution local restoration and low-resolution global restoration; the bottleneck layer preferably uses a global restoration unit to further integrate long-range structural information. The decoding path restores spatial resolution through stepwise upsampling and concatenates, adds, or fuses features of the same scale at the encoding end with features at the decoding end through attention. A local restoration unit is used again in the high-resolution decoding layer to refine the boundaries, textures, and subtle spectral transitions of the features returned after global restoration; a global restoration unit is used in the low-resolution decoding layer to maintain structural consistency. Finally, the hyperspectral reconstruction result or the prior restoration result of this stage is output through the mapping layer.

[0025] Furthermore, in each reconstruction stage of the decoder in this embodiment of the invention, the following steps are taken to fuse the encoder's same-scale encoding features through cross-layer skip connections: acquiring the encoder's same-scale layer output and the encoded features processed by differential window attention blocks or differential random reconstruction blocks; acquiring the decoder's own-level upsampling module output and the decoded features after resolution enhancement; inputting the encoder's same-scale encoding features and the decoder's own-level upsampling decoding features into the channel splicing unit, and splicing the two features along the channel dimension; sending the hybrid features obtained by channel splicing into the fusion module (Fusion), and using convolution operations to complete channel mixing, thereby realizing the fusion of the encoder's shallow spatial detail features and the decoder's deep reconstruction features; and sending the fusion module output features into the corresponding differential window attention block or differential random reconstruction block of the decoder to complete the feature reconstruction of this level.

[0026] Further, in step S1, obtaining the intermediate hyperspectral estimate and the spectral change guidance information matching the spatial dimension of the intermediate hyperspectral estimate includes: performing initial spectral reconstruction based on the two-dimensional compressed measurement, system coding parameters and dispersion parameters output by the hyperspectral compressed sensing imaging system to obtain an intermediate hyperspectral estimate with complete spectral channel dimensions; extracting features from the band variation law and spatial spectral type difference of the intermediate hyperspectral estimate to generate spectral change guidance information corresponding one-to-one with the spatial position of the intermediate hyperspectral estimate.

[0027] Specifically, this invention acquires two-dimensional compressed measurement data output by a hyperspectral compressed sensing imaging system, while simultaneously reading the system's associated encoding information, dispersion parameters, and sensing model. It then employs any combination of initial backprojection, physical data consistency iterative updates, and convolutional mapping to obtain an intermediate hyperspectral estimate containing all spectral channel dimensions. This intermediate hyperspectral estimate can be the iterative update result of any level in the deep unfolded reconstruction network, or it can be the intermediate spatial spectral features output from an end-to-end hyperspectral reconstruction network.

[0028] After obtaining the intermediate hyperspectral estimate, spatial spectral features are extracted to generate spatially matched spectral change guidance information. This spectral change guidance information characterizes the spectral change trends and differences corresponding to each spatial location and spectral band in the image. This invention does not limit the spectral difference order or the prior feature extraction network structure; it only requires that the generated spectral change guidance information and the synchronously input intermediate reconstructed features maintain strict alignment and matching in both spatial and feature dimensions. This ensures that the feature block routing module in subsequent differential random reconstruction blocks can simultaneously process the backbone features and the spectral change guidance information.

[0029] Furthermore, the intermediate hyperspectral estimation and spectral change guidance information are first passed through the gated fusion module (GatedFusion), which uses the built-in Sigmoid activation gate unit to achieve adaptive weighted fusion of the two features; the fused features output by the gated fusion module are sent to the feature embedding module (Embedding) to complete the dimension mapping and feature serialization embedding processing.

[0030] Furthermore, in the encoding path, the high-resolution layer retains a large number of spatial feature block vectors (spatial tokens), containing rich textures, boundaries, small target structures, and subtle band transitions. As downsampling progresses, the number of spatial feature block vectors (spatial tokens) in the low-resolution layer gradually decreases, and each spatial feature block vector has a larger effective receptive range, making it more suitable for cross-regional and long-range spatial interactions. Based on this scale characteristic, this invention does not reuse the same recovery unit across all scales. Instead, it sets the high-resolution encoding layer and its corresponding decoding layer as local recovery units, and the low-resolution encoding layer, bottleneck layer, and its corresponding decoding layer as global recovery units. Preferably, the high-resolution layer uses Differential Window Attention Block (DWB) for local correlation modeling and detail compensation, while the low-resolution layer uses Differential Random Reconstruction Block (DRB) for reliability routing and long-range structure modeling. During decoding, spatial resolution is restored step-by-step through upsampling, skip connections, and feature fusion.

[0031] Specifically, refer to Figure 2 (a) In step S2, the differential window attention block (DWB) performs the following operations: S21. After normalizing the original features input to the Differential-Prior Guided Window Attention (DWB) (Layer Norm), the features are fed into the Differential-Prior Guided Window Attention (DWB) for computation, and high-resolution attention features are output. S22. The high-resolution attention features and the original features input to the differential window attention block are fused and then fed into the dual-branch detail recovery block operation to compensate for image texture and edge details, thereby obtaining high-resolution detail recovery features. S23. After performing normalization (Layer Norm) on the high-resolution detail recovery features again, the data is fed into a gated feedforward network (Gated FFN) for computation. The output features of the gated feedforward network are then residually fused with the high-resolution detail recovery features to obtain the output features of the differential window attention block.

[0032] Furthermore, referring to Figure 2(d) In this embodiment of the invention, the Differential-Prior Guided Window Attention block performs the following operations: S31. Project the spectral change guidance information onto a feature space compatible with the current input features, and fuse the projected spectral change guidance information with the current input features to obtain the guidance enhancement representation; S32. The first query feature and the first key feature are generated by the guided enhancement representation, and the first value feature is generated only by the current input feature, so that the spectral change guidance information participates in the local correlation estimation of the feature. S33. Divide the first query feature, the first key feature, and the first value feature into multiple local windows, perform window attention interaction, and output the local attention features; Unlike ordinary window attention, this invention separates the sources of local relevance estimation and content aggregation: the first query feature and the first key feature are generated by guided enhanced representation, so that spectral change information participates in the similarity estimation between pixels or tokens; the first value feature is generated by the original input feature, so that the final aggregated content mainly comes from the original spatial-spectral representation.

[0033] By employing the aforementioned asymmetric feature generation method, local locations with similar spectral variation patterns are more likely to achieve high correlation, while the original texture and spatial content are not directly replaced by spectral priors. This method is particularly suitable for scenarios in compressed spectral reconstruction where adjacent band variations are weakened, but the original spatial details still need to be preserved as much as possible. Subsequently, the first query feature, the first key feature, and the first value feature are divided into several local windows for attention interaction. The windows can be non-overlapping windows, or overlapping windows, shifted windows, or other local region division methods can be used. After completing the window interaction, the original spatial arrangement is restored to obtain the local attention output.

[0034] S34. Generate a priori perception gating based on the local attention features and the spectral change guidance information after projection. Use the priori perception gating to selectively modulate the local attention output position by position, channel by channel, or group by group to enhance the attention response in reliable spectral change regions and suppress the response in unreliable regions, and finally output high-resolution attention features.

[0035] To avoid the spectral change guidance information having the same effect in all local regions, this invention further generates a priori perceptual gating based on the local attention output and the projected spectral change guidance information. The gating result is used to modulate the local attention response position-by-position, channel-by-channel, or group-by-group. When a region has a clear and reliable spectral transition, the gating enhances the corresponding attention response; when a region has weak spectral changes, unstable fluctuations, or unreliable prior information, the gating reduces its modulation amplitude. Thus, spectral change information participates in local recovery selectively, rather than being uniformly applied as a fixed weight to all features.

[0036] Furthermore, window attention may weaken texture, edges, and small-scale structure when aggregating locally relevant information. Therefore, this invention sets up a detail restoration module after prior-guided window attention, referring to... Figure 2 (f) The dual-branch detail recovery block of the present invention performs the following operations: S41. The current input features of the dual-branch detail recovery block are sent to the parallel local refinement branch and the extended neighborhood branch in two paths. The local refinement branch uses 1×1 convolution combined with standard depth separable convolution or other lightweight local operators to accurately restore image texture, edges and fine structures; the extended neighborhood branch preferably uses dilated depth separable convolution (D-Conv), large-size convolution kernels or other lightweight operators that can expand the receptive field to extract large-scale neighborhood contextual structure features.

[0037] S42. After fusing the output features of the local refinement branch and the extended neighborhood branch with the current input features of the dual-branch detail recovery block, perform channel blending transformation; In this embodiment, the output features of the two branches are concatenated with the original input features through channels or added element by element. Preliminary channel mixing is achieved through pointwise convolution, and the original input feature response is preserved by relying on residual connections. Subsequently, the complete channel mixing transformation is completed by sequentially passing through 1×1 convolution, GELU nonlinear activation function, and another 1×1 convolution.

[0038] S43. The channel-transformed features are fused with the current input feature residual of the dual-branch detail recovery block to output high-resolution detail recovery features.

[0039] The aforementioned detail compensation path complements the window attention module: window attention relies on spectral changes to guide information and optimize the local spatial-spectral correlation, while the bi-branch detail recovery block specifically compensates for the high-frequency texture, boundary, and small-scale structural information lost during the attention aggregation operation, thereby improving the detail restoration accuracy of the reconstructed hyperspectral image.

[0040] Furthermore, with network downsampling, the number of spatial feature block vectors in low-resolution layers is significantly reduced, making them suitable for larger-scale spatial interactions with lower computational cost. (See reference...) Figure 2 (b) The Differential Random Reconstruction Block (DRB) in this embodiment of the invention is used to perform the following operations: S51. Normalize the current input feature of the differential random reconstruction block (Layer Norm), project the spectral change guidance information to a feature space compatible with the current input feature (DifferPrior), and simultaneously send it to the feature block routing module (Patch Router). S52. The feature block routing module converts the normalized backbone features into a base token and converts the synchronously input spectral change guidance information into a differential feature block vector sequence. At the same time, it traverses each spatial feature block (base feature block and differential feature block) and generates a corresponding feature block-level injection mask based on the spectral statistical features such as the spectral amplitude and average fluctuation within each spatial feature block. To avoid uniformly injecting spectral variation guidance information into all feature blocks, this invention estimates prior reliability based on the variation amplitude, average variation level, or other statistical characteristics in each feature block and generates a feature block-level injection mask. This feature block-level injection mask controls the injection intensity of the differential feature block vector sequence (Different token) in different feature blocks; feature blocks with significant and reliable spectral variations receive higher injection weights, while smooth regions or regions with weak information receive lower injection weights.

[0041] S53. Input the output features of the feature block routing module into the differential grouped spectral attention module. Relying on the feature block-level injection mask differentiation to control the injection intensity of the differential feature block vector sequence, the grouped spectral attention features are obtained. S54. The grouped spectral attention features are fed into the Random Global PatchMixing module to complete the cross-block fusion of long-distance global features and obtain the hybrid feature sequence after global interaction enhancement (global interaction enhancement features). S55. The hybrid feature sequence after global interactive enhancement (global interactive enhancement feature) is fused with the current input feature of the differential random reconstruction block and then fed into the dual-branch detail recovery block (Detial Recorvery) operation to obtain low-resolution detail recovery features. S56. After normalizing the low-resolution detail recovery features again, the data is fed into a gated feedforward network (Gated FFN) to perform channel feature transformation. Finally, the output features of the gated feedforward network are residually fused with the low-resolution detail recovery features to obtain the output features of the differential random reconstruction block.

[0042] This Differential Random Reconstruction Block (DRB) is set at the low-resolution layer. Leveraging the small number of tokens in the low-resolution layer, it significantly reduces the computational overhead of global cross-regional spatial-spectral interaction. By injecting a mask at the feature block level, it adaptively filters spectral change guidance information, strengthening prior constraints only in regions with significant spectral changes, thus avoiding invalid prior interference with the reconstruction effect of uniform regions. Differential grouped spectral attention enables the modeling of grouped spectral subspace associations. Random global block mixing breaks the limitations of local neighborhoods and establishes long-distance spatial feature interaction. Combined with the dual-branch detail recovery block, it compensates for the texture edge details lost in attention operations. Under the premise of controlling the amount of computation, it simultaneously completes global spectral correction and fine spatial structure restoration, effectively improving the overall reconstruction accuracy of compressed hyperspectral images.

[0043] Furthermore, after obtaining the base token, the differential token, and the patch mask, this invention uses the Differential Grouped Spectral Attention module to perform grouped spectral interaction, such as... Figure 2 As shown in (e), the differential grouped spectral attention module performs the following operations: S61. After fusing the basic feature block vector sequence and the difference feature block vector sequence, the data is sent to the grouping module (Group) and divided into multiple grouped feature block sequences (Grouped tokens) according to the spectral channels. The grouped feature block sequences are then transformed by the linear transformation layer (Linear) to generate the second query feature, the second key feature, and the second value feature, respectively. S62. Calculate the original attention weights based on the second query features and the second key features, and then weight and fuse the original attention weights with the second value features to obtain the fused attention features. S63. Input the fused attention features and the differential feature block vector sequence into the Sigmoid gated fusion unit to complete the adaptive weighted fusion of the two features, obtain the fused spectral correlation features, and then introduce a feature block-level injection mask to perform differential scaling per feature block. S64. Perform residual fusion between the feature block-level injected mask-modulated features and the basic feature block vector sequence to output grouped spectral attention features.

[0044] This module adopts a channel grouping modeling strategy, which can complete the correlation modeling of independent spectral response subspaces separately, avoiding the massive redundant computation caused by full-channel attention. After the spectral interaction within the group is completed, the spectral prior information carried by the spectral prior is balanced by fusion of the attention interaction results and the original difference token with the help of Sigmoid gating. Then, relying on the feature block-level injection mask that represents the prior credibility of each feature block, the refined spectral change information is adaptively injected into the basic token in the form of residuals.

[0045] The mask-controlled residual injection mechanism enables differentiated spectral correction: regions with significant spectral changes and reliable prior information receive stronger spectral interaction constraints, while regions with weak spectral changes and low prior confidence retain the original basic reconstruction features, significantly improving the stability and reconstruction accuracy of low-resolution hierarchical global spatial spectrum recovery.

[0046] Furthermore, after completing the grouped spectral interaction, the present invention further performs random global feature block mixing at the low-resolution layer, such as... Figure 2 As shown in (g), the random global block mixing module performs the following operations.

[0047] S71. The feature block vector sequence of the grouped spectral attention features is added and fused with the pre-generated position encoding features element by element, and the original spatial coordinate information of each feature block is embedded in the grouped spectral attention features; the original two-dimensional spatial coordinate information is embedded in each feature block to prevent the loss of spatial position correlation during subsequent feature rearrangement.

[0048] S72. The feature block vector sequence with fused location information is sent to the Random Group module. The Random Group module generates a global random rearrangement index and an inverse rearrangement index corresponding to the global random rearrangement index based on a fixed random seed. S73. The feature block vector sequence of fused location information is shuffled and rearranged according to the global random rearrangement index to generate a global hybrid feature block vector sequence. In this embodiment, the global random rearrangement index remains fixed during the model building stage or a single reconstruction process, ensuring the repeatability of the rearrangement process; the global mixed feature block vector sequence enables feature blocks in the original image that are spatially distant and have no local neighborhood association to participate in subsequent unified global interactions.

[0049] S74. Perform layer normalization, multi-head self-attention operation, second layer normalization, and feedforward network (FFN) channel nonlinear transformation on the global mixed feature block vector sequence in sequence to obtain the global interactive enhancement feature sequence.

[0050] S75. Based on the inverse rearrangement index, restore the global interactive enhancement feature sequence to the original feature block arrangement order, and remap the feature block vector sequence after restoration of the arrangement order to a spatial feature map to obtain the global interactive enhancement feature.

[0051] After the global interactive operation is completed, the inverse rearrangement index, which corresponds one-to-one with the random rearrangement index, is called to restore the shuffled mixed feature sequence to the original token space arrangement order, and the one-dimensional token sequence is remapped into a two-dimensional spatial feature map. Because the rearrangement and inverse rearrangement operations are precisely matched, only the information interaction logic of the features is changed, without destroying the original spatial position correspondence of each feature block, thus ensuring the integrity of the feature space structure.

[0052] This invention deploys random global feature blocks in a mixed manner at a low-resolution level, which can significantly reduce the number of tokens participating in global attention calculation, thereby significantly reducing model computation overhead and hardware computing power consumption. At the same time, the random rearrangement mechanism with fixed seeds breaks the interaction limitations of the fixed local neighborhood of traditional window attention, effectively promotes the exchange of spatial spectral feature information in non-neighboring regions of the image, makes up for the perception blind spots of local interaction, and further improves the consistency of global spatial structure and spectral features of hyperspectral images.

[0053] The network model of this invention operates in two modes: a training phase and an inference phase. During the training phase, the network input includes hyperspectral compressed measurement data, the corresponding encoding and dispersion parameters of the imaging system, and real hyperspectral image labels. The network first completes initial reconstruction based on the compressed measurements and system parameters, generating intermediate hyperspectral reconstruction features and extracting spectral change guidance information. Subsequently, following the scale-adaptive local-global collaborative recovery process of this invention, it progressively performs spectral prior-guided local detail recovery, random global cross-block interactive fusion, and multi-scale feature cross-layer fusion correction, ultimately outputting the predicted hyperspectral reconstruction result. During training, various loss functions can be used individually or in combination to constitute joint loss constraints, including but not limited to L1 loss, mean squared error loss, Charbonnier loss, spectral angle loss, and structural similarity loss. These can simultaneously constrain image spatial detail similarity and spectral curve fidelity, improving the model's spatial-spectral joint reconstruction performance.

[0054] During the model inference stage, only hyperspectral compressed measurement data and necessary imaging system parameters need to be input. Without the participation of real labels, the reconstruction network can complete local spatial spectral feature recovery, long-distance global feature interaction, multi-scale jump feature fusion, and final feature mapping output step by step, quickly obtaining a high-precision, high-fidelity complete hyperspectral image.

[0055] The differential stochastic reconstruction and scale-adaptive collaborative recovery architecture proposed in this invention has strong versatility and portability. It can be embedded in each iteration of the deep unfolded reconstruction network as an iterative prior constraint module to improve the reconstruction accuracy and stability of the unfolded network. It can also be directly used as an independent prior recovery unit and integrated into various mainstream hyperspectral reconstruction frameworks such as end-to-end hyperspectral reconstruction networks, U-shaped encoding and decoding networks, Transformer global modeling networks, and convolution-attention hybrid networks. It has a wide range of applications and strong module compatibility.

[0056] Furthermore, comparative experiments were conducted to verify the advantages of the present invention. Compared with the scheme of repeatedly using the same convolutional block or attention block at all scales, the present invention allocates recovery tasks according to feature scales, so that high-resolution layers focus on protecting textures, boundaries and subtle spectral transitions, and low-resolution layers focus on performing long-range structural modeling, thus avoiding capability conflicts caused by a single module simultaneously undertaking local and global tasks.

[0057] Compared to ordinary window attention, this invention utilizes spectral variation to guide information in the construction of query features and key features, making local correlation estimation more focused on regions with similar band variation patterns; meanwhile, the value features still come from the original input, which can reduce the spatial detail damage caused by prior over-dominance of aggregated content.

[0058] Compared to solutions that rely solely on attention aggregation, this invention adds a dual-branch detail compensation path, which restores textures, edges, and small structures that may be weakened during the attention process by local refinement and extended neighborhood modeling.

[0059] Compared to the approach of uniformly injecting spectral priors into the entire region, this invention performs selective routing based on feature block-level reliability at the low-resolution layer and models interactions within the grouped spectral subspace, which can suppress weak priors or unreliable fluctuations while reducing redundant computations in full-channel interactions.

[0060] Compared to schemes that directly perform global attention on high-resolution features, this invention places global modeling in a low-resolution layer with fewer tokens and promotes information exchange between non-neighboring regions through positional encoding, random rearrangement, global interaction, and inverse rearrangement, thereby reducing the cost of global modeling while maintaining the recoverability of spatial order.

[0061] The comparative results of the stepwise component ablation setting used in this invention are shown in Table 1. The baseline results of the ordinary depth unfolding framework combined with a simple U-shaped network are 35.74 dB PSNR and 0.953 SSIM. After introducing high-resolution differential window attention, the results are improved to 38.52 dB and 0.971. Further addition of a low-resolution differential random reconstruction structure achieves 38.68 dB and 0.972. Adding detail restoration achieves 38.76 dB and 0.973. Further addition of global mixing achieves 38.91 dB and 0.973. The above results are from stepwise combined ablation, indicating that the scale-adaptive local-global restoration path can continuously improve hyperspectral reconstruction performance.

[0062] Table 1. Stepwise ablation results of the scale-adaptive local-global recovery component: This invention proposes a hyperspectral compressed sensing reconstruction method based on scale-adaptive local-global collaborative recovery, which has at least the following advantages: 1. This invention allocates restoration tasks based on the differential feature space resolution, establishing a multi-scale collaborative reconstruction architecture. It focuses on restoring fine details such as local textures and edges at high-resolution layers, while conducting long-range global spatial structure modeling at low-resolution and bottleneck layers, achieving hierarchical collaboration between local fine correction and global structural constraints. Simultaneously, relying on the overall network layout of a U-shaped encoder-bottleneck-decoder, it hierarchically configures local and global restoration units, and combines cross-scale skip fusion to compensate for the loss of upsampling details. This highly adapts to the complex degradation and strong spatial-spectral coupling of hyperspectral compressed sensing reconstruction, significantly improving overall reconstruction accuracy and structural integrity.

[0063] 2. This invention introduces spectral change guidance information to pre-enhance features, applying this guidance information to the correlation estimation process between query features and key features, and utilizing spectral priors to constrain the spatial spectral association modeling logic. Simultaneously, the retained value features are entirely derived from the original input features, achieving decoupling between the sources of attention "correlation estimation" and "content aggregation." Unlike traditional unified feature modeling methods, this mechanism accurately relies on real spectral change patterns to guide weight allocation, effectively avoiding excessive interference from prior information with the original feature content, and improving the rationality of attention modeling and spectral fidelity.

[0064] 3. After the local window attention interaction is completed, this invention generates a unique prior perception gating by combining attention output features and spectral change guidance information. This gating can adaptively and selectively enhance effective local attention responses and suppress ineffective redundant responses based on the reliability of spectral changes in different regions. This achieves dynamic modulation of local spatial-spectral interaction results, effectively solving the problems of poor adaptability and overfitting or underfitting in fixed-weight attention modeling, and significantly improving the stability of local feature recovery in complex scenarios.

[0065] 4. To address the shortcomings of traditional window attention aggregation operations in weakening texture, edges, and small-scale fine structures, this invention constructs a dual-branch detail restoration structure with parallel local refinement and extended neighborhood branches. The local refinement branch relies on lightweight local operators to repair fine texture and edge details, while the extended neighborhood branch expands the receptive field and supplements long-distance neighborhood contextual information through dilated convolution. After fusing the features from both branches with the original input features, feature reconstruction is completed through point-by-point mapping and residual connections, accurately compensating for high-frequency details lost due to attention and significantly improving the spatial clarity of the reconstructed image.

[0066] 5. At the low-resolution global modeling level, this invention converts the reconstructed backbone features and spectral change guidance information into basic tokens and differential tokens, respectively, to achieve decoupling representation of the original reconstructed features and spectral prior features. At the same time, based on the real spectral statistical characteristics such as the spectral amplitude and fluctuation degree of each feature block, a feature block-level prior injection mask is generated to accurately quantify the spectral prior reliability of each feature block, providing a reliable basis for subsequent differentiated prior injection and spectral interactive control, and avoiding reconstruction deviation caused by the global unified prior from the source.

[0067] 6. This invention designs a differential grouping spectral attention mechanism, which divides the differentially enhanced tokens into multiple independent spectral subspaces along the spectral channels. Each group independently completes band association learning and spectral interaction modeling, abandoning the massive redundant computation of traditional full-channel attention. While accurately capturing subtle spectral changes and preserving fine band association features, it significantly reduces the computational cost of high-dimensional spectral modeling. Simultaneously, relying on a reliability mask, the refined spectral change information is adaptively injected into the basic token in the form of residuals, strengthening the correction of reliable spectral regions and preserving the original features of weakly changing regions, balancing spectral reconstruction accuracy and model lightweight performance.

[0068] 7. This invention embeds spatial location information into feature block tokens at a computationally friendly, low-resolution level. It achieves reproducible and reversible random rearrangement through a fixed random seed, breaking the limitations of traditional window attention which relies on fixed local neighborhood interactions. This encourages spatially non-neighboring feature blocks to participate in globally unified interactions. After the global interaction is complete, the original token spatial order is restored through inverse rearrangement indexing. Without disrupting the spatial correspondence of features, this invention addresses the shortcomings of missing long-range dependencies in local modeling, achieving global structural optimization with extremely low computational overhead, balancing modeling capability and computational efficiency.

[0069] 8. The scale-adaptive local-global collaborative restoration scheme proposed in this invention is a pluggable and universal module with extremely strong compatibility. It can be flexibly embedded into various mainstream hyperspectral compressed sensing reconstruction frameworks such as deep unfolding networks, end-to-end reconstruction networks, convolutional networks, and Transformer networks, and is compatible with multiple reconstruction paradigms. It can also be widely applied to technical scenarios such as coded aperture snapshot spectral imaging, hyperspectral / multispectral image restoration, and reconstruction of real measurement data. It can be deployed on various electronic devices and computer-readable storage media, demonstrating extremely high engineering practical value.

[0070] It will be understood by those skilled in the art that, unless otherwise defined, all terms used herein (including technical and scientific terms) have the same meaning as commonly understood by one of ordinary skill in the art to which this invention pertains. It should also be understood that terms such as those defined in general dictionaries should be understood to have the same meaning as in the context of the prior art and should not be interpreted in an idealized or overly formal sense unless specifically defined.

[0071] For the sake of simplicity, the method embodiments are described as a series of actions. However, those skilled in the art should understand that the embodiments of the present invention are not limited to the described order of actions, because according to the embodiments of the present invention, some steps can be performed in other orders or simultaneously. Furthermore, those skilled in the art should also understand that the embodiments described in the specification are preferred embodiments, and the actions involved are not necessarily essential to the embodiments of the present invention.

[0072] Furthermore, those skilled in the art will understand that although some embodiments herein include certain features included in other embodiments but not others, combinations of features from different embodiments are intended to be within the scope of the invention and form different embodiments. For example, any of the claimed embodiments can be used in any combination.

[0073] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention, and not to limit them; although the present invention has been described in detail with reference to the foregoing embodiments, those skilled in the art should understand that modifications can still be made to the technical solutions described in the foregoing embodiments, or equivalent substitutions can be made to some of the technical features; and these modifications or substitutions do not cause the essence of the corresponding technical solutions to deviate from the spirit and scope of the technical solutions of the embodiments of the present invention.

Claims

1. A hyperspectral compressed sensing reconstruction method based on scale-adaptive local-global collaborative recovery, characterized in that, The method includes: Obtain intermediate hyperspectral estimates and spectral change guidance information that matches the spatial dimension of the intermediate hyperspectral estimates; fuse the intermediate hyperspectral estimates and spectral change guidance information, perform feature embedding, obtain embedded features, and send them to the encoder; The encoder performs at least one level of differential window attention block operation and downsampling operation on the embedded features to combine spectral change guidance information to complete local correlation estimation and local detail recovery, and outputs low-resolution encoded features; it performs at least one level of differential random reconstruction block operation and downsampling operation on the low-resolution encoded features, selectively injects spectral change guidance information to complete global spectral and spatial structure correction, and outputs bottleneck features with the smallest size. The bottleneck features are aggregated globally using differential random reconstruction blocks to perform long-range information aggregation. After completing the global structural consistency modeling of the entire graph, the data is sent to the decoder. The decoder recovers the feature space resolution step by step through multi-level upsampling. It uses hierarchically symmetrical configuration of differential random reconstruction blocks and differential window attention blocks corresponding to the encoder to achieve global structure optimization and local detail refinement on the decoding side, respectively. In each reconstruction stage, the decoder fuses the encoder's encoded features at the same scale through cross-layer skip connections to make up for the multi-scale spatial and spectral detail information lost during the upsampling process. The high-resolution features output by the decoder are fed into the mapping layer to complete the dimensionality transformation, and the output is a complete three-dimensional hyperspectral reconstructed image that matches the original input size and number of spectral channels.

2. The method according to claim 1, characterized in that, The acquisition of intermediate hyperspectral estimates and spectral change guidance information matching the spatial dimension of the intermediate hyperspectral estimates includes: Initial spectral reconstruction is performed based on the two-dimensional compressed measurement, system coding parameters, and dispersion parameters output by the hyperspectral compressed sensing imaging system to obtain an intermediate hyperspectral estimate with complete spectral channel dimensions. Feature extraction is performed on the band variation law and spatial spectral type difference of the intermediate hyperspectral estimate to generate spectral change guidance information that corresponds one-to-one with the spatial location of the intermediate hyperspectral estimate.

3. The method according to claim 1, characterized in that, The difference window attention block performs the following operations: The original features input to the differential window attention block are normalized and then fed into the differential prior guided window attention block for computation, outputting high-resolution attention features; The high-resolution attention features and the original features input to the differential window attention block are fused and then fed into the dual-branch detail recovery block operation to compensate for image texture and edge details, thereby obtaining high-resolution detail recovery features; After normalizing the high-resolution detail recovery features again, they are fed into a gated feedforward network for computation. The output features of the gated feedforward network are then residually fused with the high-resolution detail recovery features to obtain the output features of the differential window attention block.

4. The method according to claim 3, characterized in that, The differential prior guidance window attention block performs the following operations: The spectral change guidance information is projected onto a feature space compatible with the current input features, and the projected spectral change guidance information is fused with the current input features to obtain the guidance enhancement representation; The guided enhancement representation generates the first query feature and the first key feature, and generates the first value feature only from the current input feature, so that the spectral change guidance information participates in the local correlation estimation of the feature; Divide the first query feature, the first key feature, and the first value feature into multiple local windows, perform window attention interaction, and output the local attention features; Based on the local attention features and the spectral change guidance information after projection, a priori perception gating is generated. The priori perception gating is used to selectively modulate the local attention output position by position, channel by channel, or group by group, thereby enhancing the attention response in reliable spectral change regions and suppressing the response in unreliable regions, and finally outputting high-resolution attention features.

5. The method according to claim 3, characterized in that, The dual-branch detail recovery block performs the following operations: The current input features of the dual-branch detail recovery block are fed into the parallel local refinement branch and the extended neighborhood branch in two paths. After fusing the output features of the local refinement branch and the extended neighborhood branch with the current input features of the dual-branch detail recovery block, a channel blending transformation is performed. The channel-transformed features are fused with the current input feature residual of the dual-branch detail recovery block to output high-resolution detail recovery features.

6. The method according to claim 1, characterized in that, The differential random reconstruction block is used to perform the following operations: The current input features of the differential random reconstruction block are normalized, the spectral change guidance information is projected onto a feature space compatible with the current input features, and simultaneously sent to the feature block routing module. The feature block routing module converts the normalized backbone features into a sequence of basic feature block vectors and converts the synchronously input spectral change guidance information into a sequence of differential feature block vectors. At the same time, it traverses each spatial feature block and generates a corresponding feature block-level injection mask based on the spectral statistical features such as the spectral amplitude and average fluctuation within each spatial feature block. The output features of the feature block routing module are input into the differential grouped spectral attention module. The injection intensity of the differential feature block vector sequence is controlled by the feature block-level injection mask differentiation to obtain the grouped spectral attention features. The grouped spectral attention features are fed into the random global block fusion module to complete the cross-block fusion of long-distance global features and obtain the global interactive enhanced features after global interactive enhancement. The global interactive enhancement features are fused with the current input features of the differential random reconstruction block and then fed into the dual-branch detail recovery block operation to obtain low-resolution detail recovery features; After normalizing the low-resolution detail recovery features again, they are fed into the gated feedforward network for computation to complete the channel feature transformation. Finally, the output features of the gated feedforward network are residually fused with the low-resolution detail recovery features to obtain the output features of the differential random reconstruction block.

7. The method according to claim 6, characterized in that, The differential grouped spectral attention module performs the following operations: The basic feature block vector sequence and the differential feature block vector sequence are fused and then sent to the grouping module, which divides them into multiple groups of grouped feature block sequences according to spectral channels. The grouped feature block sequence is transformed by a linear transformation layer to generate the second query feature, the second key feature, and the second value feature, respectively. The original attention weights are calculated based on the second query features and the second key features. The original attention weights are then weighted and fused with the second value features to obtain the fused attention features. The fused attention features and the differential feature block vector sequence are input into the gated fusion unit to complete the adaptive weighted fusion of the two features, and the fused spectral correlation features are obtained. Then, a feature block-level injection mask is introduced to perform differential scaling for each feature block. The features after feature block-level injection mask modulation are residually fused with the basic feature block vector sequence to output grouped spectral attention features.

8. The method according to claim 6, characterized in that, The random global block mixing module performs the following operations: The feature block vector sequence of the grouped spectral attention features is added and fused with the pre-generated position encoding features element by element, and the original spatial coordinate information of each feature block is embedded in the grouped spectral attention features; The feature block vector sequence with fused location information is fed into the random grouping module, which generates a global random rearrangement index and an inverse rearrangement index corresponding to the global random rearrangement index based on a fixed random seed. The feature block vector sequence with fused location information is shuffled and rearranged according to the global random rearrangement index to generate a global hybrid feature block vector sequence. The global hybrid feature block vector sequence is subjected to layer normalization, multi-head self-attention operation, second layer normalization, and feedforward network channel nonlinear transformation in sequence to obtain the global interactive enhancement feature sequence. Based on the inverse rearrangement index, the global interactive enhancement feature sequence is restored to the original feature block arrangement order, and the feature block vector sequence after restoration of the arrangement order is remapped into a spatial feature map to obtain the global interactive enhancement feature.

9. The method according to claim 1, characterized in that, Each reconstruction stage of the decoder fuses the encoder's same-scale encoded features through cross-layer skip connections, including: Obtain the encoder's output at the same scale level, and the encoded features after processing by differential window attention blocks or differential random reconstruction blocks; Obtain the decoding features output from the decoder's local upsampling module and after resolution upsampling; The encoder's same-scale encoded features and the decoder's local upsampled decoded features are input into the channel splicing unit, and the two features are spliced ​​along the channel dimension; The hybrid features obtained by channel splicing are fed into the fusion module, and convolution operation is used to complete channel mixing, thereby realizing the fusion of shallow spatial detail features of the encoder and deep reconstruction features of the decoder. The output features of the fusion module are fed into the differential window attention block or differential random reconstruction block corresponding to the current level of the decoder to complete the feature reconstruction of this level.

Images