Dual-path super-resolution image reconstruction method fusing state space and attention mechanism

Abstract

The application discloses a kind of fusion state space and attention mechanism's dual-path super-resolution image reconstruction method and system, method includes: acquisition structure light microscopic imaging system multiple frames low signal-to-noise ratio image sequence, splicing into multi-channel input tensor, preliminary extraction feature by shallow convolution network, mapping to high-dimensional space generates initial shallow feature map.It is input Mamba-Transformer hybrid dual-path module, and the global spatial dependence is captured by the attention mechanism of Transformer to restore sample macrostructure, and Mamba state space model efficiently captures local long-distance continuous dependence, strengthens subtle structure;Fusion two-way features obtains hybrid deep features.Structure light matching physical prior pattern is constructed as mask, and optical physical constraint is applied to features, and artifact is suppressed, and physical constraint features are obtained.Lightweight reconstruction network is sent, and high-resolution image is restored by relying on compressed convolution and sub-pixel convolution up-sampling, and super-resolution reconstruction is completed.Provide reliable technical support for microscopic image super-resolution reconstruction.

Description

Technical Field

[0001] This invention belongs to the field of deep learning image processing technology, and more specifically, relates to a dual-path super-resolution image reconstruction method and system that integrates state space and attention mechanism. Background Technology

[0002] Structured light microscopy is an important super-resolution microscopy technique. It acquires high-frequency information from samples through specific illumination patterns, thereby overcoming the diffraction limit of traditional optical microscopes and achieving higher-resolution observation of cell structures. Compared to some super-resolution techniques that require strong illumination or complex labeling conditions, structured light microscopy offers advantages such as fast imaging speed, relatively low phototoxicity, and suitability for dynamic observation of live cells. Therefore, it is widely used in the study of the cytoskeleton, organelle morphological changes, and intracellular dynamic processes.

[0003] However, the final image quality of structured light microscopy is highly dependent on subsequent reconstruction algorithms. The original acquired images often contain noise, undersampling, and spectral aliasing, especially under low illumination or long-term live-cell imaging conditions, where signal quality further degrades, making high-precision reconstruction even more difficult. Traditional reconstruction methods typically rely on relatively explicit imaging models and parameter estimations, achieving good reconstruction results under ideal conditions. However, when image noise is high, illumination parameter estimation is inaccurate, or the sample structure is complex, artifacts, streaks, and detail distortion are easily generated, resulting in limited robustness.

[0004] In recent years, deep learning methods have provided new insights for structured light microscopy image reconstruction. By learning the mapping relationship between low-quality and high-quality images from large amounts of data, deep learning models have shown great potential in denoising, detail restoration, and reconstruction speed. Compared with traditional reconstruction methods, deep learning methods can more flexibly adapt to complex image degradation processes and are expected to achieve rapid inference, thus meeting the needs of large-scale data processing and real-time imaging analysis. However, existing deep learning methods typically require high-quality labeled images as training targets, but such data is difficult and costly to obtain in actual microscopic imaging. Furthermore, if the model lacks constraints on the physical processes of imaging, it may introduce inaccurate structural information during reconstruction, affecting the reliability of the results. Therefore, how to reduce dependence on labeled data while fully utilizing the inherent physical laws of structured light microscopy, and balancing reconstruction accuracy, stability, and speed, remains a pressing problem to be solved in this field. Summary of the Invention

[0005] This invention aims to address the problems of low signal-to-noise ratio and easy loss of subtle details in raw images from structured light microscopy, overcoming the shortcomings of existing reconstruction methods that cannot simultaneously capture global structure and local details, have high computational complexity, and are prone to non-physical artifacts. It constructs a dual-path feature extraction architecture by fusing a state-space model and an attention mechanism, introducing prior knowledge of structured light optics for constraints, and combining this with a lightweight network design. This reduces the number of parameters while ensuring reconstruction accuracy and detail integrity, achieving high-quality super-resolution reconstruction of microscopic images. Furthermore, this invention employs an asymmetric capacity allocation strategy, concentrating the main model parameters before the physical modulation stage (including the local feature module and the Mamba-Transformer hybrid module), while using only a single 3×3 convolutional layer for feature refinement after the physical modulation stage. This significantly reduces the number of parameters and inference latency while maintaining reconstruction accuracy.

[0006] To address the aforementioned deficiencies or improvement needs of existing technologies, as a first aspect of this invention, the present invention provides a dual-path super-resolution image reconstruction method that integrates state space and attention mechanisms, comprising: S1. Acquire a sequence of multiple low signal-to-noise ratio original images acquired by a structured light microscopy imaging system, and stitch them together into a multi-channel input tensor; perform preliminary feature extraction on the input tensor through a shallow convolutional network, map the image data to a high-dimensional feature space, and generate an initial shallow feature map; S2. The initial shallow feature map is input into the backbone structure, which includes a Mamba-Transformer hybrid module. The hybrid module contains two parallel processing paths: the first path uses a Transformer-based attention mechanism, utilizing a multi-head self-attention module to capture the global spatial dependencies of image features to reconstruct the overall macroscopic structure of cells or samples; the second path uses a Mamba-based state-space model, utilizing its linear complexity sequence modeling capability to efficiently capture long-distance local continuous dependencies through a scanning mechanism to enhance the continuity of weak signals and fine structures; the features output from the two paths are concatenated and fused to generate a hybrid deep feature that combines global perception and local details. S3. Construct a physical prior pattern that matches the structured light illumination mode, wherein the physical prior pattern represents the periodic modulation information of the structured light; use the physical prior pattern as a modulation mask to perform element-wise multiplication or affine transformation on the mixed deep features to introduce physical constraints of optical imaging in the feature space, suppress the generation of non-physical artifacts, and output physical constraint features. S4. Input the physical constraint features into a lightweight reconstruction network, which includes compressed main channel convolutional layers and sub-pixel convolutional layers; reduce the number of parameters by reducing the number of main channels, and use sub-pixel convolutional layers to perform upsampling operations to restore the low-resolution feature map to the high-resolution space, and finally output the super-resolution reconstructed image.

[0007] Furthermore, the specific process of preliminary feature extraction in S1 includes: Acquire raw image sequences acquired by a structured light microscopy system under multiple illumination directions and multiple phases; The original image sequence is stitched together along the channel dimension to construct a multi-channel input tensor; The multi-channel input tensor is convolved by a 3×3 convolutional layer containing a preset number of convolutional kernels to map the image data to a high-dimensional feature space and generate an initial shallow feature map.

[0008] Furthermore, the backbone structure in S2 includes a local feature module, a Mamba-Transformer hybrid module, and a convolution module; wherein, the local feature module is used to capture edge, texture, and detail information in the cell structure; the Mamba-Transformer hybrid module is used to enhance the model's ability to understand large-scale structural relationships, so that the network pays attention to both local pixel changes and the overall continuity of the cell structure.

[0009] Furthermore, in S2, the first path employs a Transformer-based attention mechanism, specifically constructed as a simplified multi-head self-attention module, whose processing includes: The input feature map is subjected to layer normalization, and then the local spatial feature representation is enhanced by depthwise separable convolution. The query tensor, key tensor and value tensor are generated by linear projection layer, and the number of attention heads is set to 8. Within each attention head, the dot product of the query tensor and the transpose of the key tensor is calculated and divided by a scaling factor to obtain the attention score matrix; the attention score matrix is ​​then Softmax normalized to obtain the attention weights; the value tensors are weighted and aggregated using the attention weights to generate intermediate features containing global context information. The output of the multi-head self-attention is nonlinearly transformed through a feedforward network with a bottleneck ratio of 2. The feedforward network contains two linear layers and a GELU activation function. The output of the feedforward network is added to the input features through residual connections to output the global enhancement features of the first path.

[0010] Furthermore, in S2, the second path adopts a state-space model based on Mamba, specifically including: The initial shallow feature map is scanned bidirectionally along the horizontal and vertical spatial dimensions to generate forward and backward sequences respectively. The forward sequence arranges feature pixels in the order of left to right and top to bottom, while the backward sequence arranges feature pixels in the order of right to left and bottom to top, so as to fully capture the long-range dependencies of image features in both directions. The forward and backward sequences are respectively input into the state space model. The state space model models the sequence through learnable parameter matrices A, B, C, and D. Parameter matrix A is used to control the decay rate of the state, parameter matrices B and C are used to realize the linear mapping between input and output, and parameter matrix D is used to introduce residual connections. A selective scanning mechanism is introduced to dynamically adjust the values ​​of parameter matrices B and C based on the local features of the input sequence, enabling the model to adaptively focus on key continuous regions across cell structures and suppress interference from irrelevant background. The state dimension of the state space model is set to a preset value less than 64 to reduce the number of model parameters and computational complexity; the forward and backward sequences output by the state space model are recombined to restore a two-dimensional feature map with the same spatial size as the input feature map. The recombined forward 2D feature map and backward 2D feature map are added element-wise to obtain bidirectional fused long-range dependency features. The long-range dependency features are channel-adjusted through a 1×1 convolutional layer and residually connected with the input initial shallow feature map to output the long-range enhanced features of the second path.

[0011] Furthermore, in S2, the Mamba-Transformer hybrid module further includes a feature fusion unit; the feature fusion unit is configured as follows: The global context feature map output by the first path and the long-range enhancement feature map output by the second path are concatenated in the channel dimension to generate a high-dimensional concatenated feature map. The number of channels in the high-dimensional concatenated feature map is the sum of the number of channels output by the first path and the second path, which is used to preserve the complementary characteristics of global structural information and local continuous details. A 1×1 convolution operation is performed on the high-dimensional spliced ​​feature map to compress the number of channels from the spliced ​​high dimension back to the number of channels in the original input feature map, so that the output dimension remains the original number of channels C; the 1×1 convolution operation also realizes channel shuffling, promoting information interaction between features of different paths; The output of a 1×1 convolution is the final blended deep feature.

[0012] Furthermore, in S3, the process of constructing and modulating the physical prior pattern specifically includes: Based on the illumination parameters of the structured light microscopy system, a sinusoidal fringe pattern corresponding to the original input image sequence is generated as a physical prior pattern. The physical prior image is upsampled to the same spatial resolution as the hybrid deep features using bilinear interpolation. Affine transformations are performed on physical prior maps using learnable scaling and translation factors to adapt them to the distribution of the feature space. The transformed physical prior image is used as a modulation mask and multiplied element-wise with the hybrid deep features to explicitly introduce physical constraints of optical imaging into the feature space. Furthermore, in S4, the lightweight reconstruction network adopts a main channel compression strategy, specifically including: The number of main channels in the feature space of the lightweight reconstruction network is set to a preset value that satisfies the double divisibility constraint. The double divisibility constraint includes: the number of main channels is divisible by the scaling factor of the pixel recombination operation in the network and divisible by the number of attention heads of the attention mechanism branch.

[0013] As a second aspect of the present invention, a dual-path super-resolution image reconstruction system integrating state space and attention mechanisms is also provided, comprising: The multi-channel feature mapping unit is used to acquire multiple frames of low signal-to-noise ratio original image sequences acquired by the structured light microscopy imaging system, and stitch them together into a multi-channel input tensor; the input tensor is used to perform preliminary feature extraction through a shallow convolutional network, and the image data is mapped to a high-dimensional feature space to generate an initial shallow feature map. A dual-path feature fusion unit is used to input the initial shallow feature map into the backbone structure, wherein the backbone structure includes a Mamba-Transformer hybrid module. The hybrid module contains two parallel processing paths: the first path adopts a Transformer-based attention mechanism, using a multi-head self-attention module to capture the global spatial dependencies of image features to reconstruct the overall macroscopic structure of cells or samples; the second path adopts a Mamba-based state-space model, utilizing its linear complexity sequence modeling capability to efficiently capture long-distance local continuous dependencies through a scanning mechanism to enhance the continuity of weak signals and fine structures; the features output from the two paths are concatenated and fused to generate a hybrid deep feature that combines global perception and local details. A physical prior modulation unit is used to construct a physical prior pattern that matches the structured light illumination mode. The physical prior pattern represents the periodic modulation information of the structured light. The physical prior pattern is used as a modulation mask to perform element-wise multiplication or affine transformation on the mixed deep features to introduce physical constraints of optical imaging in the feature space, suppress the generation of non-physical artifacts, and output physical constraint features. The lightweight super-resolution reconstruction unit is used to input the physical constraint features into the lightweight reconstruction network, which includes compressed main channel convolutional layers and sub-pixel convolutional layers. By reducing the number of main channels, the number of parameters is reduced, and upsampling is performed using sub-pixel convolutional layers to restore the low-resolution feature map to the high-resolution space, and finally outputs the super-resolution reconstructed image.

[0014] As a third aspect of the invention, a computer-readable storage medium is also provided, on which a computer program is stored, which is executed by a processor as described in any one of the claims, a dual-path super-resolution image reconstruction method that integrates state space and attention mechanisms.

[0015] In summary, compared with the prior art, the above-described technical solutions conceived by this invention can achieve the following beneficial effects: 1. The dual-path super-resolution image reconstruction method of the present invention, which integrates state space and attention mechanisms, acquires multiple frames of low signal-to-noise ratio original image sequences from a structured light microscopy imaging system. The image sequences are then concatenated along the channel dimension to construct a multi-channel input tensor. A shallow convolutional network is then used to perform preliminary feature extraction on the input tensor, mapping the original pixel data of the image to a high-dimensional feature space and generating an initial shallow feature map. This method unifies the input format of image data under multiple illumination directions and phases, standardizes the data dimensions required for network computation, and filters out low-level noise interference in the original image through convolutional operations, retaining basic visual features such as image edges and textures. This provides standardized feature input for subsequent deep feature mining of the backbone network, avoiding feature disorder caused by direct input of multiple heterogeneous images. The basic feature mapping method solidifies the data foundation for subsequent dual-path feature processing, ensuring the stability and standardization of the initial stage of the overall reconstruction process.

[0016] 2. The dual-path super-resolution image reconstruction method of this invention, which integrates state space and attention mechanisms, connects the initial shallow feature map to a Mamba-Transformer hybrid module containing parallel dual paths. It utilizes the Transformer attention mechanism to build a multi-head self-attention structure, capturing the global spatial dependencies of image features to achieve complete reconstruction of the overall macroscopic structure of cells and biological samples. Simultaneously, it employs the Mamba state space model, leveraging its linear complexity sequence modeling capabilities and spatial scanning mechanism to efficiently capture long-distance local continuous dependencies in the image, enhancing the continuity of microscopic weak signals and fine tissue structures. The output features of the two paths are then processed through channel splicing and fusion. This parallel dual-path design can simultaneously consider both the global structural layout and local detail relationships of the image, overcoming the limitations of single feature extraction modes that can only focus on single-dimensional feature mining. It enriches the dimensionality of feature information while reasonably controlling the computational scale, allowing the extracted deep features to possess both global perception and local detail representation capabilities.

[0017] 3. The dual-path super-resolution image reconstruction method of this invention, which integrates state space and attention mechanisms, constructs a physical prior pattern that fits the structured light illumination mode. A modulation mask is formed based on the periodic modulation information carried by the pattern. Element-wise multiplication and affine transformation operations are performed on the mixed deep features, introducing inherent physical constraints of optical imaging into the feature space to suppress the generation of non-physical artifacts during reconstruction. The physically constrained features are then input into a lightweight reconstruction network. The model parameters are reduced by compressing the main channel convolutional layers, and sub-pixel convolutional layers are used to complete the upsampling and restoration of low-resolution feature maps. This approach can constrain the feature evolution process according to the physical laws of structured light imaging, improving the realism of the reconstructed image structure. The lightweight network architecture effectively reduces model computational overhead and resource consumption. Sub-pixel convolutions can complete the mapping and transformation of resolution dimensions, outputting a super-resolution reconstructed image that meets the needs of microscopic observation while balancing computational cost and reconstruction quality. Attached Figure Description

[0018] Figure 1 This is a flowchart of a dual-path super-resolution image reconstruction method that integrates state space and attention mechanism according to an embodiment of the present invention. Figure 2-4 This is a schematic diagram of the deep learning network structure framework according to an embodiment of the present invention; Figure 5 This is a schematic diagram of the system units in an embodiment of the present invention. Detailed Implementation

[0019] To make the objectives, technical solutions, and advantages of this invention clearer, the invention will be further described in detail below with reference to the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are merely illustrative and not intended to limit the invention. Furthermore, the technical features involved in the various embodiments of this invention described below can be combined with each other as long as they do not conflict with each other.

[0020] Example

[0021] Please refer to Figure 1 This embodiment 1 provides a dual-path super-resolution image reconstruction method that integrates state space and attention mechanisms, including: S1. Acquire a sequence of multiple low signal-to-noise ratio original images acquired by a structured light microscopy imaging system, and stitch them together into a multi-channel input tensor; perform preliminary feature extraction on the input tensor through a shallow convolutional network, map the image data to a high-dimensional feature space, and generate an initial shallow feature map; S2. The initial shallow feature map is input into the backbone structure, which includes a Mamba-Transformer hybrid module. The hybrid module contains two parallel processing paths: the first path uses a Transformer-based attention mechanism, utilizing a multi-head self-attention module to capture the global spatial dependencies of image features to reconstruct the overall macroscopic structure of cells or samples; the second path uses a Mamba-based state-space model, utilizing its linear complexity sequence modeling capability to efficiently capture long-distance local continuous dependencies through a scanning mechanism to enhance the continuity of weak signals and fine structures; the features output from the two paths are concatenated and fused to generate a hybrid deep feature that combines global perception and local details. S3. Construct a physical prior pattern that matches the structured light illumination mode, wherein the physical prior pattern represents the periodic modulation information of the structured light; use the physical prior pattern as a modulation mask to perform element-wise multiplication or affine transformation on the mixed deep features to introduce physical constraints of optical imaging in the feature space, suppress the generation of non-physical artifacts, and output physical constraint features. S4. Input the physical constraint features into a lightweight reconstruction network, which includes compressed main channel convolutional layers and sub-pixel convolutional layers; reduce the number of parameters by reducing the number of main channels, and use sub-pixel convolutional layers to perform upsampling operations to restore the low-resolution feature map to the high-resolution space, and finally output the super-resolution reconstructed image.

[0022] Structured light microscopy (SIM) is a super-resolution imaging method based on modulated illumination. Traditional wide-field microscopes are limited by the optical diffraction limit, only able to record spatial information in the sample below the system cutoff frequency, with high-frequency details being filtered out by the imaging system. SIM projects a fringe light field with specific direction and phase onto the sample, causing the high-frequency structures in the sample to mix with the illumination fringes, thereby shifting high-frequency information that was originally inaccessible to direct observation to the low-frequency range that the microscope can receive. Subsequently, by using multiple original images acquired under different illumination conditions, this frequency information is separated and reconstructed to obtain a higher-resolution reconstructed image.

[0023] For a two-dimensional linear SIM, the illumination pattern in the m-th direction and n-th phase can be represented as: Where m represents the direction index of the illumination fringe; n represents the phase index of the illumination fringe; This represents the intensity distribution of the illumination pattern projected onto the sample in the m-th direction and n-th phase; r represents the spatial position. Average lighting intensity; To adjust the system; This represents the illumination frequency in the m-th direction; This represents the vector dot product operation; This represents the nth phase.

[0024] The raw image acquired by the microscope can be written as: .in, This represents the intensity distribution of the original image acquired by the microscope detector in the m-th direction and n-th phase; It is the actual sample structure to be restored; It is the point spread function of the microscopic system; Represents a two-dimensional spatial convolution operation; Represents the pointwise product of the actual sample structure and the lighting pattern; This represents the spatial domain noise introduced during the imaging process.

[0025] After converting it to the frequency domain, we get: .in, Represents the original image The two-dimensional Fourier transform spectrum; Optical transfer function; is the spectrum of the sample; i is the imaginary unit; and is a complex exponential factor, representing the complex phase rotation introduced by the phase shift in the frequency domain; This indicates that the sample spectrum has been shifted along the positive direction of the frequency axis. The result afterward; This indicates that the sample spectrum has been shifted along the negative direction of the frequency axis. The result afterward; This is the amplitude coefficient of the moved component; This represents the spectrum of spatial domain noise. The equation shows that each raw SIM image simultaneously contains three parts of information: the low-frequency information of the sample... And two sets of high-frequency information transferred in by structured light modulation. and .

[0026] This is why SIM typically requires multiple original images to reconstruct a single super-resolution image. For a fixed illumination direction, there are three sets of unknown information in the frequency domain, therefore at least three original images with different phases are needed to establish the equations and complete the separation. Usually, three phases are selected: The corresponding spectral relationships can be written in matrix form:

[0027] In this matrix form, , , These represent the original image spectra corresponding to three different phases under a fixed m-th direction; , , These correspond to the three specific phase values ​​selected above; the 1 in the matrix represents the center low-frequency component. The coefficients that are unaffected by phase rotation in each phase.

[0028] By solving this system of equations, the center spectrum and two sets of high-frequency spectra can be separated from three images with different phases. To more uniformly extend frequency coverage in a two-dimensional plane, SIM typically employs striped illumination in three different directions. Each direction requires three phases; therefore, two-dimensional SIM often uses 3 directions × 3 phases = 9 original images to reconstruct a single super-resolution image.

[0029] After completing the spectrum separation, the high-frequency components that have been moved need to be moved back to their original frequency positions and then weighted and fused with the low-frequency components: .in, This represents the reconstructed super-resolution spectrum; This represents the separated center low-frequency spectral component; and These represent the positive and negative high-frequency spectral components separated in the m-th direction, respectively. This represents the fusion weight applied to the central low-frequency component; and These represent the fusion weights applied to the positive and negative high-frequency components in the m-th direction, respectively. Represents the fusion weights related to the optical transfer function and noise level; This represents a summation operation over all illumination directions. Finally, an inverse Fourier transform is performed on the fused spectrum to obtain the super-resolution reconstructed image. .in, This represents the super-resolution image finally reconstructed in the spatial domain; This represents the two-dimensional inverse Fourier transform operation; This indicates the target of the inverse Fourier transform, namely the fused complete super-resolution spectrum.

[0030] Therefore, the core idea of ​​SIM is not simply averaging nine images, but rather using structured light illumination with different directions and phases to encode high-frequency information from the sample into multiple original images, and then reconstructing the super-resolution image through spectral separation, frequency shifting, and fusion. Theoretically, linear SIM can extend the observable frequency range by approximately two times, thereby achieving a near two-fold improvement in spatial resolution. However, in actual imaging, noise, undersampling, illumination parameter errors, and inaccurate optical transfer function estimation can all affect the spectral separation and fusion process, leading to artifacts, fringe retention, or detail distortion. This provides the necessary basis for subsequently introducing deep learning reconstruction and physical prior constraints.

[0031] Based on this, please refer to Figure 2-4This embodiment 1 will further elaborate on the implementation steps included in this embodiment.

[0032] (1) Multi-channel feature mapping First, a sequence of multiple low signal-to-noise ratio (SNR) raw images was acquired using a structured light microscopy imaging system. To cover complete frequency domain information and demodulate super-resolution images, images were acquired in multiple preset illumination directions and multiple phases within each direction. For each specific direction-phase combination, the imaging detector acquired one raw fluorescence image. Therefore, for a complete acquisition cycle, a total of [number missing] images were acquired. Nine original images are needed to reconstruct a super-resolution image. N represents the number of illumination directions, and P represents the number of phase steps per illumination direction. For example, in a traditional 2D SIM, N=3 (three different illumination directions) and P=3 (three phases per direction). Each image corresponds to a specific direction-phase combination, collectively carrying information about the sample under different spatial frequency modulations. These images constitute the raw dataset for subsequent processing, with each image carrying information about the sample under different spatial frequency modulations.

[0033] Next, the process of constructing the multi-channel input tensor involves reassembling the data from the acquired two-dimensional image sequence. The size is After processing the original images, where H represents the height of the original image (the number of pixels in the vertical direction) and W represents the width of the original image (the number of pixels in the horizontal direction), the data processing unit treats these images as data planes with different channel attributes. First, this... The images are stitched together along the channel dimension. This construction method preserves the intensity variation information of each pixel under different phases and orientations, transforming it into feature differences between channels, thus laying the data structure foundation for subsequent neural network extraction of phase and orientation features.

[0034] Then, The convolutional operation of the convolutional layer generates an initial shallow feature map. This embodiment uses a convolutional kernel with a size of [missing information]. A two-dimensional convolutional layer. The number of input channels of this convolutional layer must match the number of channels of the aforementioned multi-channel input tensor, i.e., the number of input channels is... To map low-level image data to a high-dimensional feature space, this convolutional layer contains a predetermined number of convolutional kernels. Each kernel has a size of [size missing]. This means that each convolutional kernel spatially covers The neighborhood of the input tensor covers channel information for all phases and directions in depth. During convolution, the convolution kernel slides along the spatial dimension of the input tensor with a preset stride and is padded to maintain the spatial resolution of the feature map.

[0035] The above convolution operations generate an initial shallow feature map. Each convolution kernel is element-wise multiplied with the input tensor and summed. After adding a bias term, the result is processed by a non-linear activation function to generate a two-dimensional feature map. Because of the use of... The convolutional kernels ultimately output a value of size [size missing]. The three-dimensional tensor, i.e., the initial shallow feature map. This process essentially compresses and maps the complex interference fringe and phase information contained in the original image sequence to a weighted sum. These feature maps, located in several high-dimensional feature channels, not only preserve the spatial structure of the original image but also extract the correlations between different phases and orientations, providing an initial representation for subsequent deep feature extraction and super-resolution image reconstruction.

[0036] (2) Dual-path feature fusion After generating the initial shallow feature map, it is input into the backbone structure consisting of a local feature module, a Mamba-Transformer hybrid module, and a convolutional module. The initial shallow feature map first enters the local feature module, which is composed of several standard 3×3 convolutional layers and activation functions stacked together. This module is used to capture edge, texture, and detail information in the cell structure, generating intermediate local features. These intermediate local features are then fed into the Mamba-Transformer hybrid module, which enhances the model's ability to understand large-scale structural relationships, enabling the network to focus on both local pixel changes and the overall continuity of the cell structure.

[0037] The Mamba-Transformer hybrid module contains two parallel processing paths: a first path and a second path. The first path employs a Transformer-based attention mechanism, specifically constructed as a simplified multi-head self-attention module. The process is as follows: First, the input feature map is processed by layer normalization, and then the local spatial feature representation is enhanced by depthwise separable convolution. Query tensor, key tensor, and value tensor are generated by linear projection layer, where the number of attention heads is set to 8, and the feature dimension of each attention head is the total number of channels divided by 8.

[0038] Subsequently, within each attention head, the dot product of the query tensor and the transpose of the key tensor is calculated and divided by a scaling factor to obtain the attention score matrix; the attention score matrix is ​​then Softmax normalized to obtain the attention weights; the value tensors are weighted and aggregated using the attention weights to generate intermediate features containing global context information. Finally, the output of the multi-head self-attention is nonlinearly transformed through a feedforward network with a bottleneck ratio of 2. The feedforward network contains two linear layers and a GELU activation function. The output of the feedforward network is added to the input features through residual connections to output the global enhancement features of the first path.

[0039] Meanwhile, the second path employs a Mamba-based state-space model, leveraging its linearly complex sequence modeling capabilities to efficiently capture long-range local continuity dependencies through a scanning mechanism. This enhances the continuity of weak signals and subtle structures. The specific process is as follows: First, the initial shallow feature map is scanned bidirectionally along the horizontal and vertical spatial dimensions to generate forward and backward sequences respectively. The forward sequence arranges feature pixels in the order of left to right and top to bottom, while the backward sequence arranges feature pixels in the order of right to left and bottom to top, so as to fully capture the long-range dependencies of image features in both directions. Secondly, the forward and backward sequences are respectively input into the state space model. The state space model models the sequence through learnable parameter matrices A, B, C, and D. Parameter matrix A is used to control the decay rate of the state, parameter matrices B and C are used to realize the linear mapping between input and output, and parameter matrix D is used to introduce residual connections. Then, a selective scanning mechanism is introduced to dynamically adjust the values ​​of parameter matrices B and C based on the local features of the input sequence, so that the model can adaptively focus on key continuous regions across the cell structure and suppress interference from irrelevant background. Next, the state dimension of the state space model is set to a preset value less than 64 to reduce the number of model parameters and computational complexity; the forward and backward sequences output by the state space model are recombined to restore a two-dimensional feature map with the same spatial size as the input feature map. Finally, the recombined forward two-dimensional feature map and backward two-dimensional feature map are added element-wise to obtain bidirectional fused long-range dependency features; the long-range dependency features are channel-adjusted through a 1×1 convolutional layer and residually connected with the input initial shallow feature map to output the long-range enhanced features of the second path.

[0040] Finally, at the end of the Mamba-Transformer hybrid module, a feature fusion unit is set up to integrate the outputs of the two paths. The specific process is as follows: First, the global context feature map output by the first path and the long-range enhancement feature map output by the second path are concatenated in the channel dimension to generate a high-dimensional concatenated feature map. The number of channels in the high-dimensional concatenated feature map is the sum of the number of channels output by the first path and the second path, which is used to preserve the complementary characteristics of global structural information and local continuous details. Subsequently, a 1×1 convolution operation is performed on the high-dimensional spliced ​​feature map to compress the number of channels from the spliced ​​high dimension back to the number of channels in the original input feature map, so that the output dimension remains the original number of channels C; the 1×1 convolution operation also realizes channel shuffling, promoting information interaction between features of different paths.

[0041] The output of the 1×1 convolution is the final blended deep feature, without the need for an additional spatial attention weight map mechanism. This process can be briefly represented as: ;in Indicates the branching characteristics of Transformer. Indicates Mamba branching characteristics, This is the final blended deep feature.

[0042] (3) Physical prior modulation Since structured light microscopy achieves super-resolution by projecting a specific periodic light field (usually sinusoidal fringes) onto the sample, the reconstructed image is inevitably modulated by this illumination pattern in both the frequency and spatial domains. To introduce this physical constraint into deep learning, a sinusoidal fringe pattern corresponding to the original input image sequence is first generated as a physical prior pattern based on the actual illumination parameters of the structured light microscopy system. It can be represented as ,in Background light intensity, For modulation amplitude, For the fringe frequency, This represents the phase. This physical prior pattern corresponds to the illumination state of the original image sequence, thus ensuring the physical consistency between the prior information and the input data.

[0043] Next, resolution matching and feature space adaptation are performed on the generated physical prior image. Since the network has propagated forward to this point, the mixed deep feature map has typically undergone multiple downsampling or convolution operations, and its spatial dimensions may differ from the original sinusoidal fringe pattern. Therefore, this embodiment employs a bilinear interpolation algorithm to upsample or downsample the generated physical prior image to the exact same spatial resolution as the mixed deep feature map.

[0044] To address the discrepancy between the ideal physical model and the actual imaging feature distribution, this embodiment further introduces a learnable affine transformation mechanism. First, the network initializes two trainable parameter vectors: a scaling factor... Translation factor Subsequently, through the formula The resampled images are transformed. This process allows the network to automatically adjust the strength and bias of physical priors during training, enabling their distribution to better adapt to the activation range of deep features, thereby avoiding rigid physical constraints that could destroy feature representation.

[0045] Finally, the physical constraint features are generated. The affine-transformed physical prior image is used as a modulation mask and multiplied element-wise with the blended deep feature map. The computational logic is as follows: ,in The Sigmoid activation function is used to normalize the physical prior to a range of 0 to 1, acting as a soft mask. Through this element-wise multiplication operation, regions in the physical prior pattern corresponding to high-frequency information of structured light are enhanced, while responses corresponding to non-physical imaging regions are suppressed. This mechanism is equivalent to applying an optical filter to the feature space, forcing the network to follow the imaging physics of structured light during reconstruction, thereby suppressing fringe artifacts and Gibbs effects common in traditional reconstruction algorithms, and ultimately outputting a feature map containing explicit physical constraints. This lays the foundation for subsequent high-fidelity image reconstruction.

[0046] (4) Lightweight super-resolution reconstruction The features, constrained by physical rules, need to be fed into a dedicated lightweight reconstruction network to complete the final resolution restoration and image generation.

[0047] First, a strict hardware adaptation configuration is performed on the number of main channels in the lightweight reconstruction network. To balance computational efficiency with the memory access efficiency of parallel computing hardware, this embodiment sets the number of main channels in the network's internal feature space to a preset value that satisfies the double divisibility constraint. This constraint includes two specific mathematical conditions: first, the number of main channels. The scaling factor must be able to be used by pixel recombination operations in the network. Integer division is used to ensure that feature maps can be uniformly rearranged into high-resolution images during subpixel convolution, avoiding channel truncation or padding; secondly, the number of main channels... The number of attention heads that must be able to branch out from the attention mechanism Integer division is used to ensure that the feature dimensions are evenly distributed to each attention head in multi-head self-attention computation, avoiding computational errors caused by dimension mismatch.

[0048] Specifically, in LMT-SSR, main channel compression is one of the key design features for achieving lightweight and fast inference. Here, LMT-SSR refers to the lightweight structured light microscopy super-resolution reconstruction network model proposed in this embodiment, and the main channel refers to the number of channels in the network's intermediate features. It is important to emphasize that this embodiment does not compress the original SIM input itself; the input remains 9 channels, corresponding to 3 illumination directions and 3 phases; the output remains a 1-channel super-resolution image. 9 channels refer to the channel dimension of the network input layer, 3 illumination directions refer to the number of different spatial frequency vector orientations used in structured light illumination, 3 phases refer to the number of initial phase offsets of the stripes used under each illumination direction, and 1 channel refers to the single-channel grayscale or fluorescence intensity image output by the network. Compression occurs within the network's internal feature space, mapping the original image to a more compact intermediate feature representation.

[0049] In the original PHCT structure, the number of main channels is typically set to 48. PHCT refers to the original physically inspired convolutional Transformer network structure used as a baseline for comparison, and 48 refers to the baseline value set for the intermediate feature channel dimension in the original network. LMT-SSR compresses the number of main channels to 40, allowing the network to reduce redundant computation while maintaining sufficient feature representation capabilities. 40 refers to the reset value for the intermediate feature channel dimension in the lightweight network. The input image first passes through... Convolutional mapping to 3D feature space: ; Convolution refers to a convolution operation with a spatial receptive field size of 3 pixels by 3 pixels. The channel dimension refers to the intermediate feature space of the network. The initial feature map obtained after the input image undergoes initial convolution mapping. The kernel size is The number of input channels is 9, and the number of output channels is [missing information]. The specific convolution operation, This refers to the 9-channel SIM raw input image. Number of primary channels. Taken from LMT-SSR. Since the convolutional layers, local feature modules, Transformer branches, Mamba branches, and upsampling reconstruction modules in the network all revolve around this main channel, the reduction... This will directly reduce the number of parameters, memory usage, and computational load of the entire network.

[0050] The main channel compression is effective because the number of parameters in most core layers is approximately quadratically related to the number of channels. For example, ordinary... The number of parameters in a convolution is approximately: ; Refers to ordinary The total number of parameters in a convolutional layer, 9 points The total number of spatial elements of the convolution kernel. This refers to the number of fully connected weights between the input and output channels. Therefore, when the number of main channels decreases from 48 to 40, the main computational scale of the relevant modules approximately becomes: ; 40 refers to the compressed number of main channels, and 48 refers to the original number of main channels. The ratio of the number of main channels after compression to the original number of main channels is 0.69, which is an approximate proportionality coefficient between the main computational scale of related modules after channel compression and the original scale. In other words, for modules that mainly rely on channel interaction, the number of parameters and computational cost can be reduced by about 30%. This type of compression has a very direct impact on inference speed, especially in SIM image reconstruction, where the input resolution is high, the feature map size is large, and changes in the number of channels can significantly affect the overall computational cost.

[0051] However, LMT-SSR doesn't simply aim to minimize the number of channels; instead, it strikes a balance between structural compatibility and expressive power, choosing 40 as a compromise. First, the physical prior modulation module in the network uses PixelShuffle and PixelUnShuffle operations, so the number of main channels needs to be divisible by 4. The physical prior modulation module refers to the feature processing component that introduces physical imaging constraints into the network. PixelShuffle refers to the operation of rearranging the channel dimension to the spatial dimension for upsampling, and PixelUnShuffle refers to the operation of rearranging the spatial dimension to the channel dimension for downsampling. 4 refers to the scaling factor for the spatial-to-channel dimension transformation during the rearrangement operation. Second, the Transformer branch uses 8 attention heads, and the number of main channels also needs to be divisible by 8. 8 refers to the number of heads that compute self-attention in parallel within the Transformer branch. 40 satisfies both conditions, with each attention head corresponding to 5 channels, ensuring structural feasibility while avoiding excessive channel redundancy. 5 refers to the number of feature channels allocated to each attention head.

[0052] Simultaneously, main channel compression works in conjunction with network backend simplification. LMT-SSR concentrates its main expressive power in the front-end RLFB+Mamba-Transformer+Conv3 hybrid module. After physical prior modulation, the backend uses only lightweight... Convolution refines features instead of repeatedly stacking complex modules. RLFB refers to residual local feature blocks used to extract local spatial details, Mamba-Transformer refers to a hybrid feature extraction component combining a state-space model and a self-attention mechanism, and Conv3 refers to the feature fusion component in the hybrid module. Convolutional components. The logic behind this is that the front end has already completed local detail extraction, long-range dependency modeling, and global structural association, while the back end is mainly responsible for organizing and reconstructing the modulated features. Therefore, there is no need to continue to retain high-complexity structures.

[0053] If LMT-SSR maintains 48 main channels, the number of parameters is approximately 0.272M; when the number of main channels is compressed to 40, the number of parameters is approximately 0.194M, a reduction of about 28.5%. Compared to the original two-stage PHCT backbone, LMT-SSR reduces the number of parameters by approximately 44.6% after channel compression and structural simplification. The two-stage PHCT backbone refers to the core feature extraction part of the original network consisting of two cascaded stages, and 44.6% refers to the percentage reduction in the number of parameters of LMT-SSR compared to the original two-stage PHCT backbone. This indicates that main channel compression not only reduces the model size but also provides a foundation for improving inference speed.

[0054] Therefore, the main channel compression optimization of LMT-SSR can be summarized as follows: without changing the SIM input information and physical prior constraints, the feature width within the network is compressed from a large 48 to 40, and combined with lightweight Mamba parameters and simplified backend reconstruction modules, redundant computation is reduced. This design enables the model to maintain local detail recovery, long-range structural modeling, and physical consistency while having a smaller number of parameters, lower computational overhead, and faster inference speed, making it more suitable for real-time or large-scale SIM super-resolution reconstruction tasks.

[0055] Finally, the lightweight network ends with a sub-pixel convolutional layer. This layer first expands the number of channels in the feature map using a convolutional layer. ,in This represents the super-resolution magnification. Subsequently, using periodic pixel rearrangement operations, the data in the channel dimension is rearranged across the entire spatial dimension, that is, each... The channel blocks are expanded and filled into the spatial location of the feature map, thereby increasing the spatial size of the low-resolution feature map. The number of channels is multiplied, while the number of channels is restored to the number of channels in the target image. Finally, after processing by an output activation function, a high-resolution super-resolution reconstructed image is generated, completing the mapping from physical constraint features to the final optical image.

[0056] Example 2 Please refer to Figure 5 This embodiment 2 provides a dual-path super-resolution image reconstruction system that integrates state space and attention mechanisms, including: The multi-channel feature mapping unit is used to acquire multiple frames of low signal-to-noise ratio original image sequences acquired by the structured light microscopy imaging system, and stitch them together into a multi-channel input tensor; the input tensor is used to perform preliminary feature extraction through a shallow convolutional network, and the image data is mapped to a high-dimensional feature space to generate an initial shallow feature map. A dual-path feature fusion unit is used to input the initial shallow feature map into the backbone structure, wherein the backbone structure includes a Mamba-Transformer hybrid module. The hybrid module contains two parallel processing paths: the first path adopts a Transformer-based attention mechanism, using a multi-head self-attention module to capture the global spatial dependencies of image features to reconstruct the overall macroscopic structure of cells or samples; the second path adopts a Mamba-based state-space model, utilizing its linear complexity sequence modeling capability to efficiently capture long-distance local continuous dependencies through a scanning mechanism to enhance the continuity of weak signals and fine structures; the features output from the two paths are concatenated and fused to generate a hybrid deep feature that combines global perception and local details. A physical prior modulation unit is used to construct a physical prior pattern that matches the structured light illumination mode. The physical prior pattern represents the periodic modulation information of the structured light. The physical prior pattern is used as a modulation mask to perform element-wise multiplication or affine transformation on the mixed deep features to introduce physical constraints of optical imaging in the feature space, suppress the generation of non-physical artifacts, and output physical constraint features. The lightweight super-resolution reconstruction unit is used to input the physical constraint features into the lightweight reconstruction network, which includes compressed main channel convolutional layers and sub-pixel convolutional layers. By reducing the number of main channels, the number of parameters is reduced, and upsampling is performed using sub-pixel convolutional layers to restore the low-resolution feature map to the high-resolution space, and finally outputs the super-resolution reconstructed image.

[0057] Example 3 This embodiment 3 also provides a computer-readable storage medium storing a computer program that, when executed by a processor, can implement any step of a dual-path super-resolution image reconstruction method that integrates state space and attention mechanisms.

[0058] The computer-readable storage medium may include various media capable of storing program code, such as USB flash drives, portable hard drives, read-only memory (ROM), random access memory (RAM), magnetic disks, or optical disks.

[0059] For a description of the computer-readable storage medium provided in this application, please refer to the above method embodiments; further details will not be repeated here.

[0060] Those skilled in the art will readily understand that the above description is merely a preferred embodiment of the present invention and is not intended to limit the present invention. Any modifications, equivalent substitutions, and improvements made within the spirit and principles of the present invention should be included within the scope of protection of the present invention.

Claims

1. A dual-path super-resolution image reconstruction method integrating state space and attention mechanisms, characterized in that, include: S1. Acquire a sequence of multiple low signal-to-noise ratio original images acquired by a structured light microscopy imaging system, and stitch them together into a multi-channel input tensor; perform preliminary feature extraction on the input tensor through a shallow convolutional network, map the image data to a high-dimensional feature space, and generate an initial shallow feature map; S2. The initial shallow feature map is input into the backbone structure, which includes a Mamba-Transformer hybrid module. The hybrid module contains two parallel processing paths: the first path uses a Transformer-based attention mechanism, utilizing a multi-head self-attention module to capture the global spatial dependencies of image features to reconstruct the overall macroscopic structure of cells or samples; the second path uses a Mamba-based state-space model, utilizing its linear complexity sequence modeling capability to efficiently capture long-distance local continuous dependencies through a scanning mechanism to enhance the continuity of weak signals and fine structures; the features output from the two paths are concatenated and fused to generate a hybrid deep feature that combines global perception and local details. S3. Construct a physical prior pattern that matches the structured light illumination mode, wherein the physical prior pattern represents the periodic modulation information of the structured light; use the physical prior pattern as a modulation mask to perform element-wise multiplication or affine transformation on the mixed deep features to introduce physical constraints of optical imaging in the feature space, suppress the generation of non-physical artifacts, and output physical constraint features. S4. Input the physical constraint features into a lightweight reconstruction network, which includes compressed main channel convolutional layers and sub-pixel convolutional layers; reduce the number of parameters by reducing the number of main channels, and use sub-pixel convolutional layers to perform upsampling operations to restore the low-resolution feature map to the high-resolution space, and finally output the super-resolution reconstructed image.

2. The dual-path super-resolution image reconstruction method fusing state space and attention mechanism according to claim 1, characterized in that, The specific process of preliminary feature extraction in S1 includes: Acquire raw image sequences acquired by a structured light microscopy system under multiple illumination directions and multiple phases; The original image sequence is stitched together along the channel dimension to construct a multi-channel input tensor; The multi-channel input tensor is convolved by a 3×3 convolutional layer containing a preset number of convolutional kernels to map the image data to a high-dimensional feature space and generate an initial shallow feature map.

3. The dual-path super-resolution image reconstruction method fusing state space and attention mechanism according to claim 1, characterized in that, The backbone structure of S2 includes a local feature module, a Mamba-Transformer hybrid module, and a convolution module. The local feature module is used to capture edge, texture, and detail information in the cell structure. The Mamba-Transformer hybrid module is used to enhance the model's ability to understand large-scale structural relationships, so that the network pays attention to both local pixel changes and the overall continuity of the cell structure.

4. The dual-path super-resolution image reconstruction method fusing state space and attention mechanism according to claim 3, characterized in that, In S2, the first path employs a Transformer-based attention mechanism, specifically constructed as a simplified multi-head self-attention module, whose processing includes: The input feature map is subjected to layer normalization, and then the local spatial feature representation is enhanced by depthwise separable convolution. The query tensor, key tensor and value tensor are generated by linear projection layer, and the number of attention heads is set to 8. Within each attention head, the dot product of the query tensor and the transpose of the key tensor is calculated and divided by a scaling factor to obtain the attention score matrix; the attention score matrix is ​​then Softmax normalized to obtain the attention weights; the value tensors are weighted and aggregated using the attention weights to generate intermediate features containing global context information. The output of the multi-head self-attention is nonlinearly transformed through a feedforward network with a bottleneck ratio of 2. The feedforward network contains two linear layers and a GELU activation function. The output of the feedforward network is added to the input features through residual connections to output the global enhancement features of the first path.

5. The dual-path super-resolution image reconstruction method fusing state space and attention mechanism according to claim 3, characterized in that, In S2, the second path adopts a state-space model based on Mamba, specifically including: The initial shallow feature map is scanned bidirectionally along the horizontal and vertical spatial dimensions to generate forward and backward sequences respectively. The forward sequence arranges feature pixels in the order of left to right and top to bottom, while the backward sequence arranges feature pixels in the order of right to left and bottom to top, so as to fully capture the long-range dependencies of image features in both directions. The forward and backward sequences are respectively input into the state space model. The state space model models the sequence through learnable parameter matrices A, B, C, and D. Parameter matrix A is used to control the decay rate of the state, parameter matrices B and C are used to realize the linear mapping between input and output, and parameter matrix D is used to introduce residual connections. A selective scanning mechanism is introduced to dynamically adjust the values ​​of parameter matrices B and C based on the local features of the input sequence, enabling the model to adaptively focus on key continuous regions across cell structures and suppress interference from irrelevant background. The state dimension of the state space model is set to a preset value less than 64 to reduce the number of model parameters and computational complexity; the forward and backward sequences output by the state space model are recombined to restore a two-dimensional feature map with the same spatial size as the input feature map. The recombined forward 2D feature map and backward 2D feature map are added element-wise to obtain bidirectional fused long-range dependency features. The long-range dependency features are channel-adjusted through a 1×1 convolutional layer and residually connected with the input initial shallow feature map to output the long-range enhanced features of the second path.

6. The dual-path super-resolution image reconstruction method according to claim 3, characterized in that, In S2, the Mamba-Transformer hybrid module further includes a feature fusion unit; The feature fusion unit is configured as follows: The global context feature map output by the first path and the long-range enhancement feature map output by the second path are concatenated in the channel dimension to generate a high-dimensional concatenated feature map. The number of channels in the high-dimensional concatenated feature map is the sum of the number of channels output by the first path and the second path, which is used to preserve the complementary characteristics of global structural information and local continuous details. A 1×1 convolution operation is performed on the high-dimensional spliced ​​feature map to compress the number of channels from the spliced ​​high dimension back to the number of channels in the original input feature map, so that the output dimension remains the original number of channels C; the 1×1 convolution operation also realizes channel shuffling, promoting information interaction between features of different paths; the output of the 1×1 convolution is the final mixed deep feature.

7. The dual-path super-resolution image reconstruction method fusing state space and attention mechanism according to claim 1, characterized in that, In S3, the process of constructing and modulating the physical prior pattern specifically includes: Based on the illumination parameters of the structured light microscopy system, a sinusoidal fringe pattern corresponding to the original input image sequence is generated as a physical prior pattern. The physical prior image is upsampled to the same spatial resolution as the hybrid deep features using bilinear interpolation. Affine transformations of physical prior maps are performed using learnable scaling and translation factors to adapt them to the distribution of the feature space. The transformed physical prior image is used as a modulation mask and multiplied element-wise with the hybrid deep features to explicitly introduce physical constraints of optical imaging into the feature space.

8. The dual-path super-resolution image reconstruction method fusing state space and attention mechanism according to claim 1, characterized in that, In S4, the lightweight reconstruction network adopts a main channel compression strategy, specifically including: The number of main channels in the feature space of the lightweight reconstruction network is set to a preset value that satisfies the double divisibility constraint. The double divisibility constraint includes: the number of main channels is divisible by the scaling factor of the pixel recombination operation in the network and divisible by the number of attention heads of the attention mechanism branch.

9. A dual-path super-resolution image reconstruction system integrating state space and attention mechanisms, characterized in that, include: The multi-channel feature mapping unit is used to acquire multiple frames of low signal-to-noise ratio original image sequences acquired by the structured light microscopy imaging system, and stitch them together into a multi-channel input tensor; the input tensor is used to perform preliminary feature extraction through a shallow convolutional network, and the image data is mapped to a high-dimensional feature space to generate an initial shallow feature map. A dual-path feature fusion unit is used to input the initial shallow feature map into the backbone structure, wherein the backbone structure includes a Mamba-Transformer hybrid module. The hybrid module contains two parallel processing paths: the first path adopts a Transformer-based attention mechanism, using a multi-head self-attention module to capture the global spatial dependencies of image features to reconstruct the overall macroscopic structure of cells or samples; the second path adopts a Mamba-based state-space model, utilizing its linear complexity sequence modeling capability to efficiently capture long-distance local continuous dependencies through a scanning mechanism to enhance the continuity of weak signals and fine structures; the features output from the two paths are concatenated and fused to generate a hybrid deep feature that combines global perception and local details. A physical prior modulation unit is used to construct a physical prior pattern that matches the structured light illumination mode. The physical prior pattern represents the periodic modulation information of the structured light. The physical prior pattern is used as a modulation mask to perform element-wise multiplication or affine transformation on the mixed deep features to introduce physical constraints of optical imaging in the feature space, suppress the generation of non-physical artifacts, and output physical constraint features. The lightweight super-resolution reconstruction unit is used to input the physical constraint features into the lightweight reconstruction network, which includes compressed main channel convolutional layers and sub-pixel convolutional layers. By reducing the number of main channels, the number of parameters is reduced, and the sub-pixel convolutional layers are used for upsampling to restore the low-resolution feature map to the high-resolution space, and finally output the super-resolution reconstructed image.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, The computer program is executed by a processor as described in any one of claims 1-8, which is a dual-path super-resolution image reconstruction method that integrates state space and attention mechanisms.

Images