Mamba lightweight building image super-resolution reconstruction method based on state equation model

Abstract

The invention discloses a Mama lightweight building image super-resolution reconstruction method based on a state equation model, and the method specifically comprises the following steps: 1, obtaining a building high-definition image, carrying out the down-sampling of an original high-resolution image, and obtaining a corresponding low-resolution image; 2, performing data enhancement on the low-resolution image; step 3, constructing a Mamba lightweight building image super-resolution network model based on the state equation model; and step 4, training the super-resolution model to obtain a Mamba lightweight building image super-resolution reconstruction model based on the state equation model. According to the method, the regular texture of the building image can be effectively recovered, the problem of balancing between the effective receptive field and the performance of the existing lightweight image super-resolution network is solved, and efficient feature extraction is performed on the image under the condition of not increasing the calculation overhead to obtain global information; and the lightweight performance and the reconstruction quality of the lightweight building image super-resolution model are effectively improved.

Description

Technical Field

[0001] The invention relates to the technical field of image processing, in particular to a Mamba lightweight building image super-resolution reconstruction method based on a state equation model. Background Art

[0002] As a non-contact monitoring method, visual measurement methods are widely used in application scenarios such as structural surface defect detection and structural dynamic and static response measurement, and are playing an increasingly important role in the field of structural health monitoring. Due to factors such as equipment cost constraints, large coverage area of ​​building structures, limited camera placement, and complex environmental conditions, the images collected by visual measurement systems often have insufficient resolution. Super-resolution reconstruction technology aims to reconstruct high-resolution (HR) images from corresponding low-resolution (LR) images, providing an effective way to solve this problem. However, the surface of building structures often has regular texture features, so its super-resolution reconstructed image is sensitive to artifact noise, and its reconstruction quality depends particularly on the network's ability to model long-range relationships. With the introduction of modern deep learning models (such as convolutional neural networks and Transformer models), the performance of image super-resolution models has been continuously improved. However, the high computational cost and memory consumption of many existing models make them difficult to be applied in edge and mobile devices with limited resources. In order to improve the efficiency of super-resolution reconstruction of structural building surface images, it is urgent to design lightweight and efficient SR models with long-range relationship modeling capabilities.

[0003] In recent years, researchers have proposed a large number of lightweight and efficient SR methods, but they still lack sufficient convolutional layers to increase the overall receptive field, which limits their ability to capture global information, is not conducive to extracting and retaining higher-level patterns and structures, and restricts their anchor pixel reconstruction performance.

[0004] For CNN-based super-resolution networks, although the effective receptive field is limited, it is suitable for deployment on resource-constrained devices due to the good efficiency of convolutional parallel operations. In contrast, Transformer-based super-resolution methods outperform CNN-based methods due to their larger receptive field, but their computational complexity grows quadratically with the input size, which is not friendly to lightweight super-resolution networks. In order to apply Transformer to lightweight super-resolution, existing studies have adopted some efficient attention mechanisms, such as shifted window attention. Although it effectively reduces the time complexity, it usually sacrifices the global effective receptive field and does not essentially escape the trade-off between effective receptive field and model performance.

[0005] In recent years, structured state-space sequence models (S4), especially the improved version Mamba, have gradually emerged as efficient and effective backbone networks for building deep networks. This development provides a potential solution for balancing global receptive field and computational efficiency in image inpainting. Specifically, the discretized state-space equations in Mamba can be formalized as a recursive form, and through specially designed structural reparameterization, long-range dependencies can be modeled. This means that the inpainting network based on Mamba can naturally activate more pixels and improve the reconstruction quality. In addition, the parallel scanning algorithm in Mamba can not only obtain the global features of the image with linear complexity, but also process each token in a parallel manner, which promotes efficient training on modern hardware (such as GPU). The above promising properties inspire us to explore the potential of Mamba for efficient long-range modeling in lightweight image inpainting networks.

[0006] However, standard Mamba was originally designed for one-dimensional sequence data in natural language processing (NLP), so there are some challenges in its direct application to image restoration tasks. First, since Mamba processes one-dimensional flattened image sequences in a recursive manner, this may cause distant pixels in the sequence to be mistaken for spatially close pixels, thereby causing the forgetting of local pixel information. Second, due to the need to capture long-range sequence dependencies, the number of hidden states in the state-space equation is often very large, which will cause channel redundancy and thus affect the effective learning of key channel representations. In addition, Mamba's scanning method for flattening images also needs to be further optimized so that it can better obtain global image information. Summary of the invention

[0007] In view of this, the purpose of the present invention is to provide a Mamba lightweight building image super-resolution reconstruction method based on a state equation model to solve the problems of slow speed and insufficient image information extraction capability of the image super-resolution method in the prior art, and to design a local feature enhancement module to effectively improve the shortcomings of the existing Mamba model.

[0008] To achieve the above object, the present invention adopts the following technical solution: a Mamba lightweight building image super-resolution reconstruction method based on a state equation model, comprising the following steps:

[0009] Step S1: Obtain a high-definition image of the building, and use a bicubic interpolation algorithm to obtain a low-resolution image corresponding to the lightweight image dataset;

[0010] Step S2: perform data enhancement on the input low-resolution data;

[0011] Step S3: constructing a Mamba lightweight building image super-resolution reconstruction network based on a state equation model, namely, a MambaESR network model, which is specifically divided into a Mamba feature modulation network MFM and a local feature enhancement module LFRM;

[0012] Step S4: training the MambaESR network model constructed in step S3;

[0013] Step S5: Input the test set into the MambaESR network model trained in step S4 and output a super-resolution image.

[0014] In a preferred embodiment: in step S1, the high-resolution image of the building is downsampled using a bicubic interpolation algorithm to obtain a corresponding low-resolution image.

[0015] In a preferred embodiment: in step S2, data enhancement is performed by scaling, flipping, and rotating at different scales to improve the generalization ability of the model.

[0016] In a preferred embodiment: the Mamba feature modulation network described in step S3 adopts a network architecture based on a state equation model to replace the traditional CNN and Transformer architectures in view of the fact that building images have a large dependence on long-range information, so as to reduce the computational cost and increase the receptive field; the specific design process of the visual state equation model is as follows:

[0017] Step S3.1: First, perform channel segmentation on the normalized input features. Given the input feature X, this process is expressed as:

[0018] [X0,X1]=CHUNK(X) (1)

[0019] Among them, CHUNK(·) is the channel segmentation operation, [X0,X1] is the two-part feature after the input feature is segmented;

[0020] Step S3.2: In the first branch, the number of feature channels is expanded to γC through linear mapping, where γ is a predefined channel expansion factor, and then transformed through depthwise convolution, SiLU activation function, state space module SSM layer and layer normalization LayerNorm, expressed as:

[0021]

[0022] Among them, DW-Conv 3×3 (·) represents a depth-wise separable 3×3 convolution kernel, φ(·) represents a GELU activation function, Linear(·) represents a linear mapping operation, SSM(·) represents a state-space module, and LN(·) represents a layer normalization operation. is the intermediate feature of the first branch after the above operation;

[0023] Step S3.3: In the second branch, the feature channel is also expanded to γC through the linear layer and processed by the SiLU activation function; the feature representations of the two branches are fused by element-wise product, expressed as:

[0024] X2=φ(Linear(X2)) (3)

[0025] Among them, X2 is the output feature of the second branch;

[0026] Step S3.4: The number of feature channels is mapped back to C to generate the output X out , whose shape is consistent with the input, this process can be expressed as:

[0027] X out =Linear(X1×X2) (4).

[0028] In a preferred embodiment: Step S3 uses a variety of different image scanning methods to expand and flatten an image into a one-dimensional image in different ways, thereby obtaining global information of the image without adding additional burden;

[0029] In a preferred embodiment: the specific design process of the local feature enhancement module is as follows:

[0030] Step S3.4: In the local feature enhancement module, the number of channels of the input feature is first doubled using a 1×1 convolution. This process is expressed as:

[0031] X'=Conv 1×1 (X in ) (5)

[0032] Step S3.5: Introduce a depth-wise separable convolution kernel to encode spatial local context information, and add a self-residual to the depth-wise convolution kernel to improve the reconstruction result; then apply the GELU function to the hidden layer for nonlinear mapping, pass its result through an SE attention module, and finally reduce the channel to the original input dimension through 1×1 convolution to obtain the output feature. This process is expressed as:

[0033] X out =Conv 1×1 (SE(φ(DWConv 3×3 (X')+X))) (6)

[0034] Among them, X in and X out Represents input features and output features respectively, DWConv 3×3represents a depth-wise separable 3×3 convolution kernel, SE module represents a spatial attention mechanism module, and φ(·) represents a GELU activation function.

[0035] In a preferred embodiment: the specific process of step S4 is:

[0036] Step S4.1: Input a low-resolution image of a building, generate a super-resolution image through the network model, calculate the regularization loss between the reconstructed image and the real image, use the mean absolute error loss and the FFT-based frequency loss function to optimize the network parameters, and continuously train the super-resolution reconstruction network through back-propagation to perform training and learning to obtain the final super-resolution reconstruction network after training; the loss function is defined as:

[0037] Loss=||I SR -I HR ||1+γ||F(I SR )-F(I HR )||1 (7)

[0038] Among them I HR is a high-resolution ground-truth image, ||·||1 represents the L1 norm, F represents the fast Fourier transform, and γ is a weight factor, which is empirically set to 0.1;

[0039] Step S4.2: Design and train three super-resolution networks of different sizes, named MambaESR-S, MambaESR-M, and MambaESR-L respectively; MambaESR-S has a dimension of 36 and is stacked with 8 modules; MambaESR-M has a dimension of 48; and MambaESR-L has a dimension of 48 and is composed of 12 modules; during training, each batch contains 16 LR images and the input size is 256×256; the Adam optimizer with β1=0.9 and β2=0.99 is used to solve the proposed model; the initial learning rate is set to 1×10 -3 , the minimum value is set to 1×10 -6 , the learning rate is updated by the cosine annealing algorithm; the weight of the exponential moving average EMA is set to 0.999; during the implementation process, the model uses a total of 10 6 Training is performed for iterations.

[0040] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention improves and perfects the traditional model based on the Mamba modulation mechanism, achieves a good balance between performance and efficiency, and overcomes the trade-off problem between computational complexity and reconstruction performance of the traditional super-resolution model. 2. The present invention studies the principle and application of the Mamba core scanning mechanism, and by studying the influence of various scanning directions on the image super-resolution effect, it better models the long-range feature relationship and significantly improves the detail expression of image reconstruction. 3. The present invention designs a local feature enhancement module to improve the Mamba network. This lightweight module can effectively encode and enhance local contextual information, and perform channel mixing at the same time, effectively improving the local pixel forgetting and channel redundancy of the original Mamba, and effectively improving the model effect. BRIEF DESCRIPTION OF THE DRAWINGS

[0041] Figure 1 It is a flow chart of the Mamba lightweight building image super-resolution reconstruction method based on the state equation model in the preferred embodiment of the present invention;

[0042] Figure 2 The overall flow chart and network internal structure diagram of the specific architecture of the preferred embodiment of the present invention;

[0043] Figure 3 Comparison of the local attribution map (LAM) and diffusion index (DI) of various efficient super-resolution models;

[0044] Figure 4 This is a diagram of the scanning method of the image in MFM;

[0045] Figure 5 This is a diagram showing the improvement effect of LFRN on Mamba according to a preferred embodiment of the present invention;

[0046] Figure 6 This is a comparison diagram of the effects of the preferred embodiment of the present invention and other lightweight super-resolution models;

[0047] Figure 7 A comparison diagram is shown for details of the preferred embodiment of the present invention and the existing algorithm on the ×4 super-resolution (SR) task on the architectural image. DETAILED DESCRIPTION

[0048] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0049] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0050] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0051] A Mamba lightweight building image super-resolution reconstruction method based on the state equation model, such as Figure 1-7 As shown. The low-resolution image block I LR Input to the network, can output high-resolution result image I SR .

[0052] Follow the steps below to implement it:

[0053] Step S1: obtaining a high-resolution building image, and downsampling the original high-resolution building image to obtain a corresponding low-resolution image;

[0054] The specific process of downsampling is: using a bicubic interpolation algorithm to process a high-resolution image, and obtaining a corresponding low-resolution image according to different reduction factors (such as ×2, ×3, ×4, etc.).

[0055] Step S2: perform data enhancement on the low-resolution image;

[0056] Specific data enhancement operations include scaling, flipping, rotating, and other methods to enhance data in order to improve the generalization ability of the model.

[0057] Step S3: In view of the trade-off between the effective receptive field and performance of the traditional super-resolution model, and the characteristics of regular texture and large long-range dependence in building images, a Mamba lightweight building image super-resolution network model based on the state equation model is constructed. It is specifically divided into two modules, the Mamba feature modulation unit and the local feature enhancement module, such as Figure 2 As shown in Figure 2, the state equation-based model has a more significant effect in terms of effective receptive field than the traditional CNN and Transformer-based models. Figure 3 As shown;

[0058] See Figure 2 ,The network model proposed in this paper consists of three modules: shallow ,feature extraction, deep feature extraction and upsampling feature reconstruction.

[0059] Step S3.1: Specifically, the network first applies a convolutional layer with a kernel size of 3x3 pixels to extract shallow features, convert the input LR image into feature space, and generate shallow features f0. The specific process can be expressed as:

[0060] f0=Conv 3×3 (I LR ) (1)

[0061] Among them, Conv 3×3 (·) indicates a 3×3 convolutional layer.

[0062] Step S3.2: Pass the shallow feature f0 to the deep feature extraction module composed of a series of Vision Mamba Modulation Block (VMMB). This process can be expressed as:

[0063] f k =H k (f k-1 ),k=1,…,n (2)

[0064] Among them, H k (·) represents the kth VMMB. k-1 and f k They represent the input features and output features of the kth VMMB respectively.

[0065] Step S3.3: In order to better restore the HR target image, a global residual connection is introduced. Finally, a lightweight upsampling layer consisting of a 3×3 convolution layer and a sub-pixel convolution is used to quickly reconstruct the HR image. The specific process can be expressed as:

[0066] I SR =P θ (f k +f0) (3)

[0067] Among them, I SR is the predicted HR image, P θ (·) denotes the upsampling reconstruction module described.

[0068] Step S3.4: Architectural structures often have regular texture features, so their super-resolution reconstructed images are sensitive to artifact noise, and their reconstruction quality is particularly dependent on the network's ability to model long-range relationships. The standard Mamba model processes input data in a causal manner, and can scan the information in the area through different scanning methods to obtain the long-range dependencies of the image. Therefore, the state space model is introduced into the image restoration task to replace the traditional CNN and Transformer architectures to achieve more efficient global feature modeling. The architecture of the Mamba feature modulation unit is shown in the figure. Figure 2 The specific design process is as follows:

[0069] Step S3.4.1: First, perform channel segmentation on the normalized input feature X to generate two parts of features, which are processed by two parallel branches. It can be expressed as:

[0070] [X0,X1]=CHUNK(X) (4)

[0071] Among them, CHUNK(·) is a channel splitting operation.

[0072] Step S3.4.2: In the first branch, the number of feature channels is expanded to γC through linear mapping, where γ is a predefined channel expansion factor, and then transformed through depthwise convolution, SiLU activation function, state space module (SSM) layer and layer normalization (LayerNorm). It can be expressed as:

[0073]

[0074] Among them, DW-Conv n×n (·) represents a depth-wise separable 3×3 convolution kernel, φ(·) represents a GELU function, Linear(·) represents a linear layer, and SSM represents a state-space model, which includes different ways of scanning images, such as Figure 4 As shown in Figure 3, our model uses 4 directions to expand the image to obtain the global information of the image.

[0075] Step S3.4.3: In the second branch, the feature channel is also expanded to γC through the linear layer and processed by the SiLU activation function. The feature representations of the two branches are fused through element-wise product. It can be expressed as:

[0076] X2=φ(Linear(X2)) (6)

[0077] Step S3.4.4: Finally, the number of feature channels is mapped back to C to generate the output X out , whose shape is consistent with the input. This process can be expressed as

[0078] X out =Linear(X1×X2) (7)

[0079] Step S3.5: Although the Mamba feature modulation unit can model the long-range relationship between pixels, it ignores the importance of local context information for super-resolution image reconstruction and has channel redundancy problems. Therefore, the present invention designs a local feature enhancement module to be added after the Mamba feature modulation unit to supplement the local context information. The local feature enhancement module is as follows: Figure 2shown.

[0080] Step S3.5.1: In the local feature enhancement module, the number of channels of the input feature is first doubled using a 1×1 convolution kernel. This process can be expressed as:

[0081] X'=Conv 1×1 (X in ) (8)

[0082] Where X in Represents the input image, and X' represents the intermediate features obtained after passing through the convolution kernel.

[0083] Step S3.5.2: Introduce a depthwise separable convolution kernel to encode spatial local context information, and add a self-residual to the depthwise convolution kernel to improve the reconstruction result, so that the model can further improve performance without computational cost in the inference phase. Then apply the GELU function to the hidden layer for nonlinear mapping, pass its result through an SE attention module, and finally reduce the channel to the original input dimension through convolution to obtain the output feature. The whole process can be expressed as:

[0084] X onv =Conv 1×1 (SE(φ(DW-Conv 3×3 (X')))) (15)

[0085] Among them, X out Represents the output features, DW-Conv 3×3 Denotes the use of a depth-separable 3×3 convolution kernel, SE module denotes the spatial attention mechanism module, and φ(·) denotes the activation function GELU. We apply RELU and global average pooling to the output of the last layer of the network to obtain the channel activation value, as Figure 5 shown. Figure 5 The left image does not use the Local Feature Enhancement Module (LFRM). It is obvious that most of the channels are not activated, which indicates the channel redundancy problem in the original Mamba modulation block. Figure 5 The right figure uses LFRM to effectively solve the channel redundancy problem. The local feature enhancement module effectively improves the channel redundancy phenomenon of MFM.

[0086] Step S3.6: Design the overall architecture. In recent lightweight image super-resolution models, the module design mainly follows the Transformer process, that is, global feature extraction first, and then local feature information. Figure 2 As shown. The deep feature extraction module of the network mainly consists of two parts, which can be expressed as

[0087] X'=MFM(LN(X in ))+α·X in(8)

[0088] X out =LFRM(LN(X'))+β·X' (9)

[0089] Among them, X in and X out They are respectively represented as the input features and output feature maps of the deep feature extraction module, LN(·) is the LayerNorm operation, and X' represents the intermediate features. The Mamba Feature Modulation (MFM: Mamba) feature modulation block and the Local Feature Reinforcement Module (LFRM: Local Feature Enhancement Unit) play different roles in the network. The former effectively models long-range dependencies, while the latter focuses more on fine local context information. They complement each other and work together to achieve more accurate image reconstruction. Finally, we also add learnable scale factors α and β to dynamically adjust the information flow. The network can dynamically adjust the information transfer between different layers, so that the model can more flexibly handle different input features and task requirements.

[0090] Step S4: training the network model constructed in step 3;

[0091] Step S4.1: First, define the loss function. This method optimizes our network parameters by using the mean absolute error loss and the FFT-based frequency loss function. This process can be expressed as:

[0092] Loss=||I SR -I HR ||1+γ||F(I SR )-F(I HR )||1 (12)

[0093] Among them I HR is a high-resolution ground-truth image, ||·||1 represents the L1 norm, F represents the fast Fourier transform, and γ is a weight factor, which is empirically set to 0.1.

[0094] S4.2: In order to demonstrate the performance of the present invention, three super-resolution networks of different sizes were designed and trained, named MambaESR-S, MambaESR-M and MambaESR-L respectively. Among them, MambaESR-S has a dimension of 36 and uses 8 stacked modules; MambaESR-M has a dimension of 48; and MambaESR-L has a dimension of 48 and uses 12 modules. During the training process, each batch contains 16 LR images, and the input size is. We use the Adam optimizer with and to solve the proposed model. We set the initial learning rate to, the minimum value to, and the learning rate is updated by the cosine annealing algorithm. The weight of the exponential moving average (EMA) is set to 0.999. During the implementation process, the model of the present invention was trained using a total of iterations on an NVIDIA GeForceRTX 3080 GPU.

[0095] Step S5: Input the test set into the network model trained in step 4 and output the super-resolution image.

[0096] To evaluate the performance of the present invention, we compared it with the most advanced lightweight SR methods, including CNN-based methods CARN, LAPAR, ShufferMixer, SAFMN, PAN, Transformer-based methods LatticeNet, ELAN, SwinIR-light, SPIN, and the existing Mamba-based method MambaIR. Quantitative comparison results on benchmark datasets with scale factors of ×2 and ×4 (including the building-specific dataset Urban100) are reported in Table 1. In addition to the PSNR / SSIM indicators, we also list the number of parameters (#params) and floating-point calculations (#FLOPs), where #params is related to memory consumption and #FLOPs is related to energy usage. #FLOPs is calculated by the fvcore1 library after super-resolving the LR image into a 1280×720 HR image. Table 1 and Figure 6 The results demonstrate that our proposed Mamba lightweight building image super-resolution reconstruction method based on the state equation model achieves a good trade-off between model complexity and reconstruction performance.

[0097] To further illustrate the effectiveness of our method, we also compare the visual effects of ×4 magnification on the Urban100 dataset. Figure 7As shown in the figure, most of the compared lightweight super-resolution methods cannot correctly restore the texture and details of building images, and there are obvious blurring artifacts and distortion. In contrast, the Mamba lightweight building image super-resolution reconstruction method based on the state equation model proposed by us can obtain the best visual quality and restore more building structure detail information.

[0098] In addition, in order to verify the effectiveness of our designed Mamba feature modulation block (MFM) and local feature enhancement module (LFRM), we further conducted ablation experiments on MFM and LFRM. The results in Tables 2 and 3 show that all components of MFM and LFRM are the key to performance improvement. The results in Table 4 show that our scanning mechanism has significant advantages in both performance and computational efficiency, proving its superiority and stability in feature extraction tasks, far exceeding other scanning strategies.

[0099] Table 1: PSNR / SSIM quantitative evaluation of the super-resolution reconstruction performance of the proposed method on four related datasets and performance comparison with other existing methods (“_” and “_” mark the best / second best performance), where Urban100 is a building measurement Test set (including 100 city landscape images)

[0100]

[0101] Table 2: Ablation experimental verification of the method of the present invention on MFM and LFRM

[0102]

[0103] Table 3: Ablation experiment verification of LFRM module performance in the method of the present invention

[0104]

[0105] Table 4: Ablation experiment verification of the performance of the MFM module in the method of the present invention on images in different directions

[0106]

[0107] The above are preferred embodiments of the present invention. Any changes made according to the technical solution of the present invention, as long as the resulting functions do not exceed the scope of the technical solution of the present invention, belong to the protection scope of the present invention.

Claims

1. The Mamba lightweight building image super-resolution reconstruction method based on the state equation model is characterized by: The steps include: Step S1: Obtain a high-definition image of the building, and use a bicubic interpolation algorithm to obtain a low-resolution image corresponding to the lightweight image dataset; Step S2: perform data enhancement on the input low-resolution data; Step S3: constructing a Mamba lightweight building image super-resolution reconstruction network based on a state equation model, namely, a MambaESR network model, which is specifically divided into a Mamba feature modulation network MFM and a local feature enhancement module LFRM; Step S4: training the MambaESR network model constructed in step S3; Step S5: Input the test set into the MambaESR network model trained in step S4 and output a super-resolution image.

2. The Mamba lightweight building image super-resolution reconstruction method based on the state equation model according to claim 1 is characterized in that: In step S1, the high-resolution image of the building is downsampled using a bicubic interpolation algorithm to obtain a corresponding low-resolution image.

3. The Mamba lightweight building image super-resolution reconstruction method based on the state equation model according to claim 1 is characterized in that: In step S2, data enhancement is performed by scaling, flipping, and rotating at different scales to improve the generalization ability of the model.

4. The Mamba lightweight building image super-resolution reconstruction method based on the state equation model according to claim 1 is characterized in that: The Mamba feature modulation network described in step S3 adopts a network architecture based on a state equation model to replace the traditional CNN and Transformer architectures in view of the fact that building images have a large dependence on long-range information, so as to reduce the computational cost and increase the receptive field. The specific design process of the visual state equation model is as follows: Step S3.1: First, perform channel segmentation on the normalized input features. Given the input feature X, this process is expressed as: [X0,X1]=CHUNK(X) (1) Among them, CHUNK(·) is the channel segmentation operation, [X0, X1] is the two-part feature after the input feature is segmented; Step S3.2: In the first branch, the number of feature channels is expanded to γC through linear mapping, where γ is a predefined channel expansion factor, and then transformed through depthwise convolution, SiLU activation function, state space module SSM layer and layer normalization LayerNorm, expressed as: Among them, DW-Conv 3×3 (·) represents a depth-wise separable 3×3 convolution kernel, φ(·) represents a GELU activation function, Linear(·) represents a linear mapping operation, SSM(·) represents a state-space module, and LN(·) represents a layer normalization operation. is the intermediate feature of the first branch after the above operation; Step S3.3: In the second branch, the feature channel is also expanded to γC through the linear layer and processed by the SiLU activation function; the feature representations of the two branches are fused by element-wise product, expressed as: X2=φ(Linear(X2)) (3) Where X2 is the output feature of the second branch; Step S3.4: The number of feature channels is mapped back to C to generate the output X out , whose shape is consistent with the input, this process can be expressed as: X out =Linear(X1×X2) (4)。 5. The Mamba lightweight building image super-resolution reconstruction method based on the state equation model according to claim 1 is characterized in that: Step S3 uses a variety of different image scanning methods to expand and flatten an image into a one-dimensional image in different ways, thereby obtaining global information of the image without adding additional burden.

6. The Mamba lightweight building image super-resolution reconstruction method based on the state equation model according to claim 1 is characterized in that: The specific design process of the local feature enhancement module is as follows: Step S3.4: In the local feature enhancement module, the number of channels of the input feature is first doubled using a 1×1 convolution. This process is expressed as: X'=Conv 1×1 (X in ) (5) Step S3.5: Introduce a depthwise separable convolution kernel to encode spatial local context information, and add a self-residual to the depthwise convolution kernel to improve the reconstruction result; then apply the GELU function to the hidden layer for nonlinear mapping, pass its result through an SE attention module, and finally reduce the channel to the original input dimension through 1×1 convolution to obtain the output feature. This process is expressed as: X out =Conv 1×1 (SE(φ(DWConv 3×3 (X')+X))) (6) Among them, X in and X out Represents input features and output features respectively, DWConv 3×3 represents a depth-wise separable 3×3 convolution kernel, SE module represents a spatial attention mechanism module, and φ(·) represents a GELU activation function.

7. The Mamba lightweight building image super-resolution reconstruction method based on the state equation model according to claim 1 is characterized in that: The specific process of step S4 is as follows: Step S4.1: Input a low-resolution image of a building, generate a super-resolution image through the network model, calculate the regularization loss between the reconstructed image and the real image, use the mean absolute error loss and the FFT-based frequency loss function to optimize the network parameters, and continuously train the super-resolution reconstruction network through back-propagation to perform training and learning to obtain the final super-resolution reconstruction network after training; the loss function is defined as: Loss=||I SR -I HR ||1+γ||F(I SR )-F(I HR )||1 (7) Among them I HR is a high-resolution ground-truth image, ||·||1 represents the L1 norm, F represents the fast Fourier transform, and γ is a weight factor, which is empirically set to 0.1; Step S4.2: Design and train three super-resolution networks of different sizes, named MambaESR-S, MambaESR-M, and MambaESR-L respectively; MambaESR-S has a dimension of 36 and is stacked with 8 modules; MambaESR-M has a dimension of 48; and MambaESR-L has a dimension of 48 and is composed of 12 modules; during training, each batch contains 16 LR images and the input size is 256×256; the Adam optimizer with β1=0.9 and β2=0.99 is used to solve the proposed model; the initial learning rate is set to 1×10 -3 , the minimum value is set to 1×10 -6 , the learning rate is updated by the cosine annealing algorithm; the weight of the exponential moving average EMA is set to 0.999; during the implementation, the model uses a total of 10 6 Training is performed for iterations.

Images