Lightweight building image super-resolution reconstruction method based on expansion feature modulation network

Abstract

The invention discloses a lightweight building image super-resolution reconstruction method based on an expansion feature modulation network, and the method specifically comprises the following steps: 1, obtaining a building high-resolution image, carrying out the down-sampling of an original high-resolution image, and obtaining a corresponding low-resolution image; 2, performing data enhancement on the low-resolution image; 3, constructing a lightweight building image super-resolution network model based on the expansion feature modulation network; 4, training the model to obtain a lightweight building image super-resolution reconstruction model based on the expansion feature modulation network; according to the method, information such as texture details of a low-resolution building image is effectively recovered, the defect in the aspect of remote dependency of existing lightweight image super-resolution network modeling is overcome, and accurate and efficient long-range dependency modeling is carried out in a global feature range under the condition that calculation overhead is not increased; and the lightweight performance and the reconstruction quality of the building image super-resolution model are effectively improved.

Description

Technical Field

[0001] The present invention relates to the technical field of super-resolution reconstruction of building images in the field of vision-based structural health monitoring, and in particular to a lightweight building image super-resolution reconstruction method based on an expansion feature modulation network. Background Art

[0002] Structural health monitoring is of great significance for ensuring the safety, reliability and durability of various engineering structures. As a non-contact monitoring method, visual measurement methods are used in dynamic and static displacement measurement, crack measurement and other application scenarios of structures, and are playing an increasingly important role in the field of structural health monitoring. In the visual measurement process of structural health monitoring, due to the limitations of measurement environment, equipment cost and other factors, the acquired images often have the problem of insufficient resolution. Super-resolution reconstruction technology aims to reconstruct high-resolution (HR) images from corresponding low-resolution (LR) images, providing an effective way to solve this problem. However, building structures often have regular texture features, so their super-resolution reconstructed images are sensitive to artifact noise, and their reconstruction quality depends particularly on the network's ability to model long-range relationships. The present invention proposes a lightweight and efficient super-resolution reconstruction method for building images, which includes the following features: an adaptive dilated feature modulation network is designed, multiple dilated convolutions are superimposed with equivalent large convolution kernels to obtain the effect of a large receptive field while reducing the computational cost, and a spatial attention mechanism is implemented through simple element-by-element multiplication operations, so that the model can adaptively focus on the key features in the image; further, a local feature enhancement module based on re-parameterization operations is designed to reduce the network inference time, and finally a good trade-off between performance and efficiency of super-resolution reconstruction of building images with regular textures is achieved.

[0003] At present, there is no lightweight building image super-resolution method based on expansion feature modulation network in publicly published literature and patents. Summary of the invention

[0004] In view of this, the purpose of the present invention is to provide a lightweight building image super-resolution reconstruction method based on an expansion feature modulation network, which solves the limitations of existing lightweight super-resolution models in modeling long-range dependencies, and realizes accurate and efficient long-range dependency modeling within the global feature range without increasing the number of parameters, thereby effectively improving the robustness and reconstruction quality of the lightweight image super-resolution model.

[0005] To achieve the above object, the present invention adopts the following technical solution: a lightweight building image super-resolution reconstruction method based on an expansion feature modulation network, comprising the following steps:

[0006] Step S1: obtaining a high-resolution image of the building surface, and downsampling the original high-resolution image of the building to obtain a corresponding low-resolution image;

[0007] Step S2: perform data enhancement on the low-resolution image;

[0008] Step S3: In view of the characteristics of regular texture and large long-range dependence in building images, a lightweight building image super-resolution network model based on dilation feature modulation network is constructed, which is specifically divided into dilation separable modulation unit DSMU and local feature enhancement module LFEM based on re-parameterization operation, aiming to restore high-resolution images from low-resolution winter images of buildings;

[0009] Step S4: training the lightweight building image super-resolution network model based on the dilation feature modulation network constructed in step S3;

[0010] Step S5: Input the image test set to be super-resolved into the lightweight building image super-resolution network model based on the dilation feature modulation network trained in step S4, and output the super-resolved image.

[0011] In a preferred embodiment, the specific process of step S1 is: using the bicubic interpolation algorithm to process the original high-resolution image of the building to obtain a corresponding low-resolution image; according to the super-resolution magnification, the high-resolution image HR is cut into blocks corresponding to the low-resolution image block I LR The corresponding high-resolution image block is denoted as I HR .

[0012] In a preferred embodiment: in step S2, data enhancement is performed using different scale scaling, blurring, flipping, rotating and CutBlur methods to improve the generalization ability of the model.

[0013] In a preferred embodiment: the dilated separable modulation unit DSMU described in step S3 aggregates spatial information from different receptive fields in view of the large long-range dependency of building images, thereby achieving better modeling of long-range feature relationships; in particular, multiple dilated convolutions are superimposed with equivalent large convolution kernels to obtain the effect of a large receptive field while reducing the computational cost; the specific design process of the dilated separable modulation unit is as follows:

[0014] Step S3.1: First, perform channel segmentation on the normalized input features. Given the input feature X, this process is expressed as:

[0015] [X0,X1,X2,X3]=CHUNK(X) (1)

[0016] Among them, CHUNK(·) is the channel segmentation operation, [X0, X1, X2, X3] is the four-part feature after the input feature is segmented;

[0017] Step S3.2: Use multiple dilated small convolution kernels to obtain the effect of a large convolution kernel, whose hyper parameters include convolution kernel size k, dilation rate d and reconstruction kernel size K; the equivalent process in the dilated separable modulation unit is as follows:

[0018] k=(3,3),d=(1,2)→K=5,k=(5,3,3),d=(1,2,3)→K=7,k=(5,9,3,3,3),d=(1,2,4,5,7)→K=17,k=(5,7,3,3,3),d=(1,2,3,4,5)→

[0019] K = 13; the input feature X is passed into the dilated separable modulation unit for multi-scale feature extraction; finally, these multi-scale features are connected and 1×1 convolution is used to aggregate information of different scales. This process can be expressed as:

[0020]

[0021] DW-Conv n×n (·) represents the deep convolution kernel component for multi-scale feature extraction, where n is 5, 7, 13, and 17; Concat(·) represents the concatenation operation along the channel dimension, Conv 1×1 (·) is a 1×1 convolution, Represents the features after multi-scale feature extraction, Represents different scale features obtained by four different scores n, Represents the result of aggregating features of different scales;

[0022] Step S3.3: After obtaining the aggregated multi-scale features, we normalize them through the GELU nonlinearity and then add them to the input feature X to generate the final output. This process is expressed as:

[0023]

[0024] Among them, φ(·) represents the GELU activation function, which is the element-wise product.

[0025] In a preferred embodiment: the specific design process of the local feature enhancement module LFEM based on the reparameterization operation is:

[0026] Step S3.4: In LFEM, a 1×1 convolution is first used to double the number of channels of the input features. This process is expressed as:

[0027] X'=Conv1×1 (X in ) (5)

[0028] Step S3.5: Then, a variety of multi-branch structures are introduced in parallel with 3×3 deep convolution to encode spatial local context information. The GELU function is then applied to the hidden layer for nonlinear mapping, and the result is passed through an SE attention module. Finally, the channel is reduced to the original input dimension through 1×1 convolution to obtain the output feature. This process is expressed as:

[0029] X out =Conv 1×1 (SE(φ(Rep Conv3×3 (X')))) (6)

[0030] Among them, X in and X out Represents input features and output features respectively, Rep Conv3×3 It represents the multi-branch structure which is equivalent to a 3×3 convolution kernel using the reparameterization technique, SE represents the spatial attention mechanism module, and φ(·) represents the GELU activation function.

[0031] In a preferred embodiment: the specific process of step S4 is:

[0032] Step S4.1: First, define the loss function. We optimize the parameters of our DCFMN by using the mean absolute error loss and the FFT-based frequency loss function. This process is expressed as:

[0033] L=||I SR -I HR ||1+γ||F(I SR )-F(I HR )||1 (7)

[0034] Among them I HR is a high-resolution ground-truth image, ||·||1 represents the L1 norm, F represents the fast Fourier transform, and γ is the weight factor;

[0035] Step S4.2: Train two different sizes of dilated feature modulation networks, named DCFMN-S and DCFMN-L, where DCFMN-S uses 10 modules and DCFMN-L uses 16 modules; during training, each batch contains 32 LR images and the input size is 256×256; use the Adam optimizer with β1=0.9 and β2=0.99 to solve the proposed model; set the initial learning rate to 1×10 -3 , the minimum value is set to 1×10 -6, the learning rate is updated by the cosine annealing algorithm; the weight of the exponential moving average EMA is set to 0.999; the model uses a total of 10 6 Training is performed for iterations.

[0036] Compared with the prior art, the present invention has the following beneficial effects: 1. The present invention proposes a lightweight dilated feature modulation network, which achieves a good balance between performance and efficiency through an adaptive dilated feature modulation mechanism, and overcomes the trade-off problem between computational complexity and reconstruction performance in traditional super-resolution models. 2. The present invention designs a dilated separable modulation unit (DSMU), which is equivalent to a large convolution kernel through multiple dilated convolutions, and aggregates spatial information from different scales at a lower computational cost, thereby better modeling the long-range feature relationship of building images and significantly improving the detail expression of image reconstruction. 3. The present invention designs a local feature enhancement module (LFEM) based on reparameterization operation, which can not only effectively encode and enhance local contextual information, but also incorporate more learnable parameters in the training stage to improve model performance. In the inference stage, parameters are merged through reparameterization to avoid additional computational overhead, thereby improving the SR performance of the model without increasing the inference burden. BRIEF DESCRIPTION OF THE DRAWINGS

[0037] Figure 1 It is an overall flow chart of a lightweight building image super-resolution method based on an expansion feature modulation network according to a preferred embodiment of the present invention;

[0038] Figure 2 A comparison diagram of local attribution maps (LAMs) and diffusion indices (DIs) of the preferred embodiment of the present invention and other efficient super-resolution models;

[0039] Figure 3 It is the overall framework diagram of the lightweight expansion feature modulation network of the preferred embodiment of the present invention;

[0040] Figure 4 The design diagram of the dilation separable modulation unit and the local feature enhancement module based on the re-parameterization operation according to the preferred embodiment of the present invention;

[0041] Figure 5 It is an equivalent diagram of a re-parameterized block of a local feature enhancement module based on a re-parameterization operation according to a preferred embodiment of the present invention;

[0042] Figure 6 A schematic diagram showing the performance and model complexity comparison of the present invention and the existing algorithms in performing ×4 super-resolution (SR) tasks on a benchmark dataset;

[0043] Figure 7A comparison diagram is shown for the details of the proposed method and the existing algorithms on the ×4 super-resolution (SR) task on architectural images. DETAILED DESCRIPTION

[0044] The present invention will be further described below in conjunction with the accompanying drawings and embodiments.

[0045] It should be noted that the following detailed descriptions are illustrative and are intended to provide further explanation of the present application. Unless otherwise specified, all technical and scientific terms used herein have the same meanings as those commonly understood by those skilled in the art to which the present application belongs.

[0046] It should be noted that the terms used herein are only for describing specific embodiments and are not intended to limit the exemplary embodiments according to the present application; as used herein, unless the context clearly indicates otherwise, the singular form is also intended to include the plural form. In addition, it should be understood that when the terms "comprise" and / or "include" are used in this specification, they indicate the presence of features, steps, operations, devices, components and / or their combinations.

[0047] The present invention proposes a lightweight image super-resolution reconstruction method based on an expansion feature modulation network. Figure 1-7 As shown. First, the low-resolution image block I LR Input to the network, output super-resolution result image I SR .

[0048] Follow the steps below to implement it:

[0049] Step S1: obtaining a high-resolution building image, and downsampling the original high-resolution building image to obtain a corresponding low-resolution image;

[0050] The specific process of downsampling is: using the bicubic interpolation algorithm to process the high-resolution image to obtain the corresponding low-resolution image. According to the super-resolution magnification, the high-resolution image HR is cut into blocks corresponding to the low-resolution image I LR The corresponding high-resolution image block is denoted as I HR .

[0051] Step S2: perform data enhancement on the low-resolution image;

[0052] Specific data enhancement operations include scaling, blurring, flipping, rotating, and CutBlur to improve the generalization ability of the model.

[0053] Step S3: In view of the characteristics of regular texture and large long-range dependence in building images, a lightweight image super-resolution network model based on dilation feature modulation network is constructed, which is specifically divided into two modules: dilation separable modulation unit and local feature enhancement module based on re-parameterization operation, such as Figure 3 and Figure 4 As shown;

[0054] See Figure 3 ,The expansion feature modulation network proposed in this paper consists of three modules: shallow ,feature extraction, deep feature extraction and upsampling feature reconstruction.

[0055] Step S3.1: Specifically, the network first applies a convolutional layer with a kernel size of 3x3 pixels to extract shallow features, convert the input LR image into feature space, and generate shallow features f0. The specific process can be expressed as:

[0056] f0=Conv 3×3 (I LR ) (1)

[0057] Among them, Conv 3×3 (·) indicates a 3×3 convolutional layer.

[0058] Step S3.2: pass the shallow feature f0 to the deep feature extraction module composed of a series of dilated separable modulation blocks (DSMBs). The specific process can be expressed as:

[0059] f k =H k (f k-1 ),k=1,…,n (2)

[0060] Among them, H k (·) indicates the kth DSMB. k-1 and f k They represent the input features and output features of the kth DSMB respectively.

[0061] Step S3.3: In order to better restore the HR target image, a global residual connection is introduced. Finally, a lightweight upsampling layer consisting of a 3×3 convolution layer and a sub-pixel convolution is used to quickly reconstruct the HR image. The specific process can be expressed as:

[0062] I SR =P θ (f k +f0) (3)

[0063] Step S3.4: Long-term context information involves more feature interactions. In order to reconstruct more refined results in dense pixel prediction, the present invention reconsiders the design of the basic feature extraction unit in the deep feature extraction module. Different from the self-attention mechanism, traditional feature pyramid, and large kernel convolution, the present invention proposes a lighter-weight alternative, the dilated separable modulation unit learns long-term dependencies from different scale feature representations, so as to better explore useful feature information for architectural image reconstruction. Figure 4 As shown, the specific design process is as follows.

[0064] Step S3.4.1: First, perform channel segmentation on the normalized input features to generate four-part features, and then pass the features into the dilated separable modulation unit (DSMU) for multi-scale feature extraction. Given the input feature X, this process can be expressed as:

[0065] [X0,X1,X2,X3]=CHUNK(X) (4)

[0066]

[0067] Among them, CHUNK(·) is the channel segmentation operation, DW-Conv n×n (·) represents the deep convolution kernel component for multi-scale feature extraction, where the values ​​of n are 5, 7, 13, and 17.

[0068] Step S3.4.2: For a pixel on the feature map, some distant pixels may be more relevant than its neighboring pixels. Therefore, in order to reduce the computational burden, the present invention uses the properties of ordinary convolution to be equivalent to a large convolution kernel, and uses multiple expanded small convolution kernels to replace a large convolution kernel to extract features. Its hyperparameters include the convolution kernel size k, the expansion rate d, and the reconstruction kernel size K. Figure 5 In the case shown, k = (5, 7, 3, 3, 3), d = (1, 2, 3, 4, 5) are used to generate a convolution kernel with a kernel size of K = 13. The remaining equivalent processes in the dilated separable modulation unit are as follows: k = (3, 3), d = (1, 2) → K = 5, k = (5, 3, 3), d = (1, 2, 3) → K = 7, k = (5, 9, 3, 3, 3), d = (1, 2, 4, 5, 7) → K = 17. The results in Table 3 show that this operation improves performance while reducing the number of parameters and floating point operations (FLOPs).

[0069] Step S3.4.3: Finally, we connect these multi-scale features and aggregate different scale information through 1×1 convolution. This process can be expressed as:

[0070]

[0071] Where Concat(·) represents the concatenation operation along the channel dimension, Conv 1×1 (·) is a 1×1 convolution.

[0072] Step S3.4.4: After obtaining the aggregated multi-scale features, we normalize them through the GELU nonlinearity and then add them to the input features to generate the final output. This process can be expressed as:

[0073]

[0074] Among them, φ(·) represents the GELU function, which is the element-wise product.

[0075] Step S3.5: Although the dilation separable modulation unit can model the long-range relationship between pixels by extracting and aggregating information of different scales, the importance of local context information to super-resolution image reconstruction is ignored. Therefore, the present invention designs a local feature enhancement module (LFEM) added after the dilation separable modulation unit to supplement the local context information. The specific design of LFEM is as follows.

[0076] Step S3.5.1: Figure 4 As shown in Figure 2, in LFEM, a 1×1 convolution is first used to double the number of channels of the input features. This process can be expressed as:

[0077] X'=Conv 1×1 (X in ) (8)

[0078] Step S3.5.2: Then introduce a variety of multi-branch structures in parallel with 3×3 deep convolution to encode spatial local context information, then apply the GELU function to the hidden layer for nonlinear mapping, pass its result through an SE attention module, and finally reduce the channel to the original input dimension through 1×1 convolution to obtain the output feature. This process can be expressed as:

[0079] X out =Conv 1×1 (SE(φ(Rep Conv3×3 (X')))) (9)

[0080] Among them, X in and X out Represents input features and output features respectively, Rep Conv3×3 It represents the multi-branch structure using the reparameterization technique to be equivalent to a 3×3 convolution kernel, the SE module represents the spatial attention mechanism module, and φ(·) represents the GELU activation function.

[0081] Step S3.6: Finally, design the overall architecture, such as Figure 3As shown, the dilation feature modulation block (DSMB) consists of two parts: DSMU and LFEM. This process can be expressed as:

[0082] X'=DSMU(LN(X in ))+X (10)

[0083] X out =LFEM(LN(X'))+X' (11)

[0084] Among them, X in and X out They are respectively represented as the input features and output feature maps of the dilated feature modulation block, LN(·) is the LayerNorm operation, and X' represents the intermediate features.

[0085] Step S4: training the network model constructed in step 3;

[0086] Step S4.1: First, define the loss function. This method optimizes our network parameters by using the mean absolute error loss and the FFT-based frequency loss function. This process can be expressed as:

[0087] L=||I SR -I HR ||1+γ||F(I SR )-F(I HR )||1 (12)

[0088] Among them I HR is a high-resolution ground-truth image, ||·||1 represents the L1 norm, F represents the fast Fourier transform, and γ is the weight factor.

[0089] S4.2: We train two models of different sizes, named DCFMN-S and DCFMN-L. DCFMN-S uses 10 modules, while DCFMN-L uses 16 modules. During training, each batch contains 32 LR images with an input size of 256×256. We use the Adam optimizer with β1=0.9 and β2=0.99 to solve the proposed model. We set the initial learning rate to 1×10 -3 , the minimum value is set to 1×10 -6 , the learning rate is updated by the cosine annealing algorithm. The weight of the exponential moving average (EMA) is set to 0.999. In the implementation process, the model of the present invention uses a total of 10 6 Training for iterations

[0090] Step S5: Input the test set into the network model trained in step 3 and output the super-resolution image.

[0091] To evaluate the performance of the present invention, we compare it with the most advanced lightweight SR methods, including SRCNN, VDSR, IMDN, EDSR-baseline, LAPAR, PAN, RFDN, ShuffleMixer, HPUN, HNCT, RepECN-S, SAFMN. The quantitative comparison results on the benchmark datasets with scale factors of ×2, ×3 and ×4 are reported in Table 1. In addition to the PSNR / SSIM indicators, we also list the number of parameters (#params) and floating-point calculations (#FLOPs), where #params is related to the memory consumption and #FLOPs is related to the energy usage. #FLOPs is calculated by the fvcore1 library after super-resolving the LR image into a 1280×720 HR image. Table 1 and Figure 6 The results demonstrate that our proposed lightweight building image super-resolution method based on dilated feature modulation network achieves a good trade-off between model complexity and reconstruction performance.

[0092] To further illustrate the effectiveness of our method, we also compare the visual effects of ×4 magnification on the Urban100 dataset. Figure 7 As shown in the figure, most of the compared lightweight super-resolution methods cannot correctly restore the texture and details of building images, and there are obvious blurring artifacts and distortion. In contrast, the lightweight building image super-resolution reconstruction method based on the dilation feature modulation network we proposed can achieve the best visual quality and restore more building structure detail information.

[0093] In addition, in order to verify the effectiveness of our designed DSMU and LFEM, we further conduct ablation experiments on DSMU and LFEM. The results in Tables 2 and 3 show that all components of DSMU and LFEM are the key to performance improvement.

[0094] Table 1: PSNR / SSIM quantitative evaluation of the super-resolution reconstruction performance of the proposed method on four related datasets and performance comparison with other existing methods (“” and “” mark the best

[0095] / sub-optimal performance)

[0096]

[0097]

[0098] Table 2: Ablation experiment verification of DSMU module performance in the method of the present invention

[0099]

[0100] Table 3: Ablation experiment verification of LFEM module performance in the method of the present invention

[0101]

Claims

1. A lightweight building image super-resolution reconstruction method based on an expansion feature modulation network, characterized in that: The steps include: Step S1: obtaining a high-resolution image of the building surface, and downsampling the original high-resolution image of the building to obtain a corresponding low-resolution image; Step S2: perform data enhancement on the low-resolution image; Step S3: In view of the characteristics of regular texture and large long-range dependence in building images, a lightweight building image super-resolution network model based on dilation feature modulation network is constructed, which is specifically divided into dilation separable modulation unit DSMU and local feature enhancement module LFEM based on re-parameterization operation, aiming to restore high-resolution images from low-resolution winter images of buildings; Step S4: training the lightweight building image super-resolution network model based on the dilation feature modulation network constructed in step S3; Step S5: Input the image test set to be super-resolved into the lightweight building image super-resolution network model based on the dilation feature modulation network trained in step S4, and output the super-resolved image.

2. The lightweight building image super-resolution reconstruction method based on the expansion feature modulation network according to claim 1 is characterized in that: The specific process of step S1 is: using the bicubic interpolation algorithm to process the original high-resolution image of the building to obtain the corresponding low-resolution image; according to the super-resolution magnification, the high-resolution image HR is cut into blocks corresponding to the low-resolution image block I LR The corresponding high-resolution image block is denoted as I HR .

3. The lightweight building image super-resolution reconstruction method based on the expansion feature modulation network according to claim 1 is characterized in that: In step S2, data enhancement is performed by using different scale scaling, blurring, flipping, rotating and CutBlur methods to improve the generalization ability of the model.

4. The lightweight building image super-resolution reconstruction method based on the expansion feature modulation network according to claim 1 is characterized in that: The dilated separable modulation unit DSMU described in step S3 aggregates spatial information from different receptive fields in view of the large long-range dependency of building images, thereby achieving better modeling of long-range feature relationships; in particular, multiple dilated convolutions are superimposed with equivalent large convolution kernels to obtain the effect of a large receptive field while reducing the computational cost; the specific design process of the dilated separable modulation unit is as follows: Step S3.1: First, perform channel segmentation on the normalized input features. Given the input feature X, this process is expressed as: [X0,X1,X2,X3]=CHUNK(X) (1) Among them, CHUNK(·) is the channel segmentation operation, [X0, X1, X2, X3] is the four-part feature after the input feature is segmented; Step S3.2: Use multiple dilated small convolution kernels to obtain the effect of a large convolution kernel, whose hyper parameters include convolution kernel size k, dilation rate d and reconstruction kernel size K; the equivalent process in the dilated separable modulation unit is as follows: k=(3,3),d=(1,2)→K=5,k=(5,3,3),d=(1,2,3)→K=7,k=(5,9,3,3,3),d=(1,2,4,5,7)→K=17,k=(5,7,3,3,3),d=(1,2,3,4,5)→ K = 13; the input feature X is passed into the dilated separable modulation unit for multi-scale feature extraction; finally, these multi-scale features are connected and 1×1 convolution is used to aggregate information of different scales. This process can be expressed as: Among them, DW-Conv n×n (·) represents the deep convolution kernel component for multi-scale feature extraction, where n is 5, 7, 13, and 17; Concat(·) represents the concatenation operation along the channel dimension, Conv 1×1 (·) is a 1×1 convolution, Represents the features after multi-scale feature extraction, Represents different scale features obtained by four different scores n, Represents the result of aggregating features of different scales; Step S3.3: After obtaining the aggregated multi-scale features, we normalize them through the GELU nonlinearity and then add them to the input feature X to generate the final output. This process is expressed as: Among them, φ(·) represents the GELU activation function, which is the element-wise product.

5. The lightweight building image super-resolution reconstruction method based on dilation feature modulation network according to claim 1 is characterized in that: The specific design process of the local feature enhancement module LFEM based on reparameterization operation is as follows: Step S3.4: In LFEM, a 1×1 convolution is first used to double the number of channels of the input features. This process is expressed as: X'=Conv 1×1 (X in ) (5) Step S3.5: Then, a variety of multi-branch structures are introduced in parallel with 3×3 deep convolution to encode spatial local context information. The GELU function is then applied to the hidden layer for nonlinear mapping, and the result is passed through an SE attention module. Finally, the channel is reduced to the original input dimension through 1×1 convolution to obtain the output feature. This process is expressed as: X out =Conv 1×1 (SE(φ(Rep Conv3×3 (X')))) (6) Among them, X in and X out Represents input features and output features respectively, Rep Conv3×3 It represents the multi-branch structure which is equivalent to a 3×3 convolution kernel using the reparameterization technique, SE represents the spatial attention mechanism module, and φ(·) represents the GELU activation function.

6. The lightweight building image super-resolution reconstruction method based on dilation feature modulation network according to claim 1 is characterized in that: The specific process of step S4 is as follows: Step S4.1: First, define the loss function. We optimize the parameters of our DCFMN by using the mean absolute error loss and the FFT-based frequency loss function. This process is expressed as: L=||I SR -I HR ||1+γ||F(I SR )-F(I HR )||1 (7) Among them I HR is a high-resolution ground-truth image, ||·||1 represents the L1 norm, F represents the fast Fourier transform, and γ is the weight factor; Step S4.2: Train two different sizes of dilated feature modulation networks, named DCFMN-S and DCFMN-L, where DCFMN-S uses 10 modules and DCFMN-L uses 16 modules; during training, each batch contains 32 LR images and the input size is 256×256; use the Adam optimizer with β1=0.9 and β2=0.99 to solve the proposed model; set the initial learning rate to 1×10 -3 , the minimum value is set to 1×10 -6 , the learning rate is updated by the cosine annealing algorithm; the weight of the exponential moving average EMA is set to 0.999; the model uses a total of 10 6 Training is performed for iterations.

Images