Lightweight image super-resolution reconstruction method and system, and storage medium

Abstract

This invention discloses a lightweight image super-resolution reconstruction method, system, and storage medium, relating to the field of image processing technology. Specifically, it includes: acquiring high-resolution and low-resolution images of the target as training sample pairs; constructing a lightweight image super-resolution reconstruction model comprising a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, and an image reconstruction module. The shallow feature extraction module performs shallow processing on the low-resolution image; the deep feature extraction module consists of multiple cascaded enhanced separable residual feature refinement blocks, extracting shallow features to obtain multiple deep features; the multi-layer feature fusion module is used to obtain fused features; and the image reconstruction module is used to output the high-resolution image. The trained model is used to reconstruct the low-resolution image to be processed. This invention has a simple network structure and achieves accurate reconstruction of image texture and details using only a low number of parameters and computational cost.

Description

Technical Field

[0001] This invention relates to the field of image processing technology, and in particular to a lightweight image super-resolution reconstruction method, system, and storage medium. Background Technology

[0002] In the field of image super-resolution reconstruction methods, to pursue the accuracy and clarity of reconstructed high-resolution images, a series of models such as VGG, ResNet, DenseNet, ResNext, SE-Net, and DPN have been designed. These models increase network depth and channel width, thus increasing computational complexity while improving performance. However, in some real-world applications, such as mobile portable or embedded devices, such large and complex models are difficult to deploy, severely limiting the development of the technology. To address this issue, existing research has shifted to lightweight image super-resolution reconstruction methods, improving traditional convolutional neural network structures. Models such as DRRN, IDN, RFDN, and SPAN still rely on increasing the number of operations and inference time to reduce model size, or use several 1 × 1 standard convolution operations and a cascaded operation as a feature extraction structure, resulting in a significant deterioration in inference speed. Furthermore, relying solely on traditional convolutional operations often fails to effectively aggregate global information and is lacking in the modeling of spatial and channel features, preventing some lightweight models from achieving reconstruction results similar to traditional models. Summary of the Invention

[0003] Based on the technical problems existing in the background technology, this invention proposes a lightweight image super-resolution reconstruction method, system and storage medium. It adopts a simple network structure to accurately reconstruct the texture and high-frequency details of the image, and achieves a better balance between reconstruction performance and computational cost in the image super-resolution reconstruction method.

[0004] This invention proposes a lightweight image super-resolution reconstruction method, characterized by the following steps: Acquire the low-resolution image to be processed; Inputting a low-resolution image into a trained lightweight image super-resolution reconstruction network yields the corresponding high-resolution image. The lightweight image super-resolution reconstruction network includes a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, and an image reconstruction module. The shallow feature extraction module is used to perform blueprint-separable convolution processing on low-resolution images to obtain shallow features; The deep feature extraction module is used to extract deep features from shallow features step by step, resulting in multiple deep features; The multi-layer feature fusion module is used to stitch together and fuse multiple deep features to obtain fused features; The image reconstruction module is used to reconstruct images based on fused features and output high-resolution images.

[0005] Preferably, the deep feature extraction module consists of N cascaded enhanced separable residual feature refinement modules, which extract deep features step by step from shallow features to obtain multiple deep features; The first enhanced separable residual feature refinement module processes the shallow features to obtain the first deep features; The i-th enhanced separable residual feature refinement module processes the deep features output by the (i-1)-th enhanced separable residual feature refinement module to obtain the i-th deep feature.

[0006] Preferably, each enhanced separable residual feature refinement module includes three stacked blueprint separable feature refinement modules, residual connections, a lightweight dual attention module, and a BSConvFormer layer based on large kernel blueprint separable convolution; The blueprint separable feature refinement module is used to refine and extract input features to obtain refined features; A lightweight dual attention module is used to enhance the refined features in both the channel and spatial dimensions to obtain enhanced features; The BSConvFormer layer, based on large kernel blueprint separable convolution, is used to perform large receptive field feature mixing and feedforward transformation on the enhanced features to obtain the corresponding deep features.

[0007] Preferably, each blueprint separable feature refinement module (BFB) includes a 3×3 blueprint separable convolutional layer and a GELU activation function layer; Blueprints can be used to separate convolutional layers to extract convolutional features from the input features, and the extracted feature maps are then transformed by activation function layers to obtain refined features.

[0008] Preferably, the lightweight dual attention module includes a lightweight contrast perception channel attention submodule and a lightweight enhanced spatial attention submodule; The lightweight contrast-aware channel attention submodule is used to assign channel attention weights based on the contrast information represented by the sum of the mean and standard deviation of the channel features of the input feature map, and to obtain channel attention enhancement features using two layers of blueprint separable convolution and sigmoid function. The lightweight enhanced spatial attention submodule is used to generate a spatial attention matrix through convolutional dimensionality reduction, stride convolution and pooling downsampling, convolutional feature extraction and convolutional channel recovery, and then multiply the spatial attention matrix with the channel attention enhancement features to obtain the enhanced features.

[0009] Preferably, the BSConvFormer layer based on large kernel blueprint separable convolution includes a large kernel blueprint separable feature mixer and an edge-preserving feedforward neural network; The large kernel blueprint separable feature mixer includes a first branch, a second branch, and a fusion branch. The first branch includes a 1×1 standard convolution and a blueprint separable large kernel convolution. The second branch includes a 1×1 standard convolution. The fusion branch includes a 1×1 standard convolution for fusing the outputs of the first branch and the second branch. The edge-preserving feedforward neural network consists of two 1×1 convolutional layers, a 3×3 edge-preserving blueprint separable convolution, and a GELU activation function layer.

[0010] Preferably, the multi-layer feature fusion module is used to stitch together multiple deep features, and then pass them sequentially through a 1×1 standard convolution and a 3×3 blueprint separable convolution to obtain fused features; The image reconstruction module is used to perform blueprint-separable convolution processing on the fused features and combine them with shallow features to perform subpixel convolution reconstruction to obtain a high-resolution image.

[0011] Preferably, the method further includes: Obtain training sample pairs of low-resolution and high-resolution images; The L1 loss function is used to construct a lightweight image super-resolution reconstruction network based on sample pairs; The learnable parameters in the lightweight image super-resolution reconstruction network are updated using the backpropagation algorithm until the loss converges, thus obtaining the trained lightweight image super-resolution reconstruction network.

[0012] The present invention proposes a lightweight image super-resolution reconstruction system, characterized in that it comprises: The image acquisition unit acquires the low-resolution image to be processed. The image processing unit inputs the low-resolution image into the trained lightweight image super-resolution reconstruction network to obtain the corresponding high-resolution image. The lightweight image super-resolution reconstruction network includes a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, and an image reconstruction module. The shallow feature extraction module is used to perform blueprint-separable convolution processing on low-resolution images to obtain shallow features; The deep feature extraction module is used to extract deep features from shallow features step by step, resulting in multiple deep features; The multi-layer feature fusion module is used to stitch together and fuse multiple deep features to obtain fused features; The image reconstruction module is used to reconstruct images based on fused features and output super-resolution images.

[0013] The present invention proposes a computer-readable storage medium storing a computer program thereon, characterized in that, when the computer program is executed by a processor, the processor performs a lightweight image super-resolution reconstruction method as described in any one of claims 1-8.

[0014] Beneficial technical effects of the present invention: (1) In the deep feature extraction part, the present invention designs a blueprint separable feature refinement module (BFB). Compared with the feature distillation method commonly used in existing lightweight super-resolution networks that relies on multiple 1×1 convolutions, channel segmentation and cascaded aggregation, the present invention adopts a compact connection form of a 3×3 blueprint separable convolution and a GELU activation function in the BFB, and removes the residual connections inside the module, which reduces the additional operational burden caused by intermediate branch switching, feature splitting and aggregation, thereby helping to reduce the number of network parameters, floating-point calculations and inference latency; at the same time, the stacking of three BFBs can progressively refine and extract local texture information, so that the network still has good detail recovery ability while maintaining lightweight, which is especially beneficial to the layer-by-layer enhancement of edge texture and high-frequency information.

[0015] (2) This invention sets up a lightweight dual attention module (LBDA) after BFB, which concatenates lightweight contrast-aware channel attention and lightweight enhanced spatial attention, and replaces the standard convolution in the attention structure with blueprint separable convolution as much as possible, thereby completing the important feature selection of both channel and spatial dimensions with lower computational cost. Specifically, lightweight contrast-aware channel attention can improve the attention to channels with strong effective texture response by modeling the contrast information reflected by the mean and standard deviation of channel features; lightweight enhanced spatial attention can further highlight key spatial regions in the image and suppress background redundancy and invalid responses. Thus, this invention not only makes up for the shortcomings of traditional convolutional networks in joint channel and spatial modeling, but also avoids the problem of excessive model expansion after introducing heavy attention structures, enabling the network to more accurately preserve target texture, contour and structural information under limited resource conditions.

[0016] (3) This invention further designs a BSConvFormer layer (BCFL) based on large-kernel blueprint separable convolution, replacing the traditional self-attention mechanism with a large-kernel blueprint separable feature mixer, and combining it with an edge-preserving feedforward neural network (EFN) to perform larger receptive field modeling and edge-preserving transformation on the enhanced features. Among them, the 9×9 large-kernel blueprint separable convolution branch can expand the receptive field without significantly increasing the number of parameters, enhancing the aggregation ability of long-distance dependent information; the parallel 1×1 convolution branch retains the sensitivity to local linear combination features; after the two are fused, it can improve the network's ability to express overall structural information without losing local details. At the same time, the edge-preserving blueprint separable convolution in the edge-preserving feedforward neural network performs feature enhancement on edges and high-frequency regions, which helps to reduce blurring, artifacts and texture loss problems in the reconstructed image. Therefore, this invention can achieve both "global structure restoration" and "local edge preservation"—two goals that are usually difficult to satisfy simultaneously—within a lightweight network framework.

[0017] (4) This invention connects three BFBs, one LBDA, and one BCFL in a specific order, and adds residual connections at appropriate positions to form an Enhanced Separable Residual Feature Refinement Module (ESRFB). This enables the deep feature extraction process to form a hierarchical processing route of "first local progressive refinement, then enhancement of important features, and finally large-scale feature mixing." The modules of this ESRFB are complementary and synergistic in function: the front-end BFB is responsible for extracting and purifying details at a lower cost, the middle LBDA is responsible for highlighting key channels and spatial regions, and the back-end BCFL is responsible for compensating for the lack of long-distance dependency modeling in lightweight CNNs. Thus, this invention can obtain more sufficient feature expression in a relatively simple network structure, thereby achieving a better balance between reconstruction performance and computational cost. Attached Figure Description

[0018] Figure 1 This is a schematic diagram of the lightweight image super-resolution reconstruction method proposed in this invention; Figure 2 This is a schematic diagram of the structure of the enhanced separable residual feature refinement module proposed in this invention; Figure 3 This is a schematic diagram of the overall structure of the enhanced separable residual feature refinement module proposed in this invention, as well as its constituent modules: blueprint separable feature refinement block, lightweight blueprint separable dual attention architecture, and BSConvFormer layer architecture based on large kernel blueprint separable convolution. Figure 4 The graph shows the relationship between the PSNR performance and FLOPs of the lightweight image super-resolution reconstruction method proposed in this invention and other classic reconstruction models on the Set5 dataset with a scale factor of ×4. Figure 5This image shows a comparison of the lightweight super-resolution image reconstruction method proposed in this invention with other reconstruction methods, along with the PSNR / SSIM results. Detailed Implementation

[0019] The present invention will be further explained below with reference to specific embodiments.

[0020] Example 1 This invention proposes a lightweight image super-resolution reconstruction method that can be applied to servers, desktops, laptops, mobile terminals, edge computing devices, or other electronic devices with image processing capabilities. The method is based on a lightweight image super-resolution reconstruction network, which includes a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, and an image reconstruction module. The deep feature extraction module consists of multiple cascaded Enhanced Separable Residual Feature Refinement Modules (ESRFBs), and each ESRFB consists of three Blueprint Separable Feature Refinement Modules (BFBs), residual connections, a Lightweight Dual Attention Module (LBDA), and a BSConvFormer layer (BCFL) based on a large-kernel blueprint separable convolution.

[0021] In one specific implementation, such as Figure 1 As shown, a lightweight image super-resolution reconstruction method may include the following steps: First, the low-resolution image to be processed is acquired. Specifically, the low-resolution image can be directly acquired by an image acquisition device, or it can be read from a database, cache, video stream, image file, or network interface. The low-resolution image can be a natural scene image, medical image, remote sensing image, security monitoring image, industrial inspection image, etc. This embodiment does not limit the image source. During the training phase, a high-resolution image corresponding to the low-resolution image is also acquired to form training sample pairs; during the inference phase, only the low-resolution image to be processed needs to be input.

[0022] Then, the low-resolution image is input into a lightweight image super-resolution reconstruction network to obtain the corresponding high-resolution image. In this embodiment, referencing Figure 2 A lightweight image super-resolution reconstruction network includes at least a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, and an image reconstruction module. The shallow feature extraction module performs blueprint-separable convolution processing on the low-resolution image to obtain shallow features; the deep feature extraction module extracts deep features from the shallow features at progressively deeper levels to obtain multiple deep features; the multi-layer feature fusion module concatenates and fuses these multiple deep features to obtain fused features; and the image reconstruction module reconstructs the image based on the fused features, outputting a high-resolution image.

[0023] When performing feature extraction on a low-resolution image using a shallow feature extraction module, let the input low-resolution image be I. LR ∈R C×H×W Where H and W are the length and width of the image, respectively, and C is the number of channels in the image. The shallow feature extraction module processes the initial image using the first blueprint separable convolutional layer to obtain shallow features F0∈R. C×H×W .

[0024] In this embodiment, the deep feature extraction module consists of N (N is a positive integer) cascaded enhanced separable residual feature refinement modules. When performing deep feature extraction on shallow features, the first ESRFB takes the shallow feature F0 as its input feature, and the i-th ESRFB takes the deep feature output by the (i-1)-th ESRFB as its input feature. After cascading processing, multiple deep features F1, F2, ..., F3 are obtained. N .

[0025] Furthermore, referring to Figure 3 Each enhanced separable residual feature refinement module includes three stacked Blueprint Separable Feature Refinement Modules (BFB), residual connections, a Lightweight Dual Attention Module (LBDA), and a BSConvFormer layer (BCFL) based on a large kernel Blueprint Separable Convolution.

[0026] In this embodiment, each BFB module includes a 3×3 blueprint separable convolutional layer and an activation function layer. In a preferred embodiment, the activation function is the GELU function. Unlike the internal residual connections and channel segmentation strategies commonly used in existing information distillation networks, the BFB in this embodiment eliminates internal residual connections in its structural design and does not employ channel segmentation, which splits intermediate features along the channel dimension into a retention part and a further processing part. Instead, it directly completes feature refinement through a compact structure of "single-layer 3×3 blueprint separable convolution + activation function," thereby reducing the additional overhead caused by feature splitting, feature concatenation, and multi-branch switching, and improving inference efficiency. After stacking three BFBs, the local texture and high-frequency details in the input features can be progressively refined layer by layer. To better preserve input features and suppress the gradient vanishing problem in deep training, in a specific implementation, local residual connections can be set between the input and output ends of the three BFBs and their subsequent processing units, so that the ESRFB as a whole forms a residual refinement structure.

[0027] In a BFB module, a traditional CNN convolutional layer is optimized using a 3×3 blueprint-separable convolutional layer. The kernels of CNN convolutional layers typically exhibit high intra-kernel correlation along their depth axis, which can be observed through training visualization for the convolutional filter F. (c) Its slice F1(c) F2 (c) , ......, F M (c) The blueprints displayed are identical filter-specific K×K blueprints (K being the kernel size), differing only in the multiplication factor M. The designed blueprints allow for separable convolutions consisting of C convolutional filters, each with its own blueprint. Convolutional feature extraction is performed using the following formula: In the formula, M represents M multiplication factors; m∈{1,2,......,M}; c∈{1,2,......,C}; w c,m This means that all elements in m∈{1,2,......,M} and c∈{1,2,......,C} are arranged in an ordered manner to form a two-dimensional matrix of size M×N as the kernel weights.

[0028] Taking the input shallow feature F0 as an example, the input shallow feature F0 is subjected to a blueprint-separable convolution with the kernel matrix. , ..., A 1×1 pointwise convolution is performed, followed by a K×K depthwise convolution applied to the result of the first convolution. Then, the feature map extracted by the blueprint separable convolution is transformed using the GELU activation function, and the above steps are superimposed three times to obtain the refined feature F. i BFB .

[0029] In this embodiment, a lightweight dual attention module is used to enhance the refined features in both the channel and spatial dimensions to obtain enhanced features. Preferably, the lightweight dual attention module includes a lightweight contrast-aware channel attention submodule and a lightweight enhanced spatial attention submodule.

[0030] The lightweight contrast-aware channel attention submodule includes a contrast layer, two blueprint-separable convolutional layers, and a sigmoid function. It first constructs a contrast representation based on the statistical information of each channel of the input feature, for example, combining the mean and standard deviation to form a contrast feature. Then, it passes through two blueprint-separable convolutional layers and a sigmoid activation function to obtain the corresponding channel attention weights. These weights are then applied to the input feature to enhance the discriminative texture response. The channel attention enhancement feature F output by the i-th lightweight dual attention module is... i CCA as follows: ) In the formula, H GC (‧) represents the global contrast information evaluation function.

[0031] The lightweight spatial attention enhancement submodule, consisting of a 1×1 standard convolution, a stride blueprint separable convolution, a max pooling layer, a set of convolutions, and a 1×1 convolutional layer, first uses the 1×1 standard convolution to compress the number of input channels for the features. The stride blueprint separable convolution and max pooling layer are then used to reduce the spatial size. The stride blueprint separable convolution in the lightweight spatial attention enhancement submodule further extracts features, and combined with residual connections, the original number of channels is restored by the 1×1 convolutional layer. Finally, it is converted into an attention matrix using a sigmoid function, and the channel attention enhancement feature F... i CCA The input to the lightweight augmented spatial attention submodule, after the above process, yields the augmented feature F output by the i-th cascaded lightweight dual attention module. i LBDA Compared to traditional attention modules, this embodiment replaces standard convolutions with blueprint-separable convolutions as much as possible, and further simplifies the group convolutions in enhanced spatial attention to single-layer blueprint-separable convolutions, so as to retain the ability to jointly model channels and space with lower computational cost.

[0032] Furthermore, the BSConvFormer layer (BCFL) based on large-kernel blueprint separable convolution further performs long-distance dependency modeling and high-frequency detail preservation on the attention-enhanced features. Specifically, BCFL generally adopts a Transformer-style feature mixing structure, but replaces the traditional self-attention mechanism with a large-kernel blueprint separable feature mixer. The large-kernel blueprint separable feature mixer includes at least two branches: the first branch includes a 1×1 standard convolution and a 9×9 blueprint separable large-kernel convolution to expand the receptive field and aggregate a wider range of contextual information; the second branch includes a 1×1 standard convolution to preserve the expression of local linear combination features; the outputs of the two branches are then passed through a 1×1 standard convolution for feature fusion to obtain the mixed feature representation.

[0033] Specifically, the first branch in the i-th large kernel blueprint separable feature mixer uses a 1×1 pointwise convolution to enhance feature F. i LBDA Channel fusion is performed, followed by a large kernel blueprint separable convolution with a kernel size of 9 in the i-th first branch to achieve a wide receptive field, resulting in the feature F generated by the attention branch. LKMixer2 ∈R C×H×W ; the 1×1 standard convolution pair F of the i-th second branch i LBDA Feature extraction is performed to obtain the output feature F. LKMixer1 ∈R C×H×W Finally, the feature F is applied using pixel-wise multiplication. LKMixer1 and F LKMixer2Processing yields the i-th large kernel blueprint separable feature mixer feature LKMixer. i .

[0034] In this embodiment, BCFL also includes an edge-preserving feedforward neural network, which comprises two 1×1 convolutional layers, a 3×3 edge-preserving blueprint separable convolution, and a GELU activation function layer. The focus is on using the edge-preserving blueprint separable convolution to optimize the LKMixer feature. i The process yields the deep features F of the i-th cascaded enhanced separable residual feature refinement module. i ESFRB .

[0035] In this embodiment, the multi-layer feature fusion module is used to concatenate multiple deep features, and then sequentially pass them through a 1×1 standard convolution and a 3×3 blueprint separable convolution to obtain fused features. Specifically, the deep features F1, F2, ..., F1 output by each level of ESRFB are... N Concatenated features are formed by concatenating them along the channel dimension using a Concat layer; then, a 1×1 standard convolution is used to fuse the concatenated features by channel; finally, a 3×3 Blueprint Separable Convolution is used to further refine the fused features, resulting in the fused feature F. fusion This multi-layer feature fusion method can comprehensively utilize representation information at different depths, enabling the network to retain both shallow texture information and deep contextual information.

[0036] In this embodiment, the image reconstruction module first utilizes a blueprint-separable convolutional layer to fuse the features F. fusion The processed features are then upsampled using subpixel convolution operations to obtain the reconstructed super-resolution image. In the formula, I SR For the reconstructed super-resolution image; H Rec (‧) indicates pixel reassembly; + indicates residual connection; H BSConv (‧) indicates that the blueprint can be separated into convolutional layer operations.

[0037] Furthermore, when determining the lightweight image super-resolution reconstruction network, it is necessary to train it to obtain the final lightweight image super-resolution reconstruction network. Specifically, during the training phase, K pairs of low-resolution and high-resolution image samples are obtained. The low-resolution images are input into the aforementioned lightweight image super-resolution reconstruction network, which outputs the predicted high-resolution images. The difference between the predicted results and the true high-resolution images is calculated using the mean absolute error loss function L1. Specifically, the loss function can be expressed as: In the formula, L1 represents the mean absolute error loss function.

[0038] Then, the learnable parameters in the network are updated using the backpropagation algorithm until the loss converges or a preset stopping condition is met, thus obtaining the trained image super-resolution reconstruction model.

[0039] In a more specific implementation, the image super-resolution network adopts the PyTorch deep learning architecture. Experiments were conducted on a dedicated deep learning server with an NVIDIA GeForce RTX 3090 GPU, using Ubuntu 20.04.5 as the operating system, CUDA 11.4 as the development tool, and Python 3.8 as the programming language. The number of enhanced separable residual feature refinement modules in the model was set to 6. 800 training images from the DIV2K dataset were used as the original training set, and low-resolution images were obtained using bicubic downsampling with MATLAB. These low-resolution images were randomly cropped into 48×48 image patches, and the high-resolution image patches corresponded to magnification scales of 96×96, 144×144, and 192×192 for ×2, ×3, and ×4 magnifications, respectively. The minibatch size was set to 32. The training process can be divided into two stages. In the first stage, the model was trained from scratch with an initial learning rate of 10×10⁻⁶. -4 The total number of iterations is 500K, with the learning rate halved at iterations of 250K, 400K, 450K, and 475K. The model is trained using the Adam optimizer, with β1 = 0.9, β2 = 0.999, and L1 loss function. In the second stage, the weights of the model trained in the previous stage are loaded, and the rest follows the same training scheme as the first stage.

[0040] To evaluate the performance of this invention in super-resolution reconstruction, it was compared with several lightweight super-resolution reconstruction algorithms. These methods include classic SRCNN, DRRN, CARN, RLFN, ShuffleMixer, SAFMN, MDBN, SPAN, and SeemoRe-T, and their reconstruction results outperform most convolutional neural network-based super-resolution networks. All algorithms were tested using five standard benchmark datasets: Set5, Set14, B100, Urban100, and Manga109. This invention employs two evaluation metrics, Peak Signal-to-Noise Ratio (PSNR) and Structural Similarity Index (SSIM), to comprehensively evaluate the image super-resolution reconstruction results. A comparison of the visual effect of this invention's ×4 reconstruction on the Urban100 dataset with existing state-of-the-art algorithms in terms of local detail is presented, analyzed based on subjective visual perception and objective metrics. The results are shown in Tables 1-3 and... Figure 4 As shown.

[0041] Table 1. Comparison of objective evaluation metrics for different reconstruction methods on benchmark datasets (×2 scale) Table 2 Comparison of objective evaluation metrics for different reconstruction methods on benchmark datasets (×3 scale) Table 3 Comparison of objective evaluation metrics for different reconstruction methods on benchmark datasets (×4 scale) The comparison results in Table 1-3 clearly demonstrate that the present invention surpasses other image reconstruction methods or achieves similar results in the two key image quality evaluation metrics of peak signal-to-noise ratio and structural similarity, while significantly reducing the number of parameters. This proves that the present invention can more effectively achieve a balance between improving the detail clarity and overall structural similarity of reconstructed images and reducing the computational complexity of the method. The method of the present invention can be more widely deployed and effective in practical applications.

[0042] Table 4 compares the present invention with traditional lightweight transformer models in terms of parameter count, floating-point operations (FLOPs), inference time, and PSNR and SSIM on the Urban100 dataset. The comparison results show that the present invention achieves similar objective evaluation metrics while exhibiting significant advantages in parameter count, FLOPs, and inference time, demonstrating its superiority in the field of lightweight image super-resolution reconstruction.

[0043] Table 4 Comparison of objective evaluation metrics and computational costs of different methods on the Urban100 dataset. Figure 5 This image presents a comparison of the lightweight super-resolution image reconstruction method proposed in this invention with other reconstruction methods, showing the super-resolution reconstructed images. In the four images "img_008", "img_024", "img_064", and "img_067", it is clearly evident that the method of this invention successfully recovers more edge details and reduces artifacts. In contrast, all other methods suffer from severe image blurring. The method proposed in this invention achieves a reconstruction quality far superior to other algorithms, more closely resembling a real high-resolution image.

[0044] Example 2 This embodiment provides a lightweight image super-resolution reconstruction system for implementing the lightweight image super-resolution reconstruction method in Embodiment 1.

[0045] In one specific implementation, the lightweight image super-resolution reconstruction system may include: an image acquisition module, a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, an image reconstruction module, and an optional training and update module.

[0046] The image acquisition module is used to acquire low-resolution images to be processed. In training scenarios, the image acquisition module can also acquire high-resolution images corresponding to the low-resolution images to form training sample pairs. The image acquisition module can read image data from the local file system, image database, network interface, camera module, or video capture stream.

[0047] The shallow feature extraction module performs blueprint-separable convolution on the low-resolution image to obtain shallow features F0. In a preferred implementation, the shallow feature extraction module includes at least one blueprint-separable convolution unit to map the input image to the feature space, while reducing redundant computation in subsequent deep feature extraction.

[0048] The deep feature extraction module is used to extract deep features from shallow features step by step, outputting multiple deep features. Specifically, the deep feature extraction module consists of N cascaded Enhanced Separable Residual Feature Refinement Modules (ESRFBs). Each ESRFB includes three Blueprint Separable Feature Refinement Modules (BFBs), a Lightweight Dual Attention Module (LBDA), and a BSConvFormer layer (BCFL) based on a large kernel blueprint separable convolution. The first ESRFB in the deep feature extraction module processes the shallow feature F0, and subsequent ESRFBs process the features output by the previous ESRFB, thereby generating multiple hierarchical deep features.

[0049] BFB is used to perform lightweight refinement extraction on features. It improves the efficiency of local texture extraction through compact convolutional structures while removing internal residual connections and channel segmentation strategies. LBDA is used to perform channel enhancement and spatial enhancement on the refined features output by BFB to highlight key texture regions and effective channels. BCFL is used to perform large receptive field feature mixing and edge-preserving feedforward transformation on the enhanced features, thereby taking into account both global context modeling capabilities and local high-frequency detail preservation capabilities.

[0050] The multi-layer feature fusion module is used to concatenate and fuse deep features from multiple ESRFB outputs to output a fused feature F. fusion Specifically, this module may include a Concat unit, a 1×1 standard convolutional unit, and a 3×3 blueprint separable convolutional unit. The Concat unit is used to concatenate multiple deep features along the channel dimension; the 1×1 standard convolutional unit is used to achieve channel fusion; and the 3×3 blueprint separable convolutional unit is used to further refine the fused features.

[0051] The image reconstruction module is used for fusion feature F fusion The image reconstruction module includes a shallow feature F0 model to reconstruct a high-resolution image. Specifically, the image reconstruction module may include a blueprint separable convolutional unit and a pixel reconstruction unit; wherein, the blueprint separable convolutional unit is used to transform the fused features, and the pixel reconstruction unit is used to achieve upsampling through subpixel convolution, ultimately outputting a high-resolution reconstructed image.

[0052] Optionally, the system also includes a training update module. The training update module is used to calculate the loss function based on low-resolution and high-resolution image sample pairs during the training phase, and to update the learnable parameters in the shallow feature extraction module, deep feature extraction module, multi-layer feature fusion module, and image reconstruction module through backpropagation. Preferably, the training update module calculates the difference between the predicted image and the real high-resolution image based on the L1 loss function, and uses the Adam optimizer to update the model parameters.

[0053] In one alternative implementation, the lightweight image super-resolution reconstruction system may further include a parameter storage module and a result output module. The parameter storage module is used to store the trained network weights, biases, normalization parameters, and other configuration information; the result output module is used to display, save, transmit, or continue to retrieve high-resolution images for subsequent image analysis, recognition, or display.

[0054] In another specific embodiment, the lightweight image super-resolution reconstruction system may consist of a processor, a memory, and a computer program stored in the memory. When the processor executes the computer program, it calls an image acquisition module, a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, an image reconstruction module, and an optional training and update module, thereby implementing the method steps in Embodiment 1. The processor may be a CPU, GPU, DSP, NPU, FPGA, ASIC, or any combination thereof; the memory may be RAM, ROM, Flash, hard disk, solid-state drive, or other non-transitory storage media.

[0055] It should be understood that the modules in the above system can be deployed independently, or they can be merged, split, or further refined according to the actual application. For example, the shallow feature extraction module and the deep feature extraction module can be jointly deployed as a feature encoding unit; the multi-layer feature fusion module and the image reconstruction module can be jointly deployed as a decoding and reconstruction unit; the training and update module can be independently deployed on the training server, while the aforementioned remaining modules are deployed on the terminal inference side. This application does not impose any limitations on this.

[0056] Example 3 This embodiment provides a computer-readable storage medium storing a computer program or instructions that, when executed by a processor, cause the processor to perform the various steps of the lightweight image super-resolution reconstruction method of Embodiment 1.

[0057] Specifically, the computer-readable storage medium can be a read-only memory, random access memory, disk, optical disk, flash memory, solid-state drive, memory card, or other tangible carrier capable of storing program code. The computer program or instructions stored in the storage medium are at least used to cause the processor to perform the following operations: acquire a low-resolution image to be processed; invoke a shallow feature extraction module to perform blueprint-separable convolution processing on the low-resolution image to obtain shallow features; invoke a deep feature extraction module composed of multiple cascaded ESRFBs to extract deep features from the shallow features step-by-step, obtaining multiple deep features; invoke a multi-layer feature fusion module to stitch and fuse the multiple deep features to obtain fused features; and invoke an image reconstruction module to output a high-resolution image based on the fused features and shallow features.

[0058] In one alternative implementation, the computer program or instructions are further configured to cause the processor to perform training operations, including: acquiring low-resolution image and high-resolution image sample pairs; constructing an L1 loss function based on the network output and the high-resolution image; updating the learnable parameters in the lightweight image super-resolution reconstruction network using a backpropagation algorithm; and saving the trained model parameters after the loss converges or a preset number of iterations is reached.

[0059] In another optional implementation, in addition to storing the executable program, the storage medium may also store trained network parameter files, network structure configuration files, magnification configuration parameters, inference configuration parameters, and input / output preprocessing rules. When the processor executes the program, it can further load the corresponding parameter files to achieve reconstruction inference for tasks with different magnifications of ×2, ×3, or ×4.

[0060] It should be understood that the computer-readable storage medium in this embodiment can exist as a standalone product or be integrated into the storage unit corresponding to a terminal device, server device, edge computing box, image processing board, or SoC chip. As long as the program stored in the storage medium can implement the method steps in Embodiment 1 after being executed by the processor, it should be considered to fall within the scope of this application.

[0061] The present application has been described in detail above with reference to specific embodiments, but the present application is not limited to the above-described embodiments. Equivalent substitutions or obvious modifications made by those skilled in the art to the step settings, module division, parameter selection, network stacking depth, convolutional kernel size, attention implementation, training strategy, and deployment method of the present application without departing from the spirit and substance of the present application should all be included within the protection scope of the present application.

[0062] Although embodiments of the invention have been shown and described, it will be understood by those skilled in the art that various changes, modifications, substitutions and alterations can be made to these embodiments without departing from the principles and spirit of the invention. The scope of the invention is defined by the appended claims and their equivalents, all of which should be included within the scope of protection of the invention.

Claims

1. A lightweight image super-resolution reconstruction method, characterized in that, The steps are as follows: Acquire the low-resolution image to be processed; Inputting a low-resolution image into a trained lightweight image super-resolution reconstruction network yields the corresponding high-resolution image. The lightweight image super-resolution reconstruction network includes a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, and an image reconstruction module. The shallow feature extraction module is used to perform blueprint-separable convolution processing on low-resolution images to obtain shallow features; The deep feature extraction module is used to extract deep features from shallow features step by step, resulting in multiple deep features; The multi-layer feature fusion module is used to stitch together and fuse multiple deep features to obtain fused features; The image reconstruction module is used to reconstruct images based on fused features and output high-resolution images.

2. The method of light-weight image super-resolution reconstruction according to claim 1, characterized in that, The deep feature extraction module consists of N cascaded enhanced separable residual feature refinement modules, which extract deep features step by step from shallow features to obtain multiple deep features; The first enhanced separable residual feature refinement module processes the shallow features to obtain the first deep features; The i-th enhanced separable residual feature refinement module processes the deep features output by the (i-1)-th enhanced separable residual feature refinement module to obtain the i-th deep feature.

3. The method of light-weight image super-resolution reconstruction according to claim 2, characterized in that, Each enhanced separable residual feature refinement module includes three stacked blueprint separable feature refinement modules, residual connections, a lightweight dual attention module, and a BSConvFormer layer based on large kernel blueprint separable convolution; The blueprint separable feature refinement module is used to refine and extract input features to obtain refined features; A lightweight dual attention module is used to enhance the refined features in both the channel and spatial dimensions to obtain enhanced features; The BSConvFormer layer, based on large kernel blueprint separable convolution, is used to perform large receptive field feature mixing and feedforward transformation on the enhanced features to obtain the corresponding deep features.

4. The method of light-weight image super-resolution reconstruction according to claim 3, characterized in that, Each Blueprint Separable Feature Refinement Module (BFB) includes a 3×3 Blueprint Separable Convolutional Layer and a GELU activation function layer; Blueprints can be used to separate convolutional layers to extract convolutional features from the input features, and the extracted feature maps are then transformed by activation function layers to obtain refined features.

5. The method of claim 3, wherein the step of determining the weight of each image is performed by using a weighted average method. The lightweight dual attention module includes a lightweight contrast perception channel attention submodule and a lightweight enhanced spatial attention submodule; The lightweight contrast-aware channel attention submodule is used to assign channel attention weights based on the contrast information represented by the sum of the mean and standard deviation of the channel features of the input feature map, and to obtain channel attention enhancement features using two layers of blueprint separable convolution and sigmoid function. The lightweight enhanced spatial attention submodule is used to generate a spatial attention matrix through convolutional dimensionality reduction, stride convolution and pooling downsampling, convolutional feature extraction and convolutional channel recovery, and then multiply the spatial attention matrix with the channel attention enhancement features to obtain the enhanced features.

6. The lightweight image super-resolution reconstruction method according to claim 3, characterized in that, The BSConvFormer layer based on large kernel blueprint separable convolution includes a large kernel blueprint separable feature mixer and an edge-preserving feedforward neural network. The large kernel blueprint separable feature mixer includes a first branch, a second branch, and a fusion branch. The first branch includes a 1×1 standard convolution and a blueprint separable large kernel convolution. The second branch includes a 1×1 standard convolution. The fusion branch includes a 1×1 standard convolution for fusing the outputs of the first branch and the second branch. The edge-preserving feedforward neural network consists of two 1×1 convolutional layers, a 3×3 edge-preserving blueprint separable convolution, and a GELU activation function layer.

7. The lightweight image super-resolution reconstruction method according to claim 1, characterized in that, The multi-layer feature fusion module is used to concatenate multiple deep features, and then pass them sequentially through a 1×1 standard convolution and a 3×3 blueprint separable convolution to obtain fused features; The image reconstruction module is used to perform blueprint-separable convolution processing on the fused features and combine them with shallow features to perform sub-pixel convolution reconstruction to obtain a super-resolution image.

8. The lightweight image super-resolution reconstruction method according to claim 1, characterized in that, The method also includes: Obtain training sample pairs of low-resolution and high-resolution images; The L1 loss function is used to construct a lightweight image super-resolution reconstruction network based on sample pairs; The learnable parameters in the lightweight image super-resolution reconstruction network are updated using the backpropagation algorithm until the loss converges, thus obtaining the trained lightweight image super-resolution reconstruction network.

9. A lightweight image super-resolution reconstruction system, characterized in that, include: The image acquisition unit acquires the low-resolution image to be processed. The image processing unit inputs a low-resolution image into a lightweight image super-resolution reconstruction network to obtain the corresponding high-resolution image. The lightweight image super-resolution reconstruction network includes a shallow feature extraction module, a deep feature extraction module, a multi-layer feature fusion module, and an image reconstruction module. The shallow feature extraction module is used to perform blueprint-separable convolution processing on low-resolution images to obtain shallow features; The deep feature extraction module is used to extract deep features from shallow features step by step, resulting in multiple deep features; The multi-layer feature fusion module is used to stitch together and fuse multiple deep features to obtain fused features; The image reconstruction module is used to reconstruct images based on fused features and output high-resolution images.

10. A computer-readable storage medium having a computer program stored thereon, characterized in that, When a computer program is executed by a processor, the processor performs the lightweight image super-resolution reconstruction method as described in any one of claims 1-8.

Images