Image super-resolution construction method based on multi-scale feature refining network
By designing a multi-scale feature refinement network (MSFRN) in image super-resolution tasks, combining CNN and Transformer attention mechanisms, the limitations of existing methods in terms of computational complexity and detail recovery are solved, and efficient image detail recovery and computational efficiency balance is achieved.
Patent Information
- Application Number
- CN202411735052.3
- Authority / Receiving Office
- CN · China
- Patent Type
- Applications(China)
- Current Assignee / Owner
- Filing Date
- 2024-11-29
- Publication Date
- 2025-04-11
- Estimated Expiration
- 2044-11-29
AI Technical Summary
The existing CNN-based image super-resolution method has limitations in terms of computational complexity and storage requirements, and it is difficult to effectively improve the interactivity of cross-region features when directly applying Transformer, resulting in insufficient image detail recovery.
A multi-scale feature refinement network (MSFRN) is designed, combining CNN and Transformer attention mechanisms, and by introducing a detail extraction engine, global multi-scale context module and feature refinement module, we will enhance cross-region feature interaction and image detail recovery capabilities.
It realizes that while maintaining the quality of image reconstruction, it significantly improves computing efficiency, enhances image detail recovery ability, and provides higher quality image reconstruction effects.
Smart Images

Figure CN119809931A_ABST
Abstract
Description
Technical Field
[0001] The present invention relates to the technical field of image processing, and in particular to an image super-resolution construction method based on a multi-scale feature refinement network. Background Art
[0002] Single image super-resolution (SISR) technology is a method to restore high-resolution (HR) images from low-resolution (LR) images. Its importance in the field of computer vision is self-evident. Although traditional super-resolution methods, such as interpolation techniques and reconstruction algorithms, can improve the resolution of images to a certain extent, they have obvious limitations in restoring image details. In recent years, with the vigorous development of deep learning technology, super-resolution methods based on convolutional neural networks (CNN) have gradually become mainstream. CNN has made significant progress in the field of SISR with its powerful feature extraction capabilities.
[0003] However, existing CNN models, while pursuing high-resolution images, also face the problem of high computational complexity and storage requirements. This limits the application of CNN models in resource-constrained environments, such as mobile devices or embedded systems. To solve this problem, researchers began to explore lightweight CNN models, reducing the computational complexity by reducing the number of network layers, using small convolution kernels, and other methods. Although these methods have achieved results to a certain extent, it is still difficult to achieve a significant improvement in computational efficiency while maintaining the quality of image reconstruction.
[0004] On the other hand, the successful application of Transformer in the field of natural language processing (NLP) has inspired researchers to explore its application in the field of computer vision. The self-attention mechanism of Transformer can effectively model long-distance dependencies, which is of great significance for the restoration of image details. However, it is not reasonable to directly apply Transformer to image super-resolution tasks because CNN has irreplaceable advantages in extracting local features and maintaining local invariance.
[0005] Therefore, how to build a network model that combines CNN and Transformer has become a research hotspot. Although the existing methods that combine CNN and Transformer attempt to make up for the shortcomings of a single structure, they often fail to effectively improve the interactivity of cross-regional features in practice, and only stay at the simple superposition of local and global features. In addition, during the feature extraction process, high-frequency information such as edges and textures are easily diluted or lost, resulting in a lack of clarity and naturalness in the generated images, affecting the final visual quality. Summary of the invention
[0006] The purpose of this section is to summarize some aspects of embodiments of the present invention and briefly introduce some preferred embodiments. Some simplifications or omissions may be made in this section and the specification abstract and the invention title of this application to avoid blurring the purpose of this section, the specification abstract and the invention title, and such simplifications or omissions cannot be used to limit the scope of the present invention.
[0007] In view of the problems existing in the above-mentioned existing image super-resolution construction method based on multi-scale feature refinement network, the present invention is proposed.
[0008] Therefore, the object of the present invention is to provide an image super-resolution construction method based on a multi-scale feature refinement network, which enhances the limitations of cross-region feature interaction and restores details in single image super-resolution by introducing a detail extraction engine (DEE), a global multi-scale context module (GMCM) and a feature refinement module (FRM).
[0009] In order to solve the above technical problems, the present invention provides the following technical solutions: a method for constructing image super-resolution based on a multi-scale feature refinement network, comprising the following steps:
[0010] Step 1: Use the DIV2K dataset to construct high-resolution and low-resolution image pairs, combine data augmentation methods (such as random cropping, rotation, flipping, etc.), and design diversified input features to cover different types of degradation patterns and resolution requirements to initialize the network model and improve the generalization ability during training.
[0011] Step 2: Construct a multi-scale feature refinement network: The network includes a detail extraction engine, a global multi-scale context module and a feature refinement module; the detail extraction engine focuses on extracting edge features and detail textures of the input image; the global multi-scale context module captures local and global information through a multi-scale window grouping strategy; the feature refinement module further optimizes and fuses features through up- and down-sampling and residual learning mechanisms to improve image reconstruction quality;
[0012] Step 3: During the training process, we optimize the parameters of the multi-scale feature refinement network model by minimizing the L1 loss function, and use the Adam optimization algorithm to dynamically adjust the learning rate to reduce the difference between the reconstructed image and the real high-resolution image and enhance the model's detail recovery ability. The specific optimization goals are:
[0013]
[0014] where F θ (I LR )=I SR represents the reconstructed image generated by the model parameters θ, and N is the number of images in the training set.
[0015] Step 4: Use the trained multi-scale feature refinement network model to perform super-resolution processing on the low-resolution image to produce a high-resolution image as output. Subsequently, we evaluate the model on the test datasets (Set5, Set11, B100, Urban100, Manga109) and verify its superior performance by calculating the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) indicators. This ensures that our model can provide stable and high-quality image reconstruction effects in different scenarios.
[0016] As a preferred solution of the image super-resolution construction method based on the multi-scale feature refinement network of the present invention, wherein: the detail extraction engine module in the step 2 performs preliminary feature extraction on the input image by using a 3×3 convolution layer, and the module designs a selective feature enhancer, which focuses on or emphasizes the features at different positions in the input feature map by designing different convolution kernel weights. The mathematical expression is as follows:
[0017]
[0018] Among them, v i represents the output feature of the i-th channel, x represents the input feature map, W i represents the convolution kernel weight of the i-th channel, * represents the convolution operation, x i Represents the input features of the i-th channel;
[0019] After SFE processing, the features are further refined through the ReLU activation function and the second layer of SFE:
[0020]
[0021] Finally, the detail extraction engine module uses residual connections to combine the original input x with the enhanced features Combined to ensure retention of key information:
[0022]
[0023] As a preferred solution of the method for constructing image super-resolution based on multi-scale feature refinement network of the present invention, wherein: in the step 2, the global multi-scale context module inputs the feature map X∈R b×c×h×w It is first expanded through a 1×1 convolutional layer and then divided into several groups, each of which is processed using a different window size; the window size is determined by the set [s1, s2, ..., s n ], where each s corresponds to a different window size, such as [4, 8, 16]; for each window size s, the input feature map is shifted to capture different spatial contexts; the shifted feature map is represented as:
[0024]
[0025] Where: b is the batch size, c is the number of channels, h and w are the height and width of the feature map respectively, the rolling operation cyclically moves the feature map in space, and X~ represents the moved feature map;
[0026] Based on the shifted feature map, an attention mechanism is applied to capture the spatial dependencies within each window. The attention mechanism first projects the features into query Q, key K, and value V representations as shown below:
[0027]
[0028] Among them, W Q , W K and W V is a learnable weight matrix; the self-attention weight is calculated as follows:
[0029]
[0030] Where A represents the weight matrix Q, K and V represent query, key and value respectively, and dk is the dimension of the key vector; the output features are calculated as follows:
[0031] Output=A·V(7)
[0032] Where · represents matrix multiplication; the output features of all windows are concatenated through a 1×1 convolutional layer:
[0033] v=Conv 1×1 (Concat(Output))(8).
[0034] As a preferred solution of the image super-resolution construction method based on the multi-scale feature refinement network described in the present invention, wherein: the feature refinement module efficiently captures and reconstructs the high-frequency details in the image through a series of downsampling and upsampling operations; the module first performs a 1×1 convolution projection on the input feature to obtain the feature v′, then downsamples the feature twice by 2×2 convolution, then upsamples using bilinear interpolation, and refines it through 5×5 convolution and GELU activation.
[0035] As a preferred solution of the image super-resolution construction method based on the multi-scale feature refinement network of the present invention, the mathematical expressions of downsampling and upsampling are as follows:
[0036]
[0037] Then FRM fuses the upsampled features with the previous downsampled output to ensure the preservation of details and the improvement of denoising performance; the mathematical expressions for the fusion of upsampled and downsampled features are as follows:
[0038]
[0039] The features of the final output are combined with the original input through a residual connection:
[0040] v final =Proj out (Conv 3×3 (torch.cat(v,v out ,dim=1)))+v in (11)
[0041] torch.cat means concatenating the input feature map v and the transformed feature map v_out in the channel dimension; Proj out It is a projection layer that adjusts the output of the convolutional layer to the same dimension as the original input; it realizes the flow of information and feature extraction by adding the input feature map with the feature map after a series of transformations.
[0042] Beneficial effects of the invention: The present invention provides a multi-scale feature refinement network (MSFRN) for image super-resolution tasks. The network achieves a balance between reconstruction performance and computational efficiency by combining CNN and Transformer attention mechanisms. The design of MSFRN fully considers the characteristics of image super-resolution tasks, and enhances cross-region feature interaction and restores the limitations of details in single image super-resolution by introducing a detail extraction engine (DEE), a global multi-scale context module (GMCM) and a feature refinement module (FRM). BRIEF DESCRIPTION OF THE DRAWINGS
[0043] In order to more clearly illustrate the technical solutions of the embodiments of the present invention, the following briefly introduces the drawings required for describing the embodiments. Obviously, the drawings described below are only some embodiments of the present invention. For ordinary technicians in this field, other drawings can be obtained based on these drawings without creative labor. Among them:
[0044] Figure 1 :The overall design framework of the Multi-Scale Feature Refinement Network (MSFRN);
[0045] Figure 2 : Schematic diagram of the architecture of the detail extraction engine (DEE);
[0046] Figure 3 :Schematic diagram of the architecture of the Global Multi-Scale Context Module (GMCM);
[0047] Figure 4 : Schematic diagram of the architecture of the feature refinement module (FRM);
[0048] Figure 5 :Visual comparison of the present invention and other super-resolution reconstruction methods on the image Img_012 in the Urban100 dataset;
[0049] Figure 6 : Visual comparison of the present invention and other super-resolution reconstruction methods on image 78004 in dataset B100. DETAILED DESCRIPTION
[0050] In order to make the above-mentioned objects, features and advantages of the present invention more obvious and easy to understand, the specific implementation methods of the present invention are described in detail below in conjunction with the accompanying drawings.
[0051] In the following description, many specific details are set forth to facilitate a full understanding of the present invention, but the present invention may also be implemented in other ways different from those described herein, and those skilled in the art may make similar generalizations without violating the connotation of the present invention. Therefore, the present invention is not limited to the specific embodiments disclosed below.
[0052] Secondly, the term "one embodiment" or "embodiment" as used herein refers to a specific feature, structure, or characteristic that may be included in at least one implementation of the present invention. The term "in one embodiment" that appears in different places in this specification does not necessarily refer to the same embodiment, nor does it refer to a separate or selective embodiment that is mutually exclusive with other embodiments.
[0053] Secondly, the present invention is described in detail with reference to the schematic diagram. When describing the embodiments of the present invention in detail, for the sake of convenience, the cross-sectional diagrams showing the device structure will not be partially enlarged according to the general scale, and the schematic diagrams are only examples, which should not limit the scope of protection of the present invention. In addition, in actual production, the three-dimensional dimensions of length, width and depth should be included.
[0054] Reference Figure 1-Figure 6 , provides an image super-resolution construction method based on a multi-scale feature refinement network, comprising the following steps:
[0055] Step 1: Use the DIV2K dataset to construct high-resolution and low-resolution image pairs, combine data augmentation methods (such as random cropping, rotation, flipping, etc.), and design diversified input features to cover different types of degradation patterns and resolution requirements to initialize the network model and improve the generalization ability during training.
[0056] Step 2: Construct a multi-scale feature refinement network: The network includes a detail extraction engine, a global multi-scale context module and a feature refinement module; the detail extraction engine focuses on extracting edge features and detail textures of the input image; the global multi-scale context module captures local and global information through a multi-scale window grouping strategy; the feature refinement module further optimizes and fuses features through up- and down-sampling and residual learning mechanisms to improve image reconstruction quality;
[0057] Step 3: During the training process, we optimize the parameters of the multi-scale feature refinement network model by minimizing the L1 loss function and dynamically adjust the learning rate using the Adam optimization algorithm to reduce the difference between the reconstructed image and the real high-resolution image and enhance the model's detail recovery ability;
[0058] Step 4: Use the trained multi-scale feature refinement network model to perform super-resolution processing on the low-resolution image to produce a high-resolution image as output. Subsequently, we evaluate the model on the test datasets (Set5, Set11, B100, Urban100, Manga109) and verify its superior performance by calculating the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) indicators. This ensures that our model can provide stable and high-quality image reconstruction effects in different scenarios.
[0059] Among them, as attached Figure 2 In step 2, the detail extraction engine module performs preliminary feature extraction on the input image by using a 3×3 convolution layer. The module designs a selective feature enhancer that focuses on or emphasizes features at different positions in the input feature map by designing different convolution kernel weights. The mathematical expression is as follows:
[0060]
[0061] Among them, v i represents the output feature of the i-th channel, x represents the input feature map, W i represents the convolution kernel weight of the i-th channel, * represents the convolution operation, x i Represents the input features of the i-th channel;
[0062] After SFE processing, the features are further refined through the ReLU activation function and the second layer of SFE:
[0063]
[0064] Finally, the detail extraction engine module uses residual connections to combine the original input x with the enhanced features Combined to ensure retention of key information:
[0065]
[0066] The detail extraction engine (DEE) further includes a selective feature enhancer (SFE), which is designed to selectively strengthen the features of specific regions or directions in the input feature map to enhance the representation of local features. Specifically, SFE is able to weight different parts of the input feature map by using specially designed convolution kernel weights in different directions or regions, thereby highlighting key information and enriching local features. In addition, by combining the results of all weighted channel groups, SFE not only improves the overall characteristics of the feature map, but also provides stronger support for cross-region feature interactions, thereby improving the performance of image super-resolution tasks.
[0067] Further, as attached Figure 3 , the global multi-scale context module in step 2 inputs the feature map X∈R b×c×h×w It is first expanded through a 1×1 convolutional layer and then divided into several groups, each of which is processed using a different window size; the window size is determined by the set [s1, s2, ..., s n ], where each s corresponds to a different window size, such as [4, 8, 16]; for each window size s, the input feature map is shifted to capture different spatial contexts; the shifted feature map is represented as:
[0068]
[0069] Where: b is the batch size, c is the number of channels, h and w are the height and width of the feature map respectively, and the rolling operation cyclically moves the feature map in space. Represents the feature map after the move;
[0070] Based on the shifted feature map, an attention mechanism is applied to capture the spatial dependencies within each window. The attention mechanism first projects the features into query Q, key K, and value V representations as shown below:
[0071]
[0072] Among them, W Q , W K and W V is a learnable weight matrix; the self-attention weight is calculated as follows:
[0073]
[0074] Where A represents the weight matrix Q, K and V represent query, key and value respectively, and dk is the dimension of the key vector; the output features are calculated as follows:
[0075] Output=A·V(7)
[0076] Where · represents matrix multiplication; the output features of all windows are concatenated through a 1×1 convolutional layer:
[0077] v=Conv 1×1 (Concat(Output))(8).
[0078] The global multi-scale context module (GMCM) divides the input features into several groups of windows, each of which uses different sizes (such as 4×4, 8×8 and 16×16) to capture context information of different scales. The module uses a feature fusion method based on the self-attention mechanism to calculate the attention weight within each window, and through the window rolling operation, each window can take into account the information of adjacent windows when calculating the attention weight, thereby enhancing the interaction between features. The multi-scale features are further projected back to the original space through a 1×1 convolution operation to ensure the diversity and consistency of the features, and ultimately achieve efficient modeling and fusion of local and global features. The window rolling operation breaks the limitations of fixed window division by cyclically moving the feature map by half the window size pixels along the height and width dimensions, allowing features between different windows to exchange information, thereby providing more comprehensive context information, enhancing the model's ability to capture the global structure of the image, and improving the quality of image super-resolution.
[0079] Among them, as attached Figure 4 , the feature refinement module efficiently captures and reconstructs high-frequency details in the image through a series of downsampling and upsampling operations; the module first performs a 1×1 convolution projection on the input feature to obtain the feature v′, then downsamples the feature twice by 2×2 convolution, then upsamples it using bilinear interpolation, and refines it through 5×5 convolution and GELU activation. The mathematical expressions for downsampling and upsampling are as follows:
[0080]
[0081] Then FRM fuses the upsampled features with the previous downsampled output to ensure the preservation of details and the improvement of denoising performance; the mathematical expressions for the fusion of upsampled and downsampled features are as follows:
[0082]
[0083] The features of the final output are combined with the original input through a residual connection:
[0084] v final =Proj out (Conv 3×3 (torch.cat(v,v out ,dim=1)))+v in(11)
[0085] torch.cat means concatenating the input feature map v and the transformed feature map v_out in the channel dimension; Proj out It is a projection layer that adjusts the output of the convolutional layer to the same dimension as the original input; it realizes the flow of information and feature extraction by adding the input feature map with the feature map after a series of transformations.
[0086] The Feature Refinement Module (FRM) refines the input features through multi-layer convolution operations combined with up and down sampling mechanisms. The module extracts multi-scale high-frequency features through bilinear interpolation and high-dimensional feature convolution operations, and learns the difference between the original input and the refined features through residual connections to further enhance the ability to restore detailed textures. The design of the Feature Refinement Module improves image reconstruction performance while ensuring the lightweight and efficient model.
[0087] Experimental configuration and results:
[0088] MSFRN is trained with DIV2K. To verify the performance of MSFRN, we conducted experiments on multiple public datasets, including Set5, Set14, B100, Urban100, and Manga109. In the experiment, we used the following configuration:
[0089] (1) Training parameters: The training batch size is 64, and each batch contains 256×256 pixel patches. The optimizer uses Adam, the initial learning rate is 0.0002, and the cosine annealing strategy is used to adjust the learning rate. The training cycle is 500 epochs.
[0090] (2) Hardware and software environment: The experiments were conducted on NVIDIA GTX1080 GPU using the PyTorch framework.
[0091] Table 1 shows the performance comparison of MSFRN and other lightweight SISR models on different datasets. It can be seen that MSFRN achieves the best performance on most datasets, demonstrating strong reconstruction quality and relatively efficient computational requirements. MSFRN uses a relatively small number of parameters and FLOPs, achieving a practical balance between model complexity and image quality.
[0092] Table 1: Comparison of average PSNR and SSIM of the present invention and existing lightweight super-resolution models (the best is bolded, and the suboptimal is underlined):
[0093]
[0094]
[0095] The present invention combines convolutional neural networks (CNNs) and transformer attention mechanisms to address the limitations of existing methods in enhancing cross-region feature interactions and recovering details in single image super-resolution. MSFRN adopts a modular design to synergistically enhance feature representation and visual quality. Specifically, the detail extraction engine (DEE) strengthens local features through the selective feature enhancer (SFE), indirectly promoting cross-region feature interactions. The multi-scale context module (GMCM) adopts a multi-scale window attention mechanism combined with cyclic rolling to improve cross-region feature correlation and achieve effective fusion of global and local information. In addition, the feature refinement module (FRM) gradually recovers details and textures through multi-level upsampling and downsampling, effectively solving the problem of insufficient detail and texture recovery. Experiments verify that MSFRN achieves competitive performance and enhanced visual quality compared to state-of-the-art methods while maintaining computational efficiency. Future work will explore lightweight attention mechanisms to further reduce resource requirements and expand the application of the model to a wider range of low-level vision tasks.
[0096] It should be noted that the above embodiments are only used to illustrate the technical solutions of the present invention rather than to limit it. Although the present invention has been described in detail with reference to the preferred embodiments, those skilled in the art should understand that the technical solutions of the present invention may be modified or replaced by equivalents without departing from the spirit and scope of the technical solutions of the present invention, which should all be included in the scope of the claims of the present invention.
Claims
1. A method for constructing image super-resolution based on a multi-scale feature refinement network, characterized in that: The following steps are involved: Step 1: Use the DIV2K dataset to construct high-resolution and low-resolution image pairs, combine data augmentation methods (such as random cropping, rotation, flipping, etc.), and design diversified input features to cover different types of degradation patterns and resolution requirements to initialize the network model and improve the generalization ability during training. Step 2: Construct a multi-scale feature refinement network: The network includes a detail extraction engine, a global multi-scale context module and a feature refinement module; the detail extraction engine focuses on extracting edge features and detail textures of the input image; the global multi-scale context module captures local and global information through a multi-scale window grouping strategy; the feature refinement module further optimizes and fuses features through up- and down-sampling and residual learning mechanisms to improve image reconstruction quality; Step 3: During the training process, we optimize the parameters of the multi-scale feature refinement network model by minimizing the L1 loss function, and use the Adam optimization algorithm to dynamically adjust the learning rate to reduce the difference between the reconstructed image and the real high-resolution image and enhance the model's detail recovery ability. The specific optimization goals are: Among them, F θ (I LR )=I SR represents the reconstructed image generated by the model parameters θ, and N is the number of images in the training set. Step 4: Use the trained multi-scale feature refinement network model to perform super-resolution processing on the low-resolution image to produce a high-resolution image as output. Subsequently, we evaluate the model on the test datasets (Set5, Set11, B100, Urban100, Manga109) and verify its superior performance by calculating the peak signal-to-noise ratio (PSNR) and structural similarity (SSIM) indicators. This ensures that our model can provide stable and high-quality image reconstruction effects in different scenarios.
2. The method for constructing image super-resolution based on a multi-scale feature refinement network according to claim 1, characterized in that: In the step 2, the detail extraction engine module performs preliminary feature extraction on the input image by using a 3×3 convolution layer. The module designs a selective feature enhancer, which focuses on or emphasizes the features at different positions in the input feature map by designing different convolution kernel weights. The mathematical expression is as follows: Among them, v i represents the output feature of the i-th channel, x represents the input feature map, W i represents the convolution kernel weight of the i-th channel, * represents the convolution operation, x i Represents the input features of the i-th channel; After SFE processing, the features are further refined through the ReLU activation function and the second layer of SFE: Finally, the detail extraction engine module uses residual connections to combine the original input x with the enhanced features Combined to ensure retention of key information:
3. The method for constructing image super-resolution based on a multi-scale feature refinement network according to claim 2, characterized in that: In step 2, the global multi-scale context module inputs the feature map X∈R b×c×h×w It is first expanded through a 1×1 convolutional layer and then divided into several groups, each of which is processed using a different window size; the window size is determined by the set [s1, s2, ..., s n ], where each s corresponds to a different window size, such as [4, 8, 16]; for each window size s, the input feature map is shifted to capture different spatial contexts; the shifted feature map is represented as: Where: b is the batch size, c is the number of channels, h and w are the height and width of the feature map respectively, and the rolling operation cyclically moves the feature map in space. Represents the feature map after the move; Based on the shifted feature map, an attention mechanism is applied to capture the spatial dependencies within each window. The attention mechanism first projects the features into query Q, key K, and value V representations as shown below: Among them, W Q , W K and W V is a learnable weight matrix; the self-attention weight is calculated as follows: Where A represents the weight matrix Q, K and V represent query, key and value respectively, and dk is the dimension of the key vector; the output features are calculated as follows: Output=A·V(7) Where · represents matrix multiplication; the output features of all windows are concatenated through a 1×1 convolutional layer: v=Conv 1×1 (Concat(Output))(8)。 4. The method for constructing image super-resolution based on a multi-scale feature refinement network according to claim 3, characterized in that: The feature refinement module efficiently captures and reconstructs high-frequency details in the image through a series of downsampling and upsampling operations; the module first performs a 1×1 convolution projection on the input feature to obtain the feature v′, then downsamples the feature twice by 2×2 convolution, then upsamples it using bilinear interpolation, and refines it through 5×5 convolution and GELU activation.
5. The method for constructing image super-resolution based on a multi-scale feature refinement network according to claim 4, characterized in that: The mathematical expressions of downsampling and upsampling are as follows: Then FRM fuses the upsampled features with the previous downsampled output to ensure the preservation of details and the improvement of denoising performance; the mathematical expressions for the fusion of upsampled and downsampled features are as follows: The features of the final output are combined with the original input through a residual connection: in final =Proj out (Conv. 3×3 (torch.cat(v,v out ,dim=1)))+v in (11) torch.cat means concatenating the input feature map v and the transformed feature map v_out in the channel dimension; Proj out It is a projection layer that adjusts the output of the convolutional layer to the same dimension as the original input; it realizes the flow of information and feature extraction by adding the input feature map with the feature map after a series of transformations.
Citation Information
Patent Citations
Super-resolution multi-scale residual fusion model of single image and restoration method thereof
CN111861961A
Lightweight image super-resolution reconstruction method based on deep learning
CN115100039A
Video super-resolution method based on optical flow enhancement algorithm
CN115330592A
Handwritten text image super-resolution reconstruction method based on deep expansion network
CN116611995A
Super-resolution reconstruction method based on multi-scale attention expansion
CN117333357A
Cited By
Image restoration and super-resolution reconstruction system and method based on deep learning
CN120976017A