A single image super-resolution method and system based on wavelet information step-by-step enhancement mixing

Abstract

The application discloses a kind of based on wavelet information gradually enhanced hybrid single image super-resolution method and system, comprising: constructing super-resolution network, in shallow feature extraction stage, the shallow feature of low-resolution image is extracted using convolution layer;In deep multi-direction feature extraction stage, shallow feature map is subjected to multiple iterations of direction perception wavelet feature fusion processing, four subbands are obtained by discrete wavelet transform, and the texture of strip is enhanced in horizontal direction, vertical direction and combined direction respectively, and using the cross deformable attention mechanism of rectangular window division, enhanced texture feature is gradually fused into low-frequency feature in the order of horizontal, vertical, diagonal, and depth multi-direction feature map is obtained, and after global residual connection with shallow feature map, input reconstruction stage, and reconstruct high-resolution image.The texture authenticity and direction consistency of reconstructed image are significantly improved, and excellent super-resolution reconstruction performance is realized.

Description

Technical Field

[0001] This invention belongs to the field of computer vision and image processing technology, and in particular relates to a single-image super-resolution method and system based on wavelet information progressive enhancement and mixing. Background Technology

[0002] Single Image Super-Resolution (SISR) aims to reconstruct high-resolution (HR) images from a single low-resolution (LR) input. As a fundamental task in computer vision and image processing, SISR plays a crucial role in many practical applications such as medical imaging, autonomous driving, and video surveillance. Although a number of deep learning (DL) methods exist capable of learning the complex mapping relationship between LR and HR images, recovering high-fidelity details from low-quality images remains a challenging task due to the inherent loss of high-frequency details during reconstruction.

[0003] Early research primarily relied on Convolutional Neural Networks (CNNs) for feature extraction, leveraging their powerful nonlinear fitting capabilities. However, the inherent local receptive field of CNNs limits their ability to model long-range dependencies. To overcome this bottleneck, attention-based Transformer models were introduced into computer vision tasks and achieved significant success in the SISR field. For example, the Swin Transformer-based image restoration model (SwinIR) and Hybrid Attention Transformer (HAT) capture long-range dependencies through self-attention mechanisms, achieving a wider receptive field than traditional CNN models and thus improving reconstruction performance.

[0004] While Transformer-based methods excel in spatial domain feature modeling, they often overlook the complementary information inherent in the frequency domain, which is crucial for edge and fine texture reconstruction. Recent research has demonstrated that wavelet transform (WT), as an effective frequency domain analysis tool, can capture multi-scale image context and texture features. For example, the Wavelet-Based Texture Reconstruction Network (WTRN) utilizes wavelet transform to generate enhanced high-frequency representations to aid image super-resolution; WavemixSR combines the inductive bias of CNNs with the lossless transform properties of wavelets, improving performance while reducing resource consumption. However, existing wavelet-based methods still have certain limitations: firstly, most methods employ the same processing strategy for texture features in different directions (horizontal, vertical, diagonal), lacking direction awareness; secondly, in the feature fusion stage, fixed fusion patterns are often used, making it difficult to achieve synergistic enhancement of multi-directional texture information.

[0005] In summary, existing super-resolution methods based on frequency domain decomposition generally suffer from insufficient orientation awareness, limited texture enhancement strategies, and a lack of spatial adaptability in their fusion mechanisms. To address these shortcomings, this invention proposes a single-image super-resolution method and system based on wavelet information progressive enhancement and mixing. Summary of the Invention

[0006] To address the above technical problems, this invention provides a single-image super-resolution method and system based on wavelet information progressive enhancement and mixing.

[0007] The technical solution adopted by this invention to solve its technical problem is: A single-image super-resolution method based on wavelet information progressive enhancement and mixing, the method comprising the following steps: S100: Construct a single-image super-resolution network based on wavelet information progressive enhancement and hybridization. The network includes a shallow feature extraction stage, a deep multi-directional feature extraction stage, and a high-quality image reconstruction stage. S200: In the shallow feature extraction stage, convolutional layers are used to extract features from the input low-resolution image, mapping the input image from RGB space to high-dimensional feature space to obtain a low-resolution shallow feature map. S300: In the deep multi-directional feature extraction stage, the shallow feature map undergoes multiple iterations of direction-aware wavelet feature fusion processing. In each processing step, the shallow feature map is first upsampled and then discrete wavelet transform is applied to decompose the features into four sub-bands: low-low frequency sub-band, low-high frequency sub-band, high-low frequency sub-band, and high-high frequency sub-band. Horizontal strip texture enhancement is performed on the low-high frequency sub-band, vertical strip texture enhancement is performed on the high-low frequency sub-band, and combined horizontal and vertical texture enhancement is performed on the high frequency sub-band. Smoothing feature extraction is performed on the low-low frequency sub-band. Finally, the enhanced texture features of each direction are gradually fused into the low-frequency features in the order of horizontal, vertical, and diagonal through a cross-deformable attention mechanism based on rectangular window partitioning to obtain a deep multi-directional feature map. S400: After performing a global residual connection between the depth multi-directional feature map and the shallow feature map, it is input into the high-quality image reconstruction stage to reconstruct a high-resolution image at the target scale. S500: The super-resolution network is optimized and trained using a pixel-based loss function, so that the reconstructed high-resolution image approximates the real high-resolution image.

[0008] Preferably, in the shallow feature extraction stage of S200, the input low-resolution image is processed through a hierarchical convolutional structure. Transformed into a high-dimensional feature space, where and The height and width of the image are represented by these values, respectively. First, a 7×7 convolution is used to capture broad contextual information, followed by a 3×3 convolution for local thinning, ultimately generating shallow features. , is represented as: in, This represents the shallow feature extraction function. Indicates the number of feature channels.

[0009] Preferably, in step S300, the input features are upsampled to maintain spatial resolution, and the features are decomposed into four sub-bands using discrete wavelet transform, including a low-frequency sub-band. Low and high frequency subband High and low frequency subbands and high-frequency subband The process is represented as follows: in, This represents the discrete wavelet transform operation. Indicates an upsampling operation. This represents the input features of the direction-aware wavelet feature fusion module; The S300 employs a smoothing feature extraction module for low-frequency subbands. The smooth feature extraction module consists of a Swin transformer module and an overlapping cross-attention module. The processing steps of the smooth feature extraction module include: processing the input features through convolutional layers. The feature map is processed by convolution, and then fed into the Swin transformer module. This module divides the feature map into multiple non-overlapping local windows and performs self-attention computation within each window. The output of the Swin transformer module is then fed into the overlapping cross-attention module. This module uses a sliding window strategy to perform attention computation on the overlapping windows to capture a wider range of low-frequency structural information. Finally, the output of the overlapping cross-attention module is connected to the input feature map via residual connections. Add them together to get the output of the smooth feature extraction module. , is represented as: in, Indicates a convolutional layer. This indicates a window partitioning operation. This indicates the processing by the Swintransformer module. This indicates the processing of overlapping attention modules.

[0010] Preferably, in S300, a horizontal texture enhancement module is used to enhance the horizontal texture sub-band. For enhancement processing, the horizontal texture enhancement module adopts an architecture based on convolution operations to capture horizontal texture features; firstly, two cascaded 5×3 horizontal convolution kernels are used to process the input features. Perform convolution processing to obtain the first intermediate feature; simultaneously, process the input feature... Perform global average pooling, and map the pooled features to... The tensor of the first intermediate feature is processed by a sigmoid activation function to obtain the first modulation factor. The first intermediate feature is then multiplied element-wise by the first modulation factor to obtain the second intermediate feature. The second intermediate feature is then subjected to projection mapping and 1×1 convolution operations, followed by sigmoid activation to obtain the second modulation factor. Finally, the second intermediate feature is multiplied element-wise by the second modulation factor, and the residuals are added to the second intermediate feature to obtain the enhanced horizontal texture feature. The process is represented as follows: in, This represents a 5×3 horizontal convolution kernel operation. This represents the sigmoid activation function. This represents a linear mapping operation. Represents a 1×1 convolution kernel operation. Represents projection mapping operation, and These represent the height and width of the feature map, respectively. This is the second intermediate feature.

[0011] Preferably, in S300, a vertical texture enhancement module is used to enhance the vertical texture sub-bands. To enhance the features, this module employs an architecture centered on convolutional operations to capture vertical texture features. First, two cascaded 3×5 vertical convolutional kernels are used to process the input features. Convolution processing is performed to obtain the third intermediate feature; simultaneously, the input features are processed... Perform global average pooling, and map the pooled features to... The tensor of the third intermediate feature is then processed through a sigmoid activation function to obtain the third modulation factor. The third intermediate feature is then multiplied element-wise with the third modulation factor to obtain the fourth intermediate feature. The fourth intermediate feature is then subjected to projection mapping and 1×1 convolution operations, followed by a sigmoid activation function to obtain the fourth modulation factor. Finally, the fourth intermediate feature is multiplied element-wise with the fourth modulation factor, and the residuals are added to the fourth intermediate feature to obtain the enhanced vertical texture feature. The entire process can be represented as follows: in, This represents a 3×5 vertical convolution kernel operation. It is the fourth intermediate feature; For high-frequency subband During enhancement, the texture is processed sequentially through a horizontal texture enhancement module and a vertical texture enhancement module to obtain the enhanced diagonal texture features. , is represented as: . Preferably, in S300, the enhanced texture features in each direction are gradually fused into the low-frequency features in the order of horizontal, vertical, and diagonal through a cross-deformable attention mechanism based on rectangular window partitioning, including: In the first fusion, with enhanced horizontal texture features As a high-frequency input, the low-frequency features output by the smoothing feature extraction module are used. As a low-frequency input, the low-frequency and high-frequency inputs are windowed according to the horizontal texture direction, with the window height being [value missing]. =4, width is A horizontal rectangular window with a resolution of 16; In the second fusion, enhanced vertical texture features are used. As a high-frequency input, the features output after the first fusion are used as a low-frequency input. The low-frequency and high-frequency inputs are windowed according to the vertical texture direction, with the partitioning strategy based on window height. =16, Width =4 vertical rectangular window; In the third fusion, enhanced diagonal texture features are used. As a high-frequency input, the features output after the second fusion are used as a low-frequency input. The low-frequency and high-frequency inputs are windowed according to the diagonal texture direction, with a window height of [value missing]. =8, width is =8 square window; The overall wavelet feature fusion process is represented as follows: The window partitioning operation in each merge is represented as follows: in, Represents window segmentation operation, This represents the low-frequency input features in the current fusion step. This represents the high-frequency input features in the current fusion step. This indicates the low-frequency output characteristics in the current fusion step. This represents the high-frequency output characteristics in the current fusion step.

[0012] Preferably, in the cross-deformable attention mechanism, a mask for sparse attention computation is learned from high-frequency features through a mask network. : in, Indicates a masked network; Based on this mask, K windows are selected from all windows for attention calculation, and the remaining nK windows are subjected to convolution processing, where n is the total number of windows. The calculation formula is as follows: Where ws represents the window size; During attention computation, the query matrix Q, key matrix K, value matrix V, and value matrix processed by convolution are involved. They are represented as follows: in, , and These are the weights of the query matrix, key matrix, and value matrix, respectively. The final output of the cross-deformable attention mechanism is represented as: in, The dimensions representing the query matrix and the key matrix, This represents a learnable relative positional encoding. This indicates a convolution operation.

[0013] Preferably, the deep multi-directional feature extraction stage in S300 consists of K cascaded perceptual wavelet feature fusion modules, and the output of each module is expressed as: in, This represents the depth multi-directional features output by the wavelet feature fusion module for the k-th direction. This represents the processing function of the k-th module. This is a shallow feature map; Features output by the last module Combined with shallow feature maps via global residual connections: in, This provides fusion features for the high-quality image reconstruction stage.

[0014] Preferably, in the high-quality image reconstruction stage of S400, the fused features are processed by the high-quality image reconstruction module HQIR. The processing module includes convolutional layers, upsampling layers, and dropout layers to obtain the final super-resolution image. Specifically: In the S500, the L1 pixel loss function is used to optimize the network training. in, Represents the image index. This represents the number of training sample pairs. This represents the high-resolution image reconstructed from the i-th input low-resolution image by the network. This represents the corresponding true high-resolution image.

[0015] A single-image super-resolution system based on wavelet information progressive enhancement and hybridization includes: The network construction module is used to build a single-image super-resolution network based on wavelet information progressive enhancement and hybridization. The network includes a shallow feature extraction stage, a deep multi-directional feature extraction stage, and a high-quality image reconstruction stage. The shallow feature extraction module uses convolutional layers to extract features from the input low-resolution image, mapping the input image from RGB space to a high-dimensional feature space to obtain a low-resolution shallow feature map. The deep multi-directional feature extraction module performs multiple iterations of direction-aware wavelet feature fusion processing on the shallow feature map. In each processing step, the shallow feature map is first upsampled and then decomposed into four sub-bands using discrete wavelet transform: low-low frequency sub-band, low-high frequency sub-band, high-low frequency sub-band, and high-high frequency sub-band. Horizontal strip texture enhancement is performed on the low-high frequency sub-band, vertical strip texture enhancement is performed on the high-low frequency sub-band, and combined horizontal and vertical texture enhancement is performed on the high frequency sub-band. Smoothing feature extraction is performed on the low-low frequency sub-band. Finally, the enhanced texture features of each direction are gradually fused into the low-frequency features in the order of horizontal, vertical, and diagonal through a cross-deformable attention mechanism based on rectangular window partitioning to obtain the deep multi-directional feature map. The high-quality image reconstruction module is used to reconstruct a high-resolution image at the target scale by performing a global residual connection between the depth multi-directional feature map and the shallow feature map. The optimization training module is used to optimize the training of the super-resolution network using a pixel-based loss function, so that the reconstructed high-resolution image approximates the real high-resolution image.

[0016] The aforementioned super-resolution architecture based on progressive enhancement of wavelet information and direction-guided fusion decomposes features into four sub-bands through discrete wavelet transform and designs a strip texture enhancement module for differentiated enhancement. Combined with the progressive fusion mechanism, it significantly improves the texture realism and directional consistency of the reconstructed image. A direction-guided deformable cross-attention mechanism with rectangular windows is designed. Rectangular windows of different shapes are used for grouping according to feature directionality, and the mask learned from high-frequency features is used to dynamically guide the deformable attention fusion between low-frequency and high-frequency features. This fully exploits the complementary information of multi-directional textures while maintaining high computational efficiency. An end-to-end trainable deep network system is constructed, organically integrating the above core components and jointly training them with a loss function. This enables the system to automatically learn the optimal utilization and fusion of multi-directional frequency domain information, achieving excellent super-resolution reconstruction performance. Attached Figure Description

[0017] Figure 1 This is an overall flowchart of a single-image super-resolution method based on wavelet information progressive enhancement and mixing, according to an embodiment of the present invention. Figure 2 This is a schematic diagram of the overall framework of a single-image super-resolution method based on wavelet information progressive enhancement and mixing in one embodiment of the present invention. Figure 3 This is a schematic diagram of the framework of the smooth feature extraction module in one embodiment of the present invention; Figure 4 This is a schematic diagram illustrating the principle of horizontal texture enhancement in one embodiment of the present invention; Figure 5 This is a schematic diagram illustrating the vertical texture enhancement principle in one embodiment of the present invention; Figure 6 This is a schematic diagram of the rectangular window intersection deformable attention mechanism framework in one embodiment of the present invention; Figure 7 This is a schematic diagram of a rectangular mask network framework in one embodiment of the present invention. Detailed Implementation

[0018] To enable those skilled in the art to better understand the technical solution of the present invention, the present invention will be further described in detail below with reference to the accompanying drawings.

[0019] This invention proposes a single-image super-resolution method and system based on wavelet information progressive enhancement and hybridization. This framework innovatively introduces wavelet domain texture analysis and a directed strip attention mechanism, aiming to improve the texture reconstruction quality and structure awareness in SISR. Since wavelet transform is essentially a set of direction-selective filter banks, capable of decomposing an image into sub-bands representing different texture directions, including horizontal, vertical, and diagonal textures, this method can separate texture components of different directions through wavelet transform and design strip texture enhancement strategies and direction-guided deformable attention to achieve accurate reconstruction of multi-directional textures and effective fusion of cross-domain features. The proposed method mainly consists of three parts: a shallow feature extraction part, a deep multi-directional feature extraction part, and a high-quality image reconstruction part. Specifically, this invention designs a progressive wavelet information processing structure, including a horizontal texture enhancement module (HTEM) and a vertical texture enhancement module (VTEM) based on strip enhancement, a smooth feature extraction block (SFEB) for extracting low-frequency structural features, and a rectangular-window based cross-deformable attention module (RCDA) to enhance texture information in specific directions and achieve direction-aware feature interaction. These modules together constitute a direction-aware wavelet feature fusion (DWFF) module. The following... Figure 1 This is a flowchart of the overall super-resolution method for single images based on wavelet information progressive enhancement and mixing proposed in this invention. Figure 2 A schematic diagram of the overall framework of the single-image super-resolution method based on wavelet information progressive enhancement and mixing proposed in this invention is shown below. The workflow of this invention is as follows: I. Single-Image Super-Resolution Network S100: Construct a single-image super-resolution network based on wavelet information progressive enhancement and hybridization. The network includes a shallow feature extraction stage, a deep multi-directional feature extraction stage, and a high-quality image reconstruction stage.

[0020] II. Shallow Feature Extraction Stage S200: In the shallow feature extraction stage, convolutional layers are used to extract features from the input low-resolution image, mapping the input image from RGB space to a high-dimensional feature space to obtain a low-resolution shallow feature map.

[0021] As a specific implementation, the shallow feature extraction stage in S200 uses a hierarchical convolutional structure to transform the input low-resolution image. Transformed into a high-dimensional feature space, where and The height and width of the image are represented by these values, respectively. First, a 7×7 convolution is used to capture broad contextual information, followed by a 3×3 convolution for local thinning, ultimately generating shallow features. , is represented as: in, This represents the shallow feature extraction function. Indicates the number of feature channels.

[0022] III. Deep Multi-directional Feature Extraction Stage S300: In the deep multi-directional feature extraction stage, the shallow feature map undergoes multiple iterations of direction-aware wavelet feature fusion processing. In each processing step, the shallow feature map is first upsampled and then Discrete Wavelet Transform (DWT) is applied to decompose the features into four sub-bands: low-low frequency sub-band (LL), low-high frequency sub-band (LH), high-low frequency sub-band (HL), and high-high frequency sub-band (HH), ensuring that the size of the decomposed feature map is consistent with the input. The low-high frequency sub-band is enhanced with horizontal (HTEM) strip texture, the high-low frequency sub-band is enhanced with vertical (VTEM) strip texture, the high frequency sub-band is enhanced with a combination of horizontal and vertical texture, and the low-low frequency sub-band is smoothed for feature extraction. Finally, the enhanced texture features of each direction are gradually fused into the low-frequency features in the order of horizontal, vertical, and diagonal through the cross-deformable attention mechanism RCDA based on rectangular window partitioning to obtain the deep multi-directional feature map.

[0023] Specifically, the enhanced horizontal texture (LH), vertical texture (HL), and high-frequency details (HH) are progressively fused into low-frequency features (LL) using RCDA. This module first groups features by direction using a rectangular window partitioning mechanism: horizontal features use horizontal rectangular windows, vertical features use vertical rectangular windows, and diagonal features use square windows. Then, a mask learning network learns rectangular masks from the high-frequency features to perform deformable attention. The query and value matrices are derived from the low-frequency features, the key matrix is ​​derived from the high-frequency features, and the non-attention part is processed using 3×3 convolutions. Through this step, texture enhancement features based on wavelet transforms are gradually obtained. The deep multi-directional feature extraction section consists of K directional wavelet feature fusion modules (DWFF). Each DWFF block includes: upsampling and wavelet transform, smooth feature extraction and directional texture enhancement, and progressive fusion of wavelet features. The overall framework diagram is shown below. Figure 2 As shown.

[0024] (1) Upsampling and wavelet transform As a specific implementation, in S300, the input features are upsampled to maintain spatial resolution, and the features are decomposed into four sub-bands using discrete wavelet transform, including a low-frequency sub-band. Low and high frequency subband High and low frequency subbands and high-frequency subband The process is represented as follows: in, This represents the discrete wavelet transform operation. Indicates an upsampling operation. This represents the input features of the direction-aware wavelet feature fusion module.

[0025] (2) Smoothing feature extraction In the smoothing feature extraction module, the Transformer excels at capturing low-frequency features. When constructing the SFEB, we leveraged the Transformer's powerful low-frequency information extraction capabilities and inherent long receptive field to efficiently capture low-frequency domain data in wavelet transforms. .

[0026] As a specific example, such as Figure 3 As shown, the S300 uses a smoothing feature extraction module for low-frequency subbands. The smooth feature extraction module consists of a Swin transformer module and an overlapping cross-attention module. The processing steps of the smooth feature extraction module include: processing the input features through convolutional layers. The feature map is processed by convolution, and then fed into the Swin transformer module. This module divides the feature map into multiple non-overlapping local windows and performs self-attention computation within each window. The output of the Swin transformer module is then fed into the overlapping cross-attention module. This module uses a sliding window strategy to perform attention computation on the overlapping windows to capture a wider range of low-frequency structural information. Finally, the output of the overlapping cross-attention module is connected to the input feature map via residual connections. Add them together to get the output of the smooth feature extraction module. , is represented as: in, Indicates a convolutional layer. This indicates a window partitioning operation. This indicates the processing by the Swintransformer module. This indicates the processing of overlapping attention modules.

[0027] Specifically, the Swin transformer module consists of two consecutive stacked basic blocks. Each basic block follows a fixed process: layer normalization - window self-attention - residual connection - layer normalization - multilayer perceptron - residual connection. The input features are layer normalized, the feature map is divided into multiple non-overlapping local windows, self-attention is calculated independently in each window, the output of the window self-attention is summed element-by-element with the original input, the residual features are layer normalized again, the features are non-linearly transformed by the multilayer perceptron, and finally the output of the multilayer perceptron is summed element-by-element with the output of the first residual connection to complete the processing of one basic block.

[0028] Furthermore, for the attention module, given an input feature map of size H×W×C, it is first segmented into non-overlapping local windows of size M×M (the total number of windows is...). Next, self-attention is calculated for each window. Furthermore, OCA is implemented using a sliding window strategy with an expand operation, which expands the window to a size of... × Overlapping windows. Through this operation, the network can capture a wider range of low-frequency structural information.

[0029] (3) Horizontal texture enhancement As a specific example, such as Figure 4 As shown, the S300 uses a horizontal texture enhancement module to enhance the horizontal texture subband. For enhancement processing, the horizontal texture enhancement module adopts an architecture based on convolution operations to capture horizontal texture features; firstly, two cascaded 5×3 horizontal convolution kernels are used to process the input features. Perform convolution processing to obtain the first intermediate feature; simultaneously, process the input feature... Perform global average pooling, and map the pooled features to... The tensor of the first intermediate feature is processed by a sigmoid activation function to obtain the first modulation factor. The first intermediate feature is then multiplied element-wise by the first modulation factor to obtain the second intermediate feature. The second intermediate feature is then subjected to projection mapping and 1×1 convolution operations, followed by sigmoid activation to obtain the second modulation factor. Finally, the second intermediate feature is multiplied element-wise by the second modulation factor, and the residuals are added to the second intermediate feature to obtain the enhanced horizontal texture feature. The process is represented as follows: in, This represents a 5×3 horizontal convolution kernel operation. This represents the sigmoid activation function. This represents a linear mapping operation. Represents a 1×1 convolution kernel operation. Represents projection mapping operation, and These represent the height and width of the feature map, respectively. This is the second intermediate feature. Through this horizontal texture enhancement, the model can obtain higher quality horizontal high-frequency information.

[0030] (4) Vertical texture enhancement As a specific example, such as Figure 5 As shown, the S300 uses a vertical texture enhancement module to enhance the vertical texture subband. To enhance the features, this module employs an architecture centered on convolutional operations to capture vertical texture features. First, two cascaded 3×5 vertical convolutional kernels are used to process the input features. Convolution processing is performed to obtain the third intermediate feature; simultaneously, the input features are processed... Perform global average pooling, and map the pooled features to... The tensor of the third intermediate feature is then processed through a sigmoid activation function to obtain the third modulation factor. The third intermediate feature is then multiplied element-wise with the third modulation factor to obtain the fourth intermediate feature. The fourth intermediate feature is then subjected to projection mapping and 1×1 convolution operations, followed by a sigmoid activation function to obtain the fourth modulation factor. Finally, the fourth intermediate feature is multiplied element-wise with the fourth modulation factor, and the residuals are added to the fourth intermediate feature to obtain the enhanced vertical texture feature. The entire process can be represented as follows: in, This represents a 3×5 vertical convolution kernel operation. This is the fourth intermediate feature; through this vertical texture enhancement, the model can obtain higher quality vertical high-frequency information. For high-frequency subband During enhancement, the texture is processed sequentially through a horizontal texture enhancement module and a vertical texture enhancement module to obtain the enhanced diagonal texture features. , is represented as: . (5) Gradual fusion of wavelet features Low-frequency components primarily capture global content information, while high-frequency components encode fine-grained edge details. Wavelet transform provides an efficient and lossless frequency decomposition technique; therefore, gradually and specifically integrating high-frequency features from different directions into low-frequency features can effectively solve the problem of high-frequency information recovery in SR tasks. However, in wavelet transform, a significant portion of high-frequency pixels have low values ​​and limited useful information content. Indiscriminately utilizing all high-frequency channels not only leads to excessive computational overhead but may also degrade SR performance. Therefore, the RCDA designed in this invention alleviates this problem using a deformable mechanism and guides fusion with texture direction to generate higher-quality fusion results. The specific process is as follows: Figure 6 As shown.

[0031] As a specific implementation, in S300, the enhanced texture features in each direction are gradually fused into the low-frequency features in the order of horizontal, vertical, and diagonal through a cross-deformable attention mechanism based on rectangular window partitioning, including: In the first fusion, with enhanced horizontal texture features As a high-frequency input, the low-frequency features output by the smoothing feature extraction module are used. As a low-frequency input, the low-frequency and high-frequency inputs are windowed according to the horizontal texture direction, with the window height being [value missing]. =4, width is A horizontal rectangular window with a resolution of 16; In the second fusion, enhanced vertical texture features are used. As a high-frequency input, the features output after the first fusion are used as a low-frequency input. The low-frequency and high-frequency inputs are windowed according to the vertical texture direction, with the partitioning strategy based on window height. =16, Width =4 vertical rectangular window; In the third fusion, enhanced diagonal texture features are used. As a high-frequency input, the features output after the second fusion are used as a low-frequency input. The low-frequency and high-frequency inputs are windowed according to the diagonal texture direction, with a window height of [value missing]. =8, width is =8 square window; The overall wavelet feature fusion process is represented as follows: The window partitioning operation in each merge is represented as follows: in, Represents window segmentation operation, This represents the low-frequency input features in the current fusion step. This represents the high-frequency input features in the current fusion step. This indicates the low-frequency output characteristics in the current fusion step. This represents the high-frequency output characteristics in the current fusion step.

[0032] Furthermore, in the cross-deformable attention mechanism, a mask for sparse attention computation is learned from high-frequency features through a mask network. : in, Represents a mask network; a schematic diagram of a rectangular mask network is shown below. Figure 7 As shown, high-frequency features The features are rearranged and flattened into a sequence. A linear transformation is then performed on the rearranged features to achieve feature dimensionality compression and information fusion. Next, a non-linear ReLU activation is introduced to enhance the model's expressive power. The model then passes through another linear layer to map the hidden features to a dimension consistent with the number of windows, preparing for mask generation. Finally, Softmax activation is applied to normalize the window scores into a probability distribution, resulting in interpretable attention mask weights.

[0033] Based on this mask, K windows are selected from all windows for attention calculation, and the remaining nK windows are subjected to convolution processing, where n is the total number of windows. The calculation formula is as follows: Where ws represents the window size; During attention computation, the query matrix Q, key matrix K, value matrix V, and value matrix processed by convolution are involved. They are represented as follows: in, , and These are the weights of the query matrix, key matrix, and value matrix, respectively. It can be seen that in the attention mechanism, the key matrix comes from high-frequency information, while the query matrix and value matrix come from low-frequency components.

[0034] The final output of the cross-deformable attention mechanism is represented as: in, The dimensions representing the query matrix and the key matrix, This represents a learnable relative positional encoding. This indicates a convolution operation.

[0035] Furthermore, the deep multi-directional feature extraction stage in S300 consists of K cascaded perceptual wavelet feature fusion modules, and the output of each module is represented as: in, This represents the depth multi-directional features output by the wavelet feature fusion module for the k-th direction. This represents the processing function of the k-th module. This is a shallow feature map; Features output by the last module Combined with shallow feature maps via global residual connections: in, This provides fusion features for the high-quality image reconstruction stage.

[0036] Specifically, the above architecture decomposes features into directional subbands (LH, HL, HH, LL) using Discrete Wavelet Transform (DWT), and designs direction-specific strip texture enhancement modules (HTEM, VTEM, and SFEB) to differentially enhance these subbands. Then, through a progressive direction-guided fusion mechanism, it achieves refined reconstruction of multi-directional textures and effective fusion of cross-domain features, thereby significantly improving the texture realism, directional consistency, and structural integrity of the reconstructed image and obtaining high-quality super-resolution results.

[0037] Meanwhile, the proposed Direction-Guided Rectangular Window Deformable Cross-Attention (RCDA) mechanism's core innovation lies in grouping features using rectangular windows of different shapes based on their directionality (horizontal, vertical, diagonal), and dynamically guiding the deformable attention fusion between low-frequency and high-frequency features using masks learned from high-frequency features. This design achieves direction-guided feature alignment and refinement, more fully exploring the complementary value of texture features in different directions, while effectively suppressing interference from insignificant regions, thus improving the quality of detail reconstruction while maintaining high computational efficiency.

[0038] IV. High-quality image reconstruction stage S400: After performing a global residual connection between the depth multi-directional feature map and the shallow feature map, it is input into the high-quality image reconstruction stage to reconstruct a high-resolution image at the target scale.

[0039] As a specific embodiment, the high-quality image reconstruction stage in S400 uses the high-quality image reconstruction module HQIR to process the fused features. The processing module includes convolutional layers, upsampling layers, and dropout layers to obtain the final super-resolution image. Specifically: Specifically, dropout layers are added to alleviate the overfitting problem of the model and improve its generalization ability.

[0040] V. Pixel-based loss function S500: The super-resolution network is optimized and trained using a pixel-based loss function, so that the reconstructed high-resolution image approximates the real high-resolution image.

[0041] In the S500, the L1 pixel loss function is used to optimize the network training. in, Represents the image index. This represents the number of training sample pairs. This represents the high-resolution image reconstructed from the i-th input low-resolution image by the network. This represents the corresponding true high-resolution image.

[0042] Specifically, through The loss function is jointly trained with the optimization objective, enabling the entire system to automatically learn the complex mapping from low-resolution input to high-resolution output, especially how to optimally utilize and fuse multi-directional frequency domain information, ultimately forming a complete, efficient and high-performance single-image super-resolution solution.

[0043] The above-mentioned single-image super-resolution method based on wavelet information progressive enhancement and mixing has the following main advantages: (1) This invention constructs an end-to-end trainable deep network, realizing the integrated optimization of the entire process from wavelet decomposition, directional texture enhancement, progressive feature fusion to high-resolution reconstruction, which significantly improves the overall performance and adaptability of the method.

[0044] (2) This invention innovatively combines wavelet transform with direction-aware mechanism and designs exclusive strip texture enhancement modules (HTEM and VTEM) for different directional sub-bands (LH, HL, HH), which significantly improves the ability to accurately capture and enhance horizontal, vertical and diagonal textures. Compared with existing methods that only process frequency domain features in the spatial domain or uniformly process frequency domain features, it can more effectively restore high-frequency details in the image and avoid edge blurring and texture loss.

[0045] (3) This invention designs an efficient, direction-guided deformable cross-attention fusion mechanism and progressively fuses textures in different directions. The mechanism groups features by direction through rectangular windows and dynamically guides the fusion between low-frequency and high-frequency features using masks learned from high-frequency features. This design enables the fusion process to focus on significant texture regions in specific directions, achieving more accurate cross-domain feature alignment and complementary information extraction, while ensuring operational efficiency through convolution processing of non-attention regions.

[0046] (4) This invention is based on the design An optimized pixel loss function guides network training to improve sensitivity to directional frequency domain features in the source image, avoiding the loss of high-frequency details. Simultaneously, by rationally allocating fusion weights for low-frequency and high-frequency features, a balanced reconstruction of intensity information and texture details is achieved, resulting in excellent super-resolution performance.

[0047] In one embodiment, a single-image super-resolution system based on wavelet information progressive enhancement and hybridization is also provided, comprising: The network construction module is used to build a single-image super-resolution network based on wavelet information progressive enhancement and hybridization. The network includes a shallow feature extraction stage, a deep multi-directional feature extraction stage, and a high-quality image reconstruction stage. The shallow feature extraction module uses convolutional layers to extract features from the input low-resolution image, mapping the input image from RGB space to a high-dimensional feature space to obtain a low-resolution shallow feature map. The deep multi-directional feature extraction module performs multiple iterations of direction-aware wavelet feature fusion processing on the shallow feature map. In each processing step, the shallow feature map is first upsampled and then decomposed into four sub-bands using discrete wavelet transform: low-low frequency sub-band, low-high frequency sub-band, high-low frequency sub-band, and high-high frequency sub-band. Horizontal strip texture enhancement is performed on the low-high frequency sub-band, vertical strip texture enhancement is performed on the high-low frequency sub-band, and combined horizontal and vertical texture enhancement is performed on the high frequency sub-band. Smoothing feature extraction is performed on the low-low frequency sub-band. Finally, the enhanced texture features of each direction are gradually fused into the low-frequency features in the order of horizontal, vertical, and diagonal through a cross-deformable attention mechanism based on rectangular window partitioning to obtain the deep multi-directional feature map. The high-quality image reconstruction module is used to reconstruct a high-resolution image at the target scale by performing a global residual connection between the depth multi-directional feature map and the shallow feature map. The optimization training module is used to optimize the training of the super-resolution network using a pixel-based loss function, so that the reconstructed high-resolution image approximates the real high-resolution image. Specific limitations regarding the single-image super-resolution system based on wavelet information progressive enhancement and mixing can be found in the limitations of the single-image super-resolution method based on wavelet information progressive enhancement and mixing described above, and will not be repeated here. Each module in the aforementioned single-image super-resolution system based on wavelet information progressive enhancement and mixing can be implemented entirely or partially through software, hardware, or a combination thereof. These modules can be embedded in or independent of the processor in a computer device, or stored in the memory of a computer device in software form, so that the processor can call and execute the corresponding operations of each module.

[0048] In one embodiment, a computer device includes a memory and a processor, the memory storing a computer program, the processor executing the computer program to implement the steps of a single-image super-resolution method based on wavelet information progressive enhancement mixing.

[0049] In one embodiment, a computer-readable storage medium stores a computer program that, when executed by a processor, implements the steps of a single-image super-resolution method based on wavelet information progressive enhancement mixing.

[0050] Those skilled in the art will understand that all or part of the processes in the methods of the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium, and when executed, it can include the processes of the embodiments of the above methods. Any references to memory, storage, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, or optical storage, etc. Volatile memory can include random access memory (RAM) or external cache memory. By way of illustration and not limitation, RAM can be in various forms, such as static random access memory (SRAM) or dynamic random access memory (DRAM), etc.

[0051] The present invention provides a detailed description of a single-image super-resolution method and system based on wavelet information progressive enhancement and mixing. Specific examples have been used to illustrate the principles and implementation methods of the invention. The descriptions of these embodiments are merely for the purpose of helping to understand the core ideas of the invention. It should be noted that those skilled in the art can make various improvements and modifications to the invention without departing from its principles, and these improvements and modifications also fall within the scope of protection of the claims of the present invention.

Claims

1. A single-image super-resolution method based on wavelet information progressive enhancement and mixing, characterized in that, The method includes the following steps: S100: Construct a single-image super-resolution network based on wavelet information progressive enhancement and hybridization. The network includes a shallow feature extraction stage, a deep multi-directional feature extraction stage, and a high-quality image reconstruction stage. S200: In the shallow feature extraction stage, convolutional layers are used to extract features from the input low-resolution image, mapping the input image from RGB space to high-dimensional feature space to obtain a low-resolution shallow feature map. S300: In the deep multi-directional feature extraction stage, the shallow feature map undergoes multiple iterations of direction-aware wavelet feature fusion processing. In each processing step, the shallow feature map is first upsampled and then discrete wavelet transform is applied to decompose the features into four sub-bands: low-low frequency sub-band, low-high frequency sub-band, high-low frequency sub-band, and high-high frequency sub-band. Horizontal strip texture enhancement is performed on the low-high frequency sub-band, vertical strip texture enhancement is performed on the high-low frequency sub-band, and combined horizontal and vertical texture enhancement is performed on the high frequency sub-band. Smoothing feature extraction is performed on the low-low frequency sub-band. Finally, the enhanced texture features of each direction are gradually fused into the low-frequency features in the order of horizontal, vertical, and diagonal through a cross-deformable attention mechanism based on rectangular window partitioning to obtain a deep multi-directional feature map. S400: After performing a global residual connection between the depth multi-directional feature map and the shallow feature map, it is input into the high-quality image reconstruction stage to reconstruct a high-resolution image at the target scale. S500: The super-resolution network is optimized and trained using a pixel-based loss function, so that the reconstructed high-resolution image approximates the real high-resolution image.

2. The method according to claim 1, characterized in that, In the shallow feature extraction stage of S200, a hierarchical convolutional structure is used to extract the input low-resolution image. Transformed into a high-dimensional feature space, where and The height and width of the image are represented by these values, respectively. First, a 7×7 convolution is used to capture broad contextual information, followed by a 3×3 convolution for local thinning, ultimately generating shallow features. , is represented as: in, This represents the shallow feature extraction function. Indicates the number of feature channels.

3. The method according to claim 1, characterized in that, In S300, the input features are upsampled to maintain spatial resolution, and the features are decomposed into four sub-bands using discrete wavelet transform, including a low-frequency sub-band. Low and high frequency subband High and low frequency subbands and high-frequency subband The process is represented as follows: in, This represents the discrete wavelet transform operation. Indicates an upsampling operation. This represents the input features of the direction-aware wavelet feature fusion module; The S300 employs a smoothing feature extraction module for low-frequency subbands. The smooth feature extraction module consists of a Swintransformer module and an overlapping cross-attention module. The processing steps of the smooth feature extraction module include: processing the input features through convolutional layers. The feature map is processed by convolution, and then fed into the Swin transformer module. This module divides the feature map into multiple non-overlapping local windows and performs self-attention computation within each window. The output of the Swin transformer module is then fed into the overlapping cross-attention module. This module uses a sliding window strategy to perform attention computation on the overlapping windows to capture a wider range of low-frequency structural information. Finally, the output of the overlapping cross-attention module is connected to the input feature map via residual connections. Add them together to get the output of the smooth feature extraction module. , is represented as: in, Indicates a convolutional layer. This indicates a window partitioning operation. This indicates the processing of the Swing Transformer module. This indicates the processing of overlapping attention modules.

4. The method according to claim 3, characterized in that, The S300 uses a horizontal texture enhancement module to enhance the horizontal texture subband. For enhancement processing, the horizontal texture enhancement module adopts an architecture based on convolution operations to capture horizontal texture features; firstly, two cascaded 5×3 horizontal convolution kernels are used to process the input features. Perform convolution processing to obtain the first intermediate feature; simultaneously, process the input feature... Perform global average pooling, and map the pooled features to... The tensor of the first intermediate feature is processed by a sigmoid activation function to obtain the first modulation factor. The first intermediate feature is then multiplied element-wise by the first modulation factor to obtain the second intermediate feature. The second intermediate feature is then subjected to projection mapping and 1×1 convolution operations, followed by sigmoid activation to obtain the second modulation factor. Finally, the second intermediate feature is multiplied element-wise by the second modulation factor, and the residuals are added to the second intermediate feature to obtain the enhanced horizontal texture feature. The process is represented as follows: in, This represents a 5×3 horizontal convolution kernel operation. This represents the sigmoid activation function. This represents a linear mapping operation. Represents a 1×1 convolution kernel operation. Represents projection mapping operation, and These represent the height and width of the feature map, respectively. This is the second intermediate feature.

5. The method according to claim 4, characterized in that, The S300 uses a vertical texture enhancement module to enhance the vertical texture subband. To enhance the features, this module employs an architecture centered on convolutional operations to capture vertical texture features. First, two cascaded 3×5 vertical convolutional kernels are used to process the input features. Convolution processing is performed to obtain the third intermediate feature; simultaneously, the input features are processed... Perform global average pooling, and map the pooled features to... The tensor of the third intermediate feature is then processed through a sigmoid activation function to obtain the third modulation factor. The third intermediate feature is then multiplied element-wise with the third modulation factor to obtain the fourth intermediate feature. The fourth intermediate feature is then subjected to projection mapping and 1×1 convolution operations, followed by a sigmoid activation function to obtain the fourth modulation factor. Finally, the fourth intermediate feature is multiplied element-wise with the fourth modulation factor, and the residuals are added to the fourth intermediate feature to obtain the enhanced vertical texture feature. The entire process can be represented as follows: in, This represents a 3×5 vertical convolution kernel operation. It is the fourth intermediate feature; For high-frequency subband During enhancement, the texture is processed sequentially through a horizontal texture enhancement module and a vertical texture enhancement module to obtain the enhanced diagonal texture features. , is represented as: 。 6. The method according to claim 5, characterized in that, In S300, enhanced texture features in each direction are gradually fused into low-frequency features in the order of horizontal, vertical, and diagonal through a cross-deformable attention mechanism based on rectangular window partitioning. This includes: In the first fusion, with enhanced horizontal texture features As a high-frequency input, the low-frequency features output by the smoothing feature extraction module are used. As a low-frequency input, the low-frequency and high-frequency inputs are windowed according to the horizontal texture direction, with the window height being [value missing]. =4, width is A horizontal rectangular window with a resolution of 16; In the second fusion, enhanced vertical texture features are used. As a high-frequency input, the features output after the first fusion are used as a low-frequency input. The low-frequency and high-frequency inputs are windowed according to the vertical texture direction, with the partitioning strategy based on window height. =16, Width =4 vertical rectangular window; In the third fusion, enhanced diagonal texture features are used. As a high-frequency input, the features output after the second fusion are used as a low-frequency input. The low-frequency and high-frequency inputs are windowed according to the diagonal texture direction, with a window height of [value missing]. =8, width is A square window with a value of 8; The overall wavelet feature fusion process is represented as follows: The window partitioning operation in each merge is represented as follows: in, Represents window segmentation operation, This represents the low-frequency input features in the current fusion step. This represents the high-frequency input features in the current fusion step. This indicates the low-frequency output characteristics in the current fusion step. This represents the high-frequency output characteristics in the current fusion step.

7. The method according to claim 6, characterized in that, In the cross-deformable attention mechanism, a mask for sparse attention computation is learned from high-frequency features through a mask network. : in, Indicates a masked network; Based on this mask, K windows are selected from all windows for attention calculation, and the remaining nK windows are subjected to convolution processing, where n is the total number of windows. The calculation formula is as follows: Where ws represents the window size; During attention computation, the query matrix Q, key matrix K, value matrix V, and value matrix processed by convolution are involved. They are represented as follows: in, , and These are the weights of the query matrix, key matrix, and value matrix, respectively. The final output of the cross-deformable attention mechanism is represented as: in, The dimensions representing the query matrix and the key matrix, This represents a learnable relative positional encoding. This indicates a convolution operation.

8. The method according to claim 7, characterized in that, In the S300 deep multi-directional feature extraction stage, it consists of K cascaded perceptual wavelet feature fusion modules, and the output of each module is represented as follows: in, This represents the depth multi-directional features output by the wavelet feature fusion module for the k-th direction. This represents the processing function of the k-th module. This is a shallow feature map; Features output by the last module Combined with shallow feature maps via global residual connections: in, This provides fusion features for the high-quality image reconstruction stage.

9. The method according to claim 8, characterized in that, In the S400 high-quality image reconstruction stage, the high-quality image reconstruction module HQIR is used to fuse features. The processing module includes convolutional layers, upsampling layers, and dropout layers to obtain the final super-resolution image. Specifically: In the S500, the L1 pixel loss function is used to optimize the network training. in, Represents the image index. This represents the number of training sample pairs. This represents the high-resolution image reconstructed from the i-th input low-resolution image by the network. This represents the corresponding true high-resolution image.

10. A single-image super-resolution system based on wavelet information progressive enhancement and hybridization, characterized in that, include: The network construction module is used to build a single-image super-resolution network based on wavelet information progressive enhancement and hybridization. The network includes a shallow feature extraction stage, a deep multi-directional feature extraction stage, and a high-quality image reconstruction stage. The shallow feature extraction module uses convolutional layers to extract features from the input low-resolution image, mapping the input image from RGB space to a high-dimensional feature space to obtain a low-resolution shallow feature map. The deep multi-directional feature extraction module performs multiple iterations of directional-aware wavelet feature fusion processing on the shallow feature map; In each processing step, the shallow feature map is first upsampled and then decomposed into four sub-bands using discrete wavelet transform: low-low frequency sub-band, low-high frequency sub-band, high-low frequency sub-band, and high-high frequency sub-band. Horizontal strip texture enhancement is performed on the low-high frequency sub-band, vertical strip texture enhancement on the high-low frequency sub-band, and combined horizontal and vertical texture enhancement on the high frequency sub-band. Smoothing feature extraction is then performed on the low-low frequency sub-band. Finally, the enhanced texture features in each direction are gradually fused into the low-frequency features in the order of horizontal, vertical, and diagonal using a cross-deformable attention mechanism based on rectangular window partitioning, to obtain a multi-directional depth feature map. The high-quality image reconstruction module is used to reconstruct a high-resolution image at the target scale by performing a global residual connection between the depth multi-directional feature map and the shallow feature map. The optimization training module is used to optimize the training of the super-resolution network using a pixel-based loss function, so that the reconstructed high-resolution image approximates the real high-resolution image.

Images