An image denoising method, device, medium and product based on space-frequency complementary fusion

Abstract

This application discloses an image denoising method, apparatus, medium, and product based on spatial-frequency complementary fusion, relating to the field of image processing technology. The image denoising method includes: acquiring a target noisy image; inputting the target noisy image into an image denoising model to obtain a denoised image corresponding to the target noisy image; the image denoising model is obtained by training an initial network; the initial network is constructed based on multiple spatial-frequency complementary fusion modules, including frequency-domain cross-attention blocks and dual-view difference attention blocks. The frequency-domain cross-attention blocks are constructed based on wavelet transform and attention mechanisms, and the dual-view difference attention blocks are constructed based on attention mechanisms. This application improves the network's ability to recover image structures in complex noisy scenes while maintaining a lightweight model, ultimately achieving an effective balance between denoising accuracy and computational efficiency.

Description

Technical Field

[0001] This application relates to the field of image processing technology, and in particular to an image denoising method, apparatus, medium and product based on spatial-frequency complementary fusion. Background Technology

[0002] Images are often susceptible to various external interferences and noise during acquisition and transmission, leading to a significant deterioration in image quality. Therefore, effectively recovering high-quality images from degraded observation data, i.e., image denoising, has become an important preliminary step in image processing.

[0003] Deep learning has been applied to image denoising to overcome the inherent limitations of model-driven methods. Its core idea is to train a network to learn the end-to-end mapping between noisy and clear images. A series of studies have progressed along this line: from denoising convolutional neural networks (DnCNN) employing residual learning, to fast and flexible denoising networks (FFDNet) adaptable to different noise levels, to the more sophisticated U-shaped network architecture.

[0004] However, the aforementioned improvements in network performance largely depend on continuously increasing the number of network layers. While increasing the number of network layers can effectively improve feature representation capabilities, it also leads to a significant increase in the number of parameters and inference latency. Therefore, achieving efficient denoising with limited computational resources has become a core challenge in this field. To alleviate this problem, researchers have attempted to improve performance by widening the network. These methods have certain advantages in reducing computational complexity; however, relying solely on convolutional operations to extract features makes it difficult to effectively distinguish the importance of features, often resulting in redundant extracted features.

[0005] In recent years, researchers have proposed various attention-based denoising networks, including channel attention, spatial attention, and Transformer-based architectures. Meanwhile, considering the different frequency characteristics of noise and image content in the frequency domain, and leveraging the powerful feature representation capabilities of deep learning, numerous frequency domain image denoising methods have also been proposed. For example, by introducing Discrete Cosine Transform (DCT) and Fourier Transform (FT), image features can be transformed from the spatial domain to the frequency domain, thereby removing noise more efficiently.

[0006] Although the above methods have made some progress in denoising performance, it is still difficult to achieve a satisfactory balance between denoising effect and computational efficiency, specifically in the following aspects: (1) Previous methods usually rely on a single feature paradigm, that is, only spatial domain information or only frequency domain information, and fail to effectively complement and fuse the two. This limits the model's ability to capture the diverse features of noise and image content. Especially in complex damage scenes where noise simultaneously exhibits local abrupt changes and frequency variation features, the feature representation of a single paradigm is difficult to fully characterize the degradation pattern, which restricts further improvement of denoising performance.

[0007] (2) Some methods that attempt to fuse spatial and frequency domain information typically employ a simple fusion strategy of directly adding or splicing the parallel branches. This lacks explicit modeling of the semantic relationships between cross-domain features, resulting in insufficient interaction and enhancement of complementary information. At the same time, the introduction of multi-branch frequency domain processing modules significantly increases the number of model parameters and computational overhead, creating a prominent contradiction with the goal of lightweight design and making it difficult to achieve a good balance between denoising effect and inference efficiency.

[0008] To address the aforementioned issues, this application proposes an image denoising method based on spatial-frequency complementary fusion. This method aims to solve the technical problem of how to fully exploit the complementary advantages of spatial and frequency domain features and achieve effective cross-domain feature fusion while maintaining the model's lightweight nature, thereby balancing denoising accuracy and computational efficiency in complex noisy scenarios. Summary of the Invention

[0009] The purpose of this application is to provide an image denoising method, apparatus, medium, and product based on space-frequency complementary fusion, aiming to improve the accuracy and efficiency of image denoising while maintaining the lightweight nature of the model.

[0010] To achieve the above objectives, this application provides the following solution.

[0011] Firstly, this application provides an image denoising method based on spatial-frequency complementary fusion, including: Acquire the target image containing noise; The target noisy image is input into the image denoising model to obtain the denoised image corresponding to the target noisy image; the image denoising model is obtained by training the initial network; the initial network is constructed based on multiple spatial-frequency domain complementary fusion modules, the spatial-frequency domain complementary fusion module includes a frequency domain cross attention block and a dual-view difference attention block, the frequency domain cross attention block is constructed based on wavelet transform and attention mechanism, and the dual-view difference attention block is constructed based on attention mechanism.

[0012] In one embodiment, the process of determining the image denoising model includes: Acquire multiple noisy sample images and their corresponding noise-free sample images; Construct the initial network; The initial network is trained by taking the noisy image of each sample as input and the corresponding noiseless image of the sample as output, to obtain the image denoising model.

[0013] In one embodiment, the initial network includes a first convolutional layer, a cascaded fusion module, a second convolutional layer, and a first subtractor; the cascaded fusion module includes N space-frequency fusion stitching modules; each of the space-frequency fusion stitching modules includes a spatial-frequency domain complementary fusion module and a first stitching module; N≥2; The input of the first spatial-frequency domain complementary fusion module is connected to the output of the first convolutional layer. The input of the first first stitching module is connected to the output of the first spatial-frequency domain complementary fusion module and the output of the first convolutional layer, respectively. The input of the nth spatial-frequency domain complementary fusion module is connected to the (n-1)th first stitching module. The input of the nth first stitching module is connected to the output of the nth spatial-frequency domain complementary fusion module and the output of the (n-1)th first stitching module, respectively. n = 2, 3, ..., N. The output of the Nth first stitching module is connected to the input of the second convolutional layer. The output of the second convolutional layer is connected to the input of the first subtractor.

[0014] In one embodiment, the spatial frequency domain complementary fusion module includes: a frequency domain cross attention block, a dual-view difference attention block, a second stitching module, and a third convolutional layer, a batch normalization layer, a ReLU activation function, a fourth convolutional layer, and an adder connected in sequence. The input of the adder in the first spatial-frequency domain complementary fusion module is connected to the output of the fourth convolutional layer and the output of the first convolutional layer in the first spatial-frequency domain complementary fusion module, respectively. The input of the adder in the nth spatial-frequency domain complementary fusion module is connected to the output of the fourth convolutional layer and the output of the (n-1)th first splicing module, respectively. The output of the adder in the i-th spatial-frequency domain complementary fusion module is connected to the input of the frequency domain cross-attention block and the input of the dual-view difference attention block in the i-th spatial-frequency domain complementary fusion module, respectively. The outputs of the frequency domain cross-attention block and the dual-view difference attention block in the i-th spatial-frequency domain complementary fusion module are both connected to the input of the second stitching module in the i-th spatial-frequency domain complementary fusion module. The output of the second stitching module in the i-th spatial-frequency domain complementary fusion module serves as the output of the i-th spatial-frequency domain complementary fusion module and is connected to the input of the i-th first stitching module. i = 1, 2, ..., N.

[0015] In one embodiment, the frequency domain cross-attention block in any spatial frequency domain complementary fusion module includes: a wavelet transform module, a first shaping module, a third splicing module, a second shaping module, a third shaping module, a first multiplier, a first Softmax activation function, a second multiplier, and a wavelet inverse transform module; The input of the wavelet transform module is connected to the output of the adder in the spatial-frequency domain complementary fusion module. The output of the wavelet transform module is connected to the input of the first shaping module and the input of the third splicing module. The output of the third splicing module is connected to the input of the second shaping module and the input of the third shaping module. The outputs of the second shaping module and the third shaping module are both connected to the input of the first multiplier. The output of the first multiplier is connected to the input of the first Softmax activation function. The outputs of the first Softmax activation function and the first shaping module are both connected to the input of the second multiplier. The output of the second multiplier is connected to the input of the inverse wavelet transform module. The output of the inverse wavelet transform module is connected to the input of the second splicing module in the spatial-frequency domain complementary fusion module.

[0016] In one embodiment, the dual-view difference attention block in any spatial frequency domain complementary fusion module includes: a first pooling layer, a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer, a second pooling layer, an eighth convolutional layer, an upsampling layer, a second Softmax activation function, a third Softmax activation function, a second subtractor, and a third multiplier; The inputs of the first pooling layer, the seventh convolutional layer, and the second pooling layer are all connected to the output of the adder in the spatial-frequency domain complementary fusion module. The output of the first pooling layer is connected to the inputs of the fifth and sixth convolutional layers, respectively. The output of the second pooling layer is connected to the input of the eighth convolutional layer. The output of the eighth convolutional layer is connected to the input of the upsampling layer. The outputs of the sixth and seventh convolutional layers are both connected to the input of the second Softmax activation function. The outputs of the seventh and eighth convolutional layers are both connected to the input of the third Softmax activation function. The outputs of the second and third Softmax activation functions are both connected to the input of the second subtractor. The outputs of the second subtractor and the fifth convolutional layer are both connected to the input of the third multiplier. The output of the third multiplier is connected to the input of the second splicing module in the spatial-frequency domain complementary fusion module.

[0017] In one embodiment, the target noisy image is input into an image denoising model to obtain a denoised image corresponding to the target noisy image, including: The target noisy image is input into the first convolutional layer of the image denoising model to obtain the first convolutional feature map; The first convolutional feature map is input into the cascaded fusion module in the image denoising model to obtain the cascaded fused feature map; The cascaded fused feature map is input into the second convolutional layer in the image denoising model to obtain the second convolutional feature map; The second convolutional feature map and the target noisy image are both input into the first subtractor in the image denoising model to obtain the denoised image.

[0018] Secondly, this application provides a computer device, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement the image denoising method based on space-frequency complementary fusion described above.

[0019] Thirdly, this application provides a computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, implements the image denoising method based on space-frequency complementary fusion described above.

[0020] Fourthly, this application provides a computer program product, including a computer program that, when executed by a processor, implements the image denoising method based on space-frequency complementary fusion described above.

[0021] According to the specific embodiments provided in this application, the following technical effects are disclosed: This application discloses an image denoising method, apparatus, medium, and product based on spatial-frequency complementary fusion. By constructing a spatial-frequency complementary fusion module comprising a frequency-domain cross-attention block and a dual-viewpoint difference attention block, the frequency-domain cross-attention block selectively enhances and suppresses frequency-domain features based on wavelet transform and attention mechanisms. Simultaneously, the dual-viewpoint difference attention block captures key information in the spatial domain based on an attention mechanism. This allows the network to utilize complementary features from both frequency and spatial domain paradigms, overcoming the limitations of a single paradigm's feature representation capability. The two branches process in parallel and are fused within the module. The attention mechanism enables semantic association modeling of cross-domain features, avoiding information redundancy and conflicts caused by simple addition or splicing. The high efficiency of wavelet transform and the lightweight design of the attention mechanism allow for controllable parameter quantities when multiple modules are stacked, thereby improving the network's ability to recover image structures in complex noisy scenes while maintaining model lightweightness. Ultimately, this achieves an effective balance between denoising accuracy and computational efficiency. Attached Figure Description

[0022] To more clearly illustrate the technical solutions in the embodiments of this application or related technologies, the drawings used in the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of this application. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0023] Figure 1 This is a schematic flowchart of an image denoising method based on spatial-frequency complementary fusion provided in an embodiment of this application; Figure 2 This is a schematic diagram of the initial network structure; Figure 3 This is a schematic diagram of the spatial frequency domain complementary fusion module structure; Figure 4 This is a schematic diagram of the frequency domain cross-attention block structure; Figure 5 This is a schematic diagram of the dual-view differential attention block structure; Figure 6 This is an application environment diagram of an image denoising method based on spatial-frequency complementary fusion in one embodiment of this application; Figure 7 This is a schematic diagram of the structure of a computer device provided in an embodiment of this application. Detailed Implementation

[0024] The technical solutions of the embodiments of this application will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only some embodiments of this application, and not all embodiments. Based on the embodiments of this application, all other embodiments obtained by those skilled in the art without creative effort are within the scope of protection of this application.

[0025] The purpose of this application is to provide an image denoising method, apparatus, medium, and product based on space-frequency complementary fusion, aiming to improve the accuracy and efficiency of image denoising while maintaining the lightweight nature of the model.

[0026] To make the above-mentioned objectives, features and advantages of this application more apparent and understandable, the application will be further described in detail below with reference to the accompanying drawings and specific embodiments.

[0027] In one exemplary embodiment, such as Figure 1 As shown, an image denoising method based on spatial-frequency complementary fusion is provided, which includes the following steps.

[0028] Step 1: Obtain the target image with noise.

[0029] Step 2: Input the target noisy image into the image denoising model to obtain the denoised image corresponding to the target noisy image.

[0030] The image denoising model is obtained by training the initial network. The initial network is constructed based on multiple spatial-frequency domain complementary fusion modules, which include frequency domain cross attention blocks and dual-view difference attention blocks. The frequency domain cross attention blocks are constructed based on wavelet transform and attention mechanisms, while the dual-view difference attention blocks are constructed based on attention mechanisms.

[0031] As an optional implementation method, the process of determining the image denoising model includes: Acquire multiple noisy sample images and their corresponding noise-free sample images; Build the initial network; The initial network is trained by taking the noisy image of each sample as input and the corresponding noiseless image of the sample as output, to obtain the image denoising model.

[0032] As an optional implementation method, such as Figure 2 As shown, the initial network includes a first convolutional layer, a cascaded fusion module, a second convolutional layer, and a first subtractor; the cascaded fusion module includes N space-frequency fusion stitching modules; each space-frequency fusion stitching module includes a spatial-frequency domain complementary fusion module and a first stitching module; N≥2; The input of the first spatial-frequency domain complementary fusion module is connected to the output of the first convolutional layer. The input of the first first stitching module is connected to the output of the first spatial-frequency domain complementary fusion module and the output of the first convolutional layer, respectively. The input of the nth spatial-frequency domain complementary fusion module is connected to the (n-1)th first stitching module. The input of the nth first stitching module is connected to the output of the nth spatial-frequency domain complementary fusion module and the output of the (n-1)th first stitching module, respectively. n = 2, 3, ..., N. The output of the Nth first stitching module is connected to the input of the second convolutional layer. The output of the second convolutional layer is connected to the input of the first subtractor.

[0033] As an optional implementation method, such as Figure 3 As shown, the spatial-frequency domain complementary fusion module includes: a frequency domain cross attention block, a dual-view difference attention block, a second stitching module, and a third convolutional layer, a batch normalization layer, a ReLU activation function, a fourth convolutional layer, and an adder connected in sequence. The input of the adder in the first spatial-frequency domain complementary fusion module is connected to the output of the fourth convolutional layer and the output of the first convolutional layer in the first spatial-frequency domain complementary fusion module, respectively. The input of the adder in the nth spatial-frequency domain complementary fusion module is connected to the output of the fourth convolutional layer and the output of the (n-1)th first splicing module, respectively. The output of the adder in the i-th spatial-frequency domain complementary fusion module is connected to the input of the frequency domain cross-attention block and the input of the dual-view difference attention block in the i-th spatial-frequency domain complementary fusion module, respectively. The outputs of the frequency domain cross-attention block and the dual-view difference attention block in the i-th spatial-frequency domain complementary fusion module are both connected to the input of the second stitching module in the i-th spatial-frequency domain complementary fusion module. The output of the second stitching module in the i-th spatial-frequency domain complementary fusion module serves as the output of the i-th spatial-frequency domain complementary fusion module and is connected to the input of the i-th first stitching module. i = 1, 2, ..., N.

[0034] As an optional implementation method, such as Figure 4 As shown, the frequency domain cross-attention block in any spatial-frequency domain complementary fusion module includes: a wavelet transform module, a first shaping module, a third splicing module, a second shaping module, a third shaping module, a first multiplier, a first Softmax activation function, a second multiplier, and a wavelet inverse transform module; The input of the wavelet transform module is connected to the output of the adder in the spatial-frequency domain complementary fusion module. The output of the wavelet transform module is connected to the input of the first shaping module and the input of the third splicing module. The output of the third splicing module is connected to the input of the second shaping module and the input of the third shaping module. The outputs of the second shaping module and the third shaping module are both connected to the input of the first multiplier. The output of the first multiplier is connected to the input of the first Softmax activation function. The outputs of the first Softmax activation function and the first shaping module are both connected to the input of the second multiplier. The output of the second multiplier is connected to the input of the inverse wavelet transform module. The output of the inverse wavelet transform module is connected to the input of the second splicing module in the spatial-frequency domain complementary fusion module.

[0035] As an optional implementation method, such as Figure 5 As shown, the dual-view difference attention block in any spatial-frequency domain complementary fusion module includes: a first pooling layer, a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer, a second pooling layer, an eighth convolutional layer, an upsampling layer, a second Softmax activation function, a third Softmax activation function, a second subtractor, and a third multiplier. The inputs of the first pooling layer, the seventh convolutional layer, and the second pooling layer are all connected to the output of the adder in the spatial-frequency domain complementary fusion module. The output of the first pooling layer is connected to the inputs of the fifth and sixth convolutional layers, respectively. The output of the second pooling layer is connected to the input of the eighth convolutional layer. The output of the eighth convolutional layer is connected to the input of the upsampling layer. The outputs of the sixth and seventh convolutional layers are both connected to the input of the second Softmax activation function. The outputs of the seventh and eighth convolutional layers are both connected to the input of the third Softmax activation function. The outputs of the second and third Softmax activation functions are both connected to the input of the second subtractor. The outputs of the second subtractor and the fifth convolutional layer are both connected to the input of the third multiplier. The output of the third multiplier is connected to the input of the second splicing module in the spatial-frequency domain complementary fusion module.

[0036] Specifically, the first and second pooling layers are the same pooling layers; the weight matrices and bias vectors of the first, sixth, third, fourth, fifth, sixth, seventh, and eighth convolutional layers are different; and the first, second, and third softmax activation functions are all the same softmax activation function.

[0037] As an optional implementation, step 2 includes the following steps.

[0038] Step 21: Input the noisy target image into the first convolutional layer of the image denoising model to obtain the first convolutional feature map.

[0039] Step 22: Input the first convolutional feature map into the cascaded fusion module in the image denoising model to obtain the cascaded fused feature map.

[0040] Specifically, step 22 includes: inputting the first convolutional feature map into the cascaded fusion module in the image denoising model; processing the first convolutional feature map by N spatial-frequency fusion stitching modules in the cascaded fusion module; and outputting the spatial-frequency domain complementary fusion feature map output by the Nth spatial-frequency fusion stitching module as the cascaded fusion feature map.

[0041] Among them, the i-th spatial frequency domain complementary fusion module is used for the input feature map The processing steps include the following steps.

[0042] Step S1: After sequential processing through the third convolutional layer, batch normalization layer, and ReLU activation function in the i-th spatial-frequency domain complementary fusion module, the first intermediate feature map in the i-th spatial-frequency domain complementary fusion module is obtained. When i=1, This is the first convolutional feature map; when i = 2, 3, ..., N, This is the output feature map of the first stitching module in the (i-1)th spatial-frequency domain complementary fusion module. The output feature map of the first stitching module in the ith spatial-frequency domain complementary fusion module is obtained by stitching the spatial-frequency domain complementary fusion feature map output by the ith spatial-frequency domain complementary fusion module with the output feature map of the first stitching module in the (i-1)th spatial-frequency domain complementary fusion module. The output feature map of the first stitching module in the eleventh spatial-frequency domain complementary fusion module is obtained by stitching the spatial-frequency domain complementary fusion feature map output by the eleventh spatial-frequency domain complementary fusion module with the first convolutional feature map.

[0043] Step S2: Input the first intermediate feature map in the i-th spatial-frequency domain complementary fusion module into the fourth convolutional layer in the i-th spatial-frequency domain complementary fusion module to obtain the second intermediate feature map in the i-th spatial-frequency domain complementary fusion module.

[0044] Step S3: [The text appears to be incomplete and contains several grammatical errors. A more accurate translation would require the full context.] Both the second intermediate feature map in the i-th spatial frequency domain complementary fusion module and the second intermediate feature map in the i-th spatial frequency domain complementary fusion module are input into the adder in the i-th spatial frequency domain complementary fusion module to obtain the third intermediate feature map in the i-th spatial frequency domain complementary fusion module.

[0045] Step S4: Input the third intermediate feature map in the i-th spatial-frequency domain complementary fusion module into the frequency domain cross attention block and the dual-view difference attention block in the i-th spatial-frequency domain complementary fusion module respectively, to obtain the fourth and fifth intermediate feature maps in the i-th spatial-frequency domain complementary fusion module.

[0046] The frequency domain cross-attention block in the i-th spatial frequency domain complementary fusion module is represented as: ; ; ; ; ; ; in, This refers to the low-frequency sub-band in the i-th spatial frequency domain complementary fusion module; For vertical high-frequency sub-band; This refers to the horizontal high-frequency sub-band in the i-th spatial frequency domain complementary fusion module; This refers to the diagonal high-frequency sub-band in the i-th spatial frequency domain complementary fusion module; This is a wavelet transform module; This is the third intermediate feature map in the i-th spatial frequency domain complementary fusion module; Let be the value vector in the frequency domain cross-attention block of the i-th spatial frequency domain complementary fusion module; This is the first shaping module; Let be the key vector in the frequency domain cross-attention block of the i-th spatial frequency domain complementary fusion module; This is the second shaping module; This is the third splicing module; For the query vector in the frequency domain cross-attention block of the i-th spatial frequency domain complementary fusion module; This is the third shaping module; This is the intermediate feature map in the i-th spatial frequency domain complementary fusion module; This is the first Softmax activation function; For transpose; Let be the dimensions of the key vector and query vector in the frequency domain cross-attention block of the i-th spatial frequency domain complementary fusion module; This is the fourth intermediate feature map in the i-th spatial frequency domain complementary fusion module; This is the inverse wavelet transform module.

[0047] The dual-view difference attention block in the i-th spatial frequency domain complementary fusion module is represented as follows: ; ; ; ; ; ; ; ; in, For the value vector in the dual-view difference attention block in the i-th spatial frequency domain complementary fusion module; This is the fifth convolutional layer; This is the first pooling layer; Let be the first key vector in the dual-view difference attention block of the i-th spatial frequency domain complementary fusion module; It is the sixth convolutional layer; For the query vector in the dual-view difference attention block of the i-th spatial frequency domain complementary fusion module; It is the seventh convolutional layer; The second key vector in the dual-view difference attention block of the i-th spatial frequency domain complementary fusion module; For upsampling layer; This is the eighth convolutional layer; This is the second pooling layer; This is the first-view attention map in the dual-view difference attention block of the i-th spatial frequency domain complementary fusion module; This is the second Softmax activation function; Let be the dimension of the first key vector, query vector, and second key vector in the dual-view difference attention block of the i-th spatial frequency domain complementary fusion module; This is the second-view attention map in the dual-view difference attention block of the i-th spatial frequency domain complementary fusion module; This is the third Softmax activation function; This is the difference attention map in the dual-view difference attention block of the i-th spatial frequency domain complementary fusion module; This is the parameter matrix of the second subtractor; This is the fifth intermediate feature map in the i-th spatial frequency domain complementary fusion module; It is the instrument for the third multiplication.

[0048] Step S5: Input the fourth and fifth intermediate feature maps from the i-th spatial frequency domain complementary fusion module into the second stitching module of the i-th spatial frequency domain complementary fusion module to obtain the spatial frequency domain complementary fusion feature map output by the i-th spatial frequency domain complementary fusion module.

[0049] Specifically, the formula for determining the spatial frequency domain complementary fusion feature map is as follows: ; in, This is the spatial frequency domain complementary fusion feature map output by the i-th spatial frequency domain complementary fusion module; This is the second splicing module; Step 23: Input the cascaded fused feature map into the second convolutional layer in the image denoising model to obtain the second convolutional feature map.

[0050] Step 24: Input both the second convolutional feature map and the target noisy image into the first subtractor in the image denoising model to obtain the denoised image.

[0051] Furthermore, to verify the effectiveness of the method in this application, comparative experiments were conducted with typical denoising methods on the publicly available dataset Kodak24. The experimental results are shown in Tables 1-3.

[0052] Table 1 Comparison of Denoising Results

[0053] Table 2 Ablation Experiment Results

[0054] Table 3. Comparison of Noise Reduction Efficiency

[0055] As shown in Table 1, under different noise levels (values ​​of 15, 25, 35, 50, and 75), the method proposed in this application achieved good denoising results in terms of peak signal-to-noise ratio and structural similarity index, indicating that the method can effectively suppress the impact of noise of different intensities on image quality. Furthermore, the ablation experiments shown in Table 2 demonstrate that the dual-view differential attention block and frequency domain cross-attention block designed in this application can effectively improve network performance. The optimal denoising result is obtained after the two modules work together, verifying the effectiveness of the proposed network structure and its constituent modules. In addition, to analyze the computational complexity and practical application efficiency of the method proposed in this application, Table 3 compares the floating-point operation volume, model parameter count, and training and testing time of different methods. Experimental results show that the method proposed in this application has lower model complexity than some high-performance denoising networks and can achieve better denoising performance with lower computational overhead, while also possessing high computational efficiency. The experimental results summarized in Tables 1 to 3 show that the method in this application, through the design of a spatial-frequency domain complementary fusion mechanism, can fully utilize the spatial and frequency domain information of the image in complex noise environments, thereby improving noise suppression and detail recovery capabilities while maintaining a low model computational burden. Therefore, this application, while maintaining a lightweight model, enhances the network's ability to recover image structures in complex noise scenes, ultimately achieving an effective balance between denoising accuracy and computational efficiency.

[0056] The image denoising method based on spatial-frequency complementary fusion provided in this application can be applied to, for example... Figure 6In the application environment shown, terminal 102 communicates with server 104 via a network. A data storage system can store the data that server 104 needs to process. The data storage system can be set up independently, integrated into server 104, or placed in the cloud or on another server. Terminal 102 can send a target noisy image to server 104. After receiving the target noisy image, server 104 inputs it into an image denoising model to obtain a denoised image corresponding to the target noisy image. Server 104 can then feed back the obtained denoised image to terminal 102. Furthermore, in some embodiments, the image denoising method based on space-frequency complementary fusion can also be implemented independently by server 104 or terminal 102. For example, terminal 102 can directly perform image denoising based on space-frequency complementary fusion on the target noisy image, or server 104 can obtain the target noisy image from the data storage system and perform image denoising based on space-frequency complementary fusion on the target noisy image.

[0057] In one exemplary embodiment, a computer device is provided, including: a memory, a processor, and a computer program stored in the memory and executable on the processor, wherein the processor executes the computer program to implement an image denoising method based on spatial-frequency complementary fusion.

[0058] In one exemplary embodiment, a computer-readable storage medium is provided having a computer program stored thereon that, when executed by a processor, implements an image denoising method based on spatial-frequency complementary fusion.

[0059] In one exemplary embodiment, a computer program product is provided, including a computer program that, when executed by a processor, implements an image denoising method based on spatial-frequency complementary fusion.

[0060] In one exemplary embodiment, a computer device is provided, which may be a server or a terminal, and its internal structure diagram may be as follows. Figure 7As shown, this computer device includes a processor, memory, input / output (I / O) interfaces, and a communication interface. The processor, memory, and I / O interfaces are connected via a system bus, and the communication interface is also connected to the system bus via the I / O interfaces. The processor provides computational and control capabilities. The memory includes non-volatile storage media and internal memory. The non-volatile storage media stores the operating system, computer programs, and databases. The internal memory provides the environment for the operating system and computer programs stored in the non-volatile storage media. The I / O interfaces are used for exchanging information between the processor and external devices. The communication interface is used for communicating with external terminals via a network connection. When the computer program is executed by the processor, it implements an image denoising method based on space-frequency complementary fusion.

[0061] Those skilled in the art will understand that Figure 7 The structure shown is merely a block diagram of a portion of the structure related to the present application and does not constitute a limitation on the computer device to which the present application is applied. Specific computer devices may include more or fewer components than those shown in the figure, or combine certain components, or have different component arrangements.

[0062] Those skilled in the art will understand that all or part of the processes in the above embodiments can be implemented by a computer program instructing related hardware. The computer program can be stored in a non-volatile computer-readable storage medium. When executed, the computer program can include the processes of the embodiments described above. Any references to memory, databases, or other media used in the embodiments provided in this application can include at least one of non-volatile and volatile memory. Non-volatile memory can include read-only memory (ROM), magnetic tape, floppy disk, flash memory, optical memory, high-density embedded non-volatile memory, resistive random access memory (ReRAM), magnetic random access memory (MRAM), ferroelectric random access memory (FRAM), phase change memory (PCM), graphene memory, etc. Volatile memory can include random access memory (RAM) or external cache memory, etc. By way of illustration and not limitation, RAM can take many forms, such as Static Random Access Memory (SRAM) or Dynamic Random Access Memory (DRAM).

[0063] The databases involved in the embodiments provided in this application may include at least one type of relational database and non-relational database. Non-relational databases may include, but are not limited to, blockchain-based distributed databases. The processors involved in the embodiments provided in this application may be general-purpose processors, central processing units, graphics processing units, digital signal processors, programmable logic devices, quantum computing-based data processing logic devices, etc., and are not limited to these.

[0064] It should be noted that the user information (including but not limited to user device information, user personal information, etc.) and data (including but not limited to data used for analysis, data stored, data displayed, etc.) involved in this application are all information and data authorized by the user or fully authorized by all parties, and the collection, use and processing of the relevant data must comply with relevant regulations.

[0065] The technical features of the above embodiments can be combined in any way. For the sake of brevity, not all possible combinations of the technical features in the above embodiments are described. However, as long as there is no contradiction in the combination of these technical features, they should be considered to be within the scope of this specification.

[0066] This document uses specific examples to illustrate the principles and implementation methods of this application. The descriptions of the above embodiments are only for the purpose of helping to understand the methods and core ideas of this application. Furthermore, those skilled in the art will recognize that, based on the ideas of this application, there will be changes in the specific implementation methods and application scope. Therefore, the content of this specification should not be construed as a limitation of this application.

Claims

1. An image denoising method based on spatial-frequency complementary fusion, characterized in that, The image denoising method includes: Acquire the target image containing noise; The target noisy image is input into the image denoising model to obtain the denoised image corresponding to the target noisy image; the image denoising model is obtained by training the initial network; the initial network is constructed based on multiple spatial-frequency domain complementary fusion modules, the spatial-frequency domain complementary fusion module includes a frequency domain cross attention block and a dual-view difference attention block, the frequency domain cross attention block is constructed based on wavelet transform and attention mechanism, and the dual-view difference attention block is constructed based on attention mechanism.

2. The image denoising method based on spatial-frequency complementary fusion according to claim 1, characterized in that, The process of determining the image denoising model includes: Acquire multiple noisy sample images and their corresponding noise-free sample images; Construct the initial network; The initial network is trained by taking the noisy image of each sample as input and the corresponding noiseless image of the sample as output, to obtain the image denoising model.

3. The image denoising method based on spatial-frequency complementary fusion according to claim 1, characterized in that, The initial network includes a first convolutional layer, a cascaded fusion module, a second convolutional layer, and a first subtractor; the cascaded fusion module includes N space-frequency fusion stitching modules; each space-frequency fusion stitching module includes a spatial-frequency domain complementary fusion module and a first stitching module; N≥2; The input of the first spatial-frequency domain complementary fusion module is connected to the output of the first convolutional layer. The input of the first first stitching module is connected to the output of the first spatial-frequency domain complementary fusion module and the output of the first convolutional layer, respectively. The input of the nth spatial-frequency domain complementary fusion module is connected to the (n-1)th first stitching module. The input of the nth first stitching module is connected to the output of the nth spatial-frequency domain complementary fusion module and the output of the (n-1)th first stitching module, respectively. n = 2, 3, ..., N. The output of the Nth first stitching module is connected to the input of the second convolutional layer. The output of the second convolutional layer is connected to the input of the first subtractor.

4. The image denoising method based on spatial-frequency complementary fusion according to claim 3, characterized in that, The spatial frequency domain complementary fusion module includes: a frequency domain cross attention block, a dual-view difference attention block, a second stitching module, and a third convolutional layer, a batch normalization layer, a ReLU activation function, a fourth convolutional layer, and an adder connected in sequence. The input of the adder in the first spatial-frequency domain complementary fusion module is connected to the output of the fourth convolutional layer and the output of the first convolutional layer in the first spatial-frequency domain complementary fusion module, respectively. The input of the adder in the nth spatial-frequency domain complementary fusion module is connected to the output of the fourth convolutional layer and the output of the (n-1)th first splicing module, respectively. The output of the adder in the i-th spatial-frequency domain complementary fusion module is connected to the input of the frequency domain cross-attention block and the input of the dual-view difference attention block in the i-th spatial-frequency domain complementary fusion module, respectively. The outputs of the frequency domain cross-attention block and the dual-view difference attention block in the i-th spatial-frequency domain complementary fusion module are both connected to the input of the second stitching module in the i-th spatial-frequency domain complementary fusion module. The output of the second stitching module in the i-th spatial-frequency domain complementary fusion module serves as the output of the i-th spatial-frequency domain complementary fusion module and is connected to the input of the i-th first stitching module. i = 1, 2, ..., N.

5. The image denoising method based on spatial-frequency complementary fusion according to claim 4, characterized in that, Each frequency domain cross-attention block in any spatial frequency domain complementary fusion module includes: a wavelet transform module, a first shaping module, a third splicing module, a second shaping module, a third shaping module, a first multiplier, a first Softmax activation function, a second multiplier, and a wavelet inverse transform module; The input of the wavelet transform module is connected to the output of the adder in the spatial-frequency domain complementary fusion module. The output of the wavelet transform module is connected to the input of the first shaping module and the input of the third splicing module. The output of the third splicing module is connected to the input of the second shaping module and the input of the third shaping module. The outputs of the second shaping module and the third shaping module are both connected to the input of the first multiplier. The output of the first multiplier is connected to the input of the first Softmax activation function. The outputs of the first Softmax activation function and the first shaping module are both connected to the input of the second multiplier. The output of the second multiplier is connected to the input of the inverse wavelet transform module. The output of the inverse wavelet transform module is connected to the input of the second splicing module in the spatial-frequency domain complementary fusion module.

6. The image denoising method based on spatial-frequency complementary fusion according to claim 5, characterized in that, Each dual-view difference attention block in any spatial frequency domain complementary fusion module includes: a first pooling layer, a fifth convolutional layer, a sixth convolutional layer, a seventh convolutional layer, a second pooling layer, an eighth convolutional layer, an upsampling layer, a second softmax activation function, a third softmax activation function, a second subtractor, and a third multiplier. The inputs of the first pooling layer, the seventh convolutional layer, and the second pooling layer are all connected to the output of the adder in the spatial-frequency domain complementary fusion module. The output of the first pooling layer is connected to the inputs of the fifth and sixth convolutional layers, respectively. The output of the second pooling layer is connected to the input of the eighth convolutional layer. The output of the eighth convolutional layer is connected to the input of the upsampling layer. The outputs of the sixth and seventh convolutional layers are both connected to the input of the second Softmax activation function. The outputs of the seventh and eighth convolutional layers are both connected to the input of the third Softmax activation function. The outputs of the second and third Softmax activation functions are both connected to the input of the second subtractor. The outputs of the second subtractor and the fifth convolutional layer are both connected to the input of the third multiplier. The output of the third multiplier is connected to the input of the second splicing module in the spatial-frequency domain complementary fusion module.

7. The image denoising method based on spatial-frequency complementary fusion according to claim 3, characterized in that, The target noisy image is input into an image denoising model to obtain a denoised image corresponding to the target noisy image, including: The target noisy image is input into the first convolutional layer of the image denoising model to obtain the first convolutional feature map; The first convolutional feature map is input into the cascaded fusion module in the image denoising model to obtain the cascaded fused feature map; The cascaded fused feature map is input into the second convolutional layer in the image denoising model to obtain the second convolutional feature map; The second convolutional feature map and the target noisy image are both input into the first subtractor in the image denoising model to obtain the denoised image.

8. A computer device, comprising: A memory, a processor, and a computer program stored in the memory and executable on the processor, characterized in that the processor executes the computer program to implement the image denoising method based on spatial-frequency complementary fusion as described in any one of claims 1-7.

9. A computer-readable storage medium having a computer program stored thereon, characterized in that, When executed by a processor, the computer program implements the image denoising method based on spatial-frequency complementary fusion as described in any one of claims 1-7.

10. A computer program product, comprising a computer program, characterized in that, When executed by a processor, the computer program implements the image denoising method based on spatial-frequency complementary fusion as described in any one of claims 1-7.

Images