Image denoising method and device, electronic equipment and storage medium

Abstract

This invention relates to an image denoising method, apparatus, electronic device, and storage medium. The method includes: acquiring a denoising system model, the denoising system model including a multi-level feature extraction module, a feature fusion module, and a Transformer module; acquiring an experimental noisy image and corresponding auxiliary feature images; preprocessing the experimental noisy image to obtain a target image; inputting the target image and auxiliary feature images into the denoising system model; extracting features from the target image and auxiliary feature images through the multi-level feature extraction module to obtain feature images; fusing the feature images through the feature fusion module to obtain a fused image; and processing the fused image based on an auxiliary feature-guided attention mechanism through the Transformer module to obtain a denoised image.

Description

Technical Field

[0001] This invention relates to the field of image denoising technology, and more specifically to an image denoising method, apparatus, electronic device, and storage medium. Background Technology

[0002] Monte Carlo (MC) path tracing is a popular realistic rendering technique widely used in computer animation, filmmaking, video games, and other fields. Compared to other rendering techniques, the Monte Carlo method is unbiased and highly versatile for various visual rendering effects. However, to produce high-quality rendered images, MC path tracing requires a large number of pixel samples, which is very time-consuming. Therefore, current graphics researchers tend to reduce the number of light samples and improve image quality.

[0003] Image-space-based Monte Carlo (MC) denoising is a popular technique. It first renders a noisy image at a low path sampling rate, then applies filtering operators to remove noise and reconstruct a high-quality target image. Traditional image-space MC denoising is achieved through weighted local regression: assuming that pixel values ​​can be locally approximated by the Taylor multinomial expansion of neighboring pixels; however, these methods typically require a trade-off between bias and variance. In recent years, deep learning methods for MC denoising have been proposed: employing multilayer perceptrons to learn parameters for joint filtering and non-local mean filtering from pairs of noisy and non-noisy images; using neural networks to predict the filter kernel for each pixel; eliminating intermediate filters to directly output the denoised image, and so on.

[0004] In existing technologies, Convolutional Neural Networks (CNNs) typically use small convolutional kernels. Although a larger receptive field can be obtained by stacking more convolutional and pooling layers, the interaction between two distant features is indirectly calculated, and the weights between pixels obtained by convolution are content-independent. Secondly, denoising is usually guided by cascading auxiliary features with noisy images, which limits the influence of auxiliary features. Furthermore, the cascading method can be understood as adding biases to the hidden layers based on conditional representations. Finally, the loss function plays an important role in the training and performance evaluation of the model, and choosing an appropriate loss function is crucial for building a high-performance model. Summary of the Invention

[0005] In view of this, it is necessary to provide an image denoising method, apparatus, electronic device and storage medium that can effectively use auxiliary features to guide the denoising of MC images.

[0006] To achieve the above objectives, in a first aspect, the present invention provides an image denoising method, comprising:

[0007] Obtain a denoising system model, which includes a multi-level feature extraction module, a feature fusion module, and a Transformer module;

[0008] Acquire the noisy experimental image and its corresponding auxiliary feature image, and preprocess the noisy experimental image to obtain the target image;

[0009] The target image and the auxiliary feature image are input into the denoising system model. The multi-level feature extraction module extracts features from the target image and the auxiliary feature image to obtain a feature image. The feature fusion module then fuses the feature image to obtain a fused image. Finally, the Transformer module processes the fused image based on an attention mechanism guided by auxiliary features to obtain a denoised image.

[0010] Further, the preprocessing of the noisy experimental image to obtain the target image includes:

[0011] The noisy experimental image was preprocessed by logarithmic transformation to obtain the target image.

[0012] Furthermore, the denoising system model includes a five-layer Transformer module, which includes: a channel attention unit, a spatial attention unit, and a fusion auxiliary feature unit;

[0013] The input of each Transformer module is connected to the feature fusion module, and the output of the previous Transformer module is connected to the fusion auxiliary feature unit in the next Transformer module.

[0014] Specifically, in the i-th layer Transformer module:

[0015] The channel attention unit is used to receive the fused image, obtain channel weights based on the fused image, and perform a dot product calculation between the channel weights and the fused image to obtain the channel attention result;

[0016] A spatial attention unit is used to obtain spatial position weights based on the channel attention results, and to perform a dot product calculation between the spatial position weights and the fused image to obtain a fused attention image;

[0017] The fusion auxiliary feature unit is used to process the fused attention image and the denoised image output by the (i-1)th layer Transformer module through an attention mechanism guided by auxiliary features to obtain the denoised image output by the i-th layer Transformer module.

[0018] Furthermore, the fusion auxiliary feature unit is used to standardize the fused attention image and the denoised image output by the (i-1)th layer Transformer module to obtain a processed image, and to obtain a dimensionality-reduced image by global pooling based on the processed image, and to obtain a weighted fused image by weighted fusion based on the dimensionality-reduced image, and to obtain a denoised image output by the i-th layer Transformer module by inverse standardization based on the weighted fused image.

[0019] Furthermore, the channel attention unit is used to calculate the channel weights based on the fused image through adaptive average pooling and two-layer linear transformation.

[0020] Furthermore, the spatial attention unit is used to calculate pixel similarity and correlation based on the channel attention results to obtain spatial location weights.

[0021] Furthermore, the loss function of the denoising system model is:

[0022] L(n,f,r)=l1(n,r)*(1+pinish(n,f,r));

[0023] In the formula: L(n,f,r) is the asymmetric loss function, pinish(,f,r) is the penalty function, n is the target image, f is the auxiliary feature image, and r is the reference image.

[0024] In a second aspect, the present invention also provides an image denoising apparatus, comprising:

[0025] The model acquisition unit is used to acquire the denoising system model, which includes a multi-level feature extraction module, a feature fusion module, and a Transformer module.

[0026] An image preprocessing unit is used to acquire the experimental noisy image and the corresponding auxiliary feature image, and to preprocess the experimental noisy image to obtain the target image;

[0027] The image denoising unit is used to input the target image and the auxiliary feature image into the denoising system model, extract features from the target image and the corresponding auxiliary feature image through the multi-level feature extraction module to obtain a feature image, and fuse the feature image through the feature fusion module to obtain a fused image. The Transformer module is used to process the fused image based on the auxiliary feature-guided attention mechanism to obtain a denoised image.

[0028] Thirdly, the present invention also provides an electronic device for executing the program stored in the memory to implement the steps of an image denoising method as described in any of the above implementations.

[0029] Fourthly, the present invention also provides a non-transitory computer-readable storage medium for storing a computer program capable of implementing the steps of an image denoising method in any of the above implementations.

[0030] This invention provides an image denoising method, apparatus, electronic device, and storage medium. By acquiring a denoising system model, including a multi-level feature extraction module, a feature fusion module, and a Transformer module, an experimental noisy image and corresponding auxiliary feature images are acquired. The noisy experimental image is preprocessed to obtain a target image. The target image and the auxiliary feature images are input into the denoising system model. The multi-level feature extraction module extracts features from the target image and the auxiliary feature images to obtain feature images. The feature fusion module fuses the feature images to obtain a fused image. The Transformer module processes the fused image through an auxiliary feature-guided attention mechanism to obtain a denoised image. Compared to existing technologies, this invention proposes an image denoising method that can effectively calculate the interaction between long-distance features in a noisy image, calculate pixel weights based on content relevance, more effectively use auxiliary features to guide MC image denoising, and improve the loss function to make the denoising model more robust and converge faster. Attached Figure Description

[0031] To more clearly illustrate the technical solutions in the embodiments of the present invention, the accompanying drawings used in the description of the embodiments will be briefly introduced below. Obviously, the accompanying drawings described below are only some embodiments of the present invention. For those skilled in the art, other drawings can be obtained based on these drawings without creative effort.

[0032] Figure 1 A flowchart illustrating an embodiment of an image denoising method provided by the present invention;

[0033] Figure 2 A flowchart illustrating the structure of a denoising system model according to an embodiment of an image denoising method provided by the present invention;

[0034] Figure 3 A flowchart illustrating the structure of the Transformer module in an embodiment of an image denoising method provided by the present invention;

[0035] Figure 4This is an effect diagram of a denoising system model according to an embodiment of an image denoising method provided by the present invention;

[0036] Figure 5 A line graph of the asymmetric loss function of an embodiment of an image denoising method provided by the present invention;

[0037] Figure 6 A line graph of the L1 loss function of an embodiment of an image denoising method provided by the present invention;

[0038] Figure 7 This is a schematic diagram of an embodiment of an image denoising device provided by the present invention;

[0039] Figure 8 A schematic diagram of an embodiment of the electronic device provided by the present invention. Detailed Implementation

[0040] The technical solutions of the embodiments of the present invention will be clearly and completely described below with reference to the accompanying drawings. Obviously, the described embodiments are only a part of the embodiments of the present invention, and not all of them. All other embodiments obtained by those skilled in the art based on the embodiments of the present invention without creative effort are within the scope of protection of the present invention.

[0041] In the description of the embodiments of this application, unless otherwise stated, "a plurality of" means two or more.

[0042] In this embodiment of the invention, the terms "comprising" and "having" and any variations thereof are intended to cover non-exclusive inclusion, for example, a process, method, apparatus, product or device that includes a series of steps or modules is not necessarily limited to those steps or modules that are explicitly listed, but may include other steps or modules that are not explicitly listed or that are inherent to such process, method, product or device.

[0043] The naming or numbering of steps in the embodiments of the present invention does not mean that the steps in the method flow must be executed in the time / logical order indicated by the naming or numbering. The execution order of the named or numbered process steps can be changed according to the technical purpose to be achieved, as long as the same or similar technical effect can be achieved.

[0044] In this document, the term "embodiment" means that a particular feature, structure, or characteristic described in connection with an embodiment may be included in at least one embodiment of the invention. The appearance of this phrase in various places throughout the specification does not necessarily refer to the same embodiment, nor is it a separate or alternative embodiment mutually exclusive with other embodiments. It will be explicitly and implicitly understood by those skilled in the art that the embodiments described herein can be combined with other embodiments.

[0045] This invention provides an image denoising method, apparatus, electronic device, and computer-readable storage medium, which are described below.

[0046] Figure 1 A flowchart of an embodiment of an image denoising method provided by the present invention includes:

[0047] S110. Obtain a denoising system model, wherein the denoising system model includes a multi-level feature extraction module, a feature fusion module, and a Transformer module;

[0048] S120. Acquire the experimental noisy image and the corresponding auxiliary feature image, and preprocess the experimental noisy image to obtain the target image;

[0049] S130. Input the target image and the auxiliary feature image into the denoising system model. Extract features from the target image and the auxiliary feature image through the multi-level feature extraction module to obtain a feature image. Then, fuse the feature image through the feature fusion module to obtain a fused image. Finally, process the fused image based on the auxiliary feature-guided attention mechanism through the Transformer module to obtain a denoised image.

[0050] It is understandable that this invention combines spatial attention mechanism and channel attention mechanism, comprehensively considering the interaction between pixels from the spatial dimension and channel dimension. Moreover, the attention mechanism is essentially a non-local mean filter with edge preservation, resulting in a more reasonable denoising effect. Furthermore, it improves the method of guiding noisy images by auxiliary features. The cascaded conditionalization method is equivalent to the conditional bias of noisy images. Inspired by conditional normalization in image style transfer, in addition to conditional bias, it should also involve conditional scaling, that is, scaling the hidden layer through conditional representation.

[0051] Figure 2 A flowchart illustrating the structure of a denoising system model according to an embodiment of an image denoising method provided by the present invention includes:

[0052] In step S110, a denoising system model is obtained, which includes a multi-level feature extraction module, a feature fusion module, and a Transformer module.

[0053] It is understandable that the denoising system model can recover the path-sampled Monte Carlo rendered image well. Using mean square error, peak signal-to-noise ratio, and structured loss as evaluation indicators, the denoising system model has improved in mean square error, peak signal-to-noise ratio, and structured loss compared with the current advanced Monte Carlo denoising methods.

[0054] In step S120, the preprocessing of the noisy experimental image to obtain the target image includes:

[0055] The noisy experimental image was preprocessed by logarithmic transformation to obtain the target image;

[0056] in,

[0057]

[0058]

[0059] In the formula: c noisy The original experimental image with noise. For the noisy image after logarithmic transformation, This is the noisy image after anti-logarithmic transformation.

[0060] Understandably, training and validation were performed using the publicly available ACFM dataset, which contains images provided by [Bitterli 2016], rendered using the Physical Graphics Renderer (Tungsten). Each shot provides a noisy color image and a corresponding noise-assisted feature buffer, including depth, normal, and albedo. Corresponding reference images for these experimental noisy images are also provided. All experimental noisy images are rendered at 32 spp per pixel, while the reference images are rendered at 32 spp per pixel. We randomly selected 95% of the shots as the training dataset and the remaining 5% as the validation dataset, dividing all shots into 128×128 blocks. We used the entire low-path sampling rate noisy image as input, performing a logarithmic transformation on the noisy image before feeding it into our system. Because specular reflections have a high dynamic range, large variations and arbitrary correlations in the input make the iterative process highly unstable. Therefore, performing a logarithmic transformation on each color channel of the input image significantly reduces the range of color value variations, greatly improving the model output.

[0061] In step S130, the target image and the auxiliary feature image are input into the denoising system model. The multi-level feature extraction module extracts features from the target image and the auxiliary feature image to obtain a feature image. The feature fusion module then fuses the feature image to obtain a fused image. Finally, the Transformer module processes the fused image through an auxiliary feature-guided attention mechanism to obtain a denoised image.

[0062] in,

[0063]

[0064] In the formula: To assist in the normalization of feature images, The original auxiliary feature image,

[0065] The mean of the auxiliary feature image, The standard deviation of the auxiliary feature image;

[0066]

[0067] In the formula: The result of normalization for noisy images. The original noisy image, The mean of the noisy image. The standard deviation of the noisy image;

[0068]

[0069] In the formula: W fusion W f and W n These are learnable linear transformations;

[0070]

[0071] It is understandable that the auxiliary feature image and the noisy image are standardized, then global average pooling is used to reduce the dimensionality of each image, the processed feature images are weighted and fused, and then inverse standardization is performed to obtain the final fused image.

[0072] Figure 3 A flowchart illustrating the structure of a Transformer module in an embodiment of an image denoising method provided by the present invention includes:

[0073] The denoising system model includes a five-layer Transformer module, which includes: a channel attention unit, a spatial attention unit, and a fusion auxiliary feature unit.

[0074] The input of each Transformer module is connected to the feature fusion module, and the output of the previous Transformer module is connected to the fusion auxiliary feature unit in the next Transformer module.

[0075] Specifically, in the i-th layer Transformer module:

[0076] The channel attention unit is used to receive the fused image, obtain channel weights based on the fused image, and perform a dot product calculation between the channel weights and the fused image to obtain the channel attention result;

[0077] A spatial attention unit is used to obtain spatial position weights based on the channel attention results, and to perform a dot product calculation between the spatial position weights and the fused image to obtain a fused attention image;

[0078] The fusion auxiliary feature unit is used to process the fused attention image and the denoised image output by the (i-1)th layer Transformer module through an attention mechanism guided by auxiliary features to obtain the denoised image output by the i-th layer Transformer module.

[0079] Furthermore,

[0080] The channel attention unit is used to calculate channel weights based on the fused image through adaptive average pooling and two-layer linear transformation.

[0081] It is understandable that the channel attention mechanism focuses on the relationship between different channels in the fused image. It enhances the channels with more important information after the auxiliary feature image and the noisy image are fused by weighting each channel, while suppressing unimportant channels. The channel attention used in the denoising system model is calculated by one layer of adaptive average pooling and two layers of linear transformation to calculate the attention weight of each channel. Then, the input fused image and the attention weight are multiplied to achieve the effect of channel attention. It should be noted that the channel attention used in this model does not reduce the number of channels; the value of i is in the range of [2, 5], where i is the number of layers in the Transformer module.

[0082] Furthermore,

[0083] The spatial attention unit is used to calculate pixel similarity and correlation based on the channel attention results to obtain spatial location weights.

[0084] Understandably, spatial attention focuses on the relationships between different spatial locations (pixels) in the feature map. It determines the weight of each location by calculating the similarity and relevance of different pixels, thus strengthening the weights of pixels with important information and suppressing the weights of unimportant locations. To balance computation and memory usage, we divide the input fused image into non-overlapping blocks of spatial size b*b, thereby generating... The algorithm divides the computation into blocks and then performs global attention within each block, significantly reducing the overall computational complexity. Meanwhile, in order to maintain information interaction between blocks, a pixel band is extended around each block and added to the global attention calculation of each block; the spatial attention unit also includes attention fusion. After fusing the noisy image and the auxiliary feature image, it is first put into channel attention for channel weight calculation. The result of the dot product of the channel weight and the image after fusing the original auxiliary features is used as the input of spatial attention to calculate the spatial position weight. The spatial position weight result is multiplied by the image after fusing the original auxiliary features to obtain the final output result of the fused attention.

[0085] Figure 4 The image denoising system model of an embodiment of the image denoising method provided by the present invention is shown in the following diagram:

[0086] The fusion auxiliary feature unit is used to standardize the fused attention image and the denoised image output by the (i-1)th layer Transformer module to obtain a processed image, and then perform global pooling dimensionality reduction on the processed image to obtain a dimensionality-reduced image. Based on the dimensionality-reduced image, a weighted fusion image is obtained through weighted fusion processing, and finally, the denoised image output by the i-th layer Transformer module is obtained through inverse standardization on the weighted fusion image.

[0087] It is understandable that the value of i is in the range of [2, 5], where i is the layer number of the Transformer module. The function of the fusion auxiliary feature unit is to enable the auxiliary features to better guide the noisy image. The traditional cascade method of splicing auxiliary features and noisy images is abandoned. A new scheme is adopted, which standardizes the fusion attention image and the processed image, and then uses global average pooling to reduce the dimensionality. The processed fusion attention image and the processed image are then weighted and fused, and then inverse standardization is performed to obtain the final fusion feature image. Finally, the image is decoded by the encoder to obtain the final denoised image.

[0088] Furthermore, the formula for the attention mechanism is as follows:

[0089]

[0090] Q = W Q X i-1 =W K X i-1 =W Q X i-1 ;

[0091]

[0092]

[0093] in: f is the output of the (i-1)th Transformer module. A The image is fused (the auxiliary feature image and the noisy image are obtained through the feature fusion module).

[0094] Understandably, it would be intuitively unreasonable to directly use the standard attention mechanism to calculate Q, K, and V using auxiliary features and a noisy image. Q and K are used to calculate the attention score (weights) in the attention operator, and these weights can be calculated using auxiliary features. However, V represents the idea that the pixel values ​​of the image should not be affected by auxiliary features (depth, normals, and albedo). Therefore, we use auxiliary features and a noisy image to calculate Q and K in the attention operator, while V is calculated using the noisy image alone. Q, K, and... and f A Related, while V is only related to Related.

[0095] Furthermore,

[0096]

[0097]

[0098] in: This is the output of the (i-1)th layer Transformer module. is the input to the i-th layer Transformer module, and CSA is the fusion attention mechanism.

[0099] Understandably, the entire denoising system model uses five Transformer modules connected in series. Each layer takes the fused auxiliary features and the result of the previous Transformer module as input, while the first Transformer module takes the fused image after feature extraction as input.

[0100] Figure 5 A line graph of the asymmetric loss function of an embodiment of the image denoising method provided by the present invention and Figure 6 A line graph of the L1 loss function of an embodiment of the image denoising method provided by the present invention includes:

[0101] The loss function of the denoising system model is:

[0102] L(n,f,r)=l1(n,r)*(1+pinish(n,f,r));

[0103] Where: L(n,f,r) is the asymmetric loss function, pinish(,f,r) is the penalty function, n is the target image, f is the auxiliary feature image, and r is the reference image.

[0104] It's understandable that common loss functions like L1 and L2 exhibit symmetry on the coordinate axes; that is, when the distance between the denoised image and the reference image is the same, the loss value of the denoising system model is also the same. However, this ignores the offset between the denoised image and the noisy image. When the denoising system model produces two results with the same distance from the reference image, intuitively, we tend to prefer the result with a smaller offset from the original noisy image because it retains more details of the original image. The new loss function is determined by the noisy image, the auxiliary feature image, and the reference image. First, it determines whether the noisy image and the auxiliary feature image are on the same side of the reference image. If they are, the penalty term is 0; otherwise, it is determined by the offset distance of the denoised image. With other parameters unchanged, the L1 loss function and the new loss function are used to train the denoising system model. The loss value of the denoising system model changes after multiple iterations of training, as shown below. Figure 5 and Figure 6 As shown, it can be seen that the modified loss function significantly improves the robustness and convergence of the entire denoising system model.

[0105] To better implement the image denoising method in this embodiment of the invention, based on this method, please refer to the corresponding documentation. Figure 7 , Figure 7 A schematic diagram of an embodiment of the device 700 provided by the present invention includes:

[0106] The model acquisition unit 701 is used to acquire a denoising system model, which includes a multi-level feature extraction module, a feature fusion module, and a Transformer module.

[0107] The image preprocessing unit 702 is used to acquire the experimental noisy image and the corresponding auxiliary feature image, and to preprocess the experimental noisy image to obtain the target image;

[0108] The image denoising unit 703 is used to input the target image and the auxiliary feature image into the denoising system model, extract features from the target image and the corresponding auxiliary feature image through the multi-level feature extraction module to obtain a feature image, and fuse the feature image through the feature fusion module to obtain a fused image. Finally, the fused image is processed by the Transformer module based on the auxiliary feature-guided attention mechanism to obtain a denoised image.

[0109] The image denoising device provided in the above embodiments can realize the technical solution described in the above image denoising method embodiments. The specific implementation principle of each module or unit can be found in the corresponding content in the above image denoising method embodiments, and will not be repeated here.

[0110] like Figure 8As shown, the present invention also provides an electronic device 800. The electronic device 800 includes a processor 801, a memory 802, and a display 803. Figure 8 Only some components of the electronic device 800 are shown, but it should be understood that it is not required to implement all the components shown, and more or fewer components may be implemented instead.

[0111] In some embodiments, memory 802 may be an internal storage unit of electronic device 800, such as a hard disk or memory of electronic device 800. In other embodiments, memory 802 may also be an external storage device of electronic device 800, such as a plug-in hard disk, smart media card (SMC), secure digital (SD) card, flash card, etc. equipped on electronic device 800.

[0112] Furthermore, the memory 802 may include both internal storage units of the electronic device 800 and external storage devices. The memory 802 is used to store application software and various types of data installed on the electronic device 800.

[0113] In some embodiments, processor 801 may be a central processing unit (CPU), microprocessor, or other data processing chip, used to run program code stored in memory 802 or process data, such as an image denoising method in this invention.

[0114] In some embodiments, display 803 may be an LED display, a liquid crystal display, a touch-sensitive liquid crystal display, or an OLED (Organic Light-Emitting Diode) touchscreen. Display 803 is used to display information from electronic device 800 and to display a visual user interface. Components 801-803 of electronic device 800 communicate with each other via a system bus.

[0115] In some embodiments of the present invention, when the processor 801 executes the image denoising program in the memory 802, the following steps can be implemented:

[0116] Obtain a denoising system model, which includes a multi-level feature extraction module, a feature fusion module, and a Transformer module;

[0117] Acquire the noisy experimental image and its corresponding auxiliary feature image, and preprocess the noisy experimental image to obtain the target image;

[0118] The target image and the auxiliary feature image are input into the denoising system model. The multi-level feature extraction module extracts features from the target image and the corresponding auxiliary feature image to obtain a feature image. The feature fusion module fuses the feature image to obtain a fused image. The Transformer module processes the fused image based on an attention mechanism guided by auxiliary features to obtain a denoised image.

[0119] It should be understood that when the processor 801 executes the computer program in the memory 802, in addition to the functions described above, it can also perform other functions, as can be found in the description of the corresponding method embodiments above.

[0120] Furthermore, this embodiment of the invention does not specifically limit the type of electronic device 800 mentioned. Electronic device 800 can be a mobile phone, tablet computer, personal digital assistant (PDA), wearable device, laptop computer, or other portable electronic device. Exemplary embodiments of portable electronic devices include, but are not limited to, portable electronic devices running iOS, Android, Microsoft, or other operating systems. The aforementioned portable electronic device can also be other portable electronic devices, such as a laptop computer with a touch-sensitive surface (e.g., a touch panel). It should also be understood that in some other embodiments of the invention, electronic device 800 may not be a portable electronic device, but rather a desktop computer with a touch-sensitive surface (e.g., a touch panel).

[0121] In another aspect, the present invention also provides a non-transitory computer-readable storage medium having a computer program stored thereon, which, when executed by a processor, is implemented to perform a method for denoising an image provided by the methods described above, the method comprising:

[0122] Obtain a denoising system model, which includes a multi-level feature extraction module, a feature fusion module, and a Transformer module;

[0123] Acquire the noisy experimental image and its corresponding auxiliary feature image, and preprocess the noisy experimental image to obtain the target image;

[0124] The target image and the auxiliary feature image are input into the denoising system model. The multi-level feature extraction module extracts features from the target image and the corresponding auxiliary feature image to obtain a feature image. The feature fusion module then fuses the feature images to obtain a fused image. Finally, the Transformer module processes the fused image using an attention mechanism guided by auxiliary features to obtain a denoised image. Those skilled in the art will understand that all or part of the process of the above embodiments can be implemented by a computer program instructing related hardware, and the program can be stored in a computer-readable storage medium. The computer-readable storage medium can be a disk, optical disk, read-only memory, or random access memory, etc.

[0125] The image denoising method provided by the present invention has been described in detail above. Specific examples have been used to illustrate the principle and implementation of the present invention. The description of the above embodiments is only for the purpose of helping to understand the method and core idea of ​​the present invention. At the same time, for those skilled in the art, there will be changes in the specific implementation and application scope based on the idea of ​​the present invention. Therefore, the content of this specification should not be construed as a limitation of the present invention.

Claims

1. An image denoising method, characterized in that, The method comprises the following steps: obtaining a denoising system model, wherein the denoising system model comprises a multi-level feature extraction module, a feature fusion module and a Transformer module, the denoising system model comprises five layers of Transformer modules, the Transformer module comprises a channel attention unit, a spatial attention unit and a fusion auxiliary feature unit, the input of each layer of the Transformer module is connected with the feature fusion module, and the output of the previous layer of the Transformer module is connected with the fusion auxiliary feature unit in the next layer of the Transformer module; obtaining an experimental noisy image and a corresponding auxiliary feature image, and pre-processing the experimental noisy image to obtain a target image; inputting the target image and the auxiliary feature image into the denoising system model, performing feature extraction on the target image and the auxiliary feature image through the multi-level feature extraction module to obtain a feature image, performing fusion processing on the feature image through the feature fusion module to obtain a fusion image, and performing attention mechanism processing on the fusion image based on auxiliary features through the Transformer module to obtain a denoised image; wherein in the i-th layer of the Transformer module: the channel attention unit is configured to receive the fusion image, obtain channel weights based on the fusion image, and perform point multiplication calculation on the channel weights and the fusion image to obtain a channel attention result; the spatial attention unit is configured to obtain spatial position weights based on the channel attention result, and perform point multiplication calculation on the spatial position weights and the fusion image to obtain a fusion attention image; the fusion auxiliary feature unit is configured to perform attention mechanism processing on the fusion attention image and the denoised image output by the i-1-th layer of the Transformer module based on auxiliary features to obtain a denoised image output by the i-th layer of the Transformer module.

2. The image denoising method of claim 1, wherein, The pre-processing of the experimental noisy image to obtain a target image comprises: performing logarithmic transformation pre-processing on the experimental noisy image to obtain a target image.

3. The image denoising method according to claim 1, wherein the fusion auxiliary feature unit is configured to perform standardization processing on the fusion attention image and the denoised image output by the i-1-th layer of the Transformer module to obtain a processed image, perform global pooling dimension reduction processing based on the processed image to obtain a dimension reduction image, perform weighted fusion processing based on the dimension reduction image to obtain a weighted fusion image, and perform inverse standardization processing based on the weighted fusion image to obtain the denoised image output by the i-th layer of the Transformer module.

4. The image denoising method according to claim 1, wherein the channel attention unit is configured to perform calculation on the fusion image based on adaptive average pooling and two layers of linear transformation to obtain channel weights.

5. The image denoising method according to claim 1, wherein The spatial attention unit is configured to calculate pixel similarity and correlation based on the channel attention result, and obtain a spatial position weight.

6. The image denoising method of claim 1, wherein, The loss function of the denoising system model is: ； In the formula: is an asymmetric loss function, is a penalty function, n is the target image, f is the auxiliary feature image, and r is a reference image, is a loss function.

7. An image denoising apparatus characterized by comprising: The method comprises the following steps: The model acquisition unit is configured to acquire a denoising system model, wherein the denoising system model comprises a multi-level feature extraction module, a feature fusion module and a Transformer module, the denoising system model comprises five layers of Transformer modules, and each layer of the Transformer module comprises a channel attention unit, a spatial attention unit and a fusion auxiliary feature unit; the input of each layer of the Transformer module is connected with the feature fusion module, and the output of a previous layer of the Transformer module is connected with the fusion auxiliary feature unit in a next layer of the Transformer module. The image preprocessing unit is configured to acquire an experimental noisy image and a corresponding auxiliary feature image, and preprocess the experimental noisy image to obtain a target image. The image denoising unit is configured to input the target image and the auxiliary feature image into the denoising system model, extract features of the target image and the corresponding auxiliary feature image through the multi-level feature extraction module to obtain a feature image, perform fusion processing on the feature image through the feature fusion module to obtain a fusion image, and process the fusion image based on an attention mechanism guided by auxiliary features through the Transformer module to obtain a denoised image. In the i-th layer of the Transformer module: The channel attention unit is configured to receive the fusion image, obtain a channel weight based on the fusion image, and perform point multiplication calculation on the channel weight and the fusion image to obtain a channel attention result. The spatial attention unit is configured to obtain a spatial position weight based on the channel attention result, and perform point multiplication calculation on the spatial position weight and the fusion image to obtain a fusion attention image. The fusion auxiliary feature unit is configured to process the fusion attention image and a denoised image output by the i-1-th layer of the Transformer module based on an attention mechanism guided by auxiliary features to obtain a denoised image output by the i-th layer of the Transformer module.

8. An electronic device, comprising: The device comprises a memory and a processor, wherein The memory is configured to store a program. The processor is coupled with the memory and is configured to execute the program stored in the memory to implement the steps of the image denoising method according to any one of claims 1 to 6. 9.A non-transitory computer-readable storage medium having stored thereon a computer program, characterized in that, The computer program is executed by the processor to implement the image denoising method according to any one of claims 1 to 6.

Images