Image denoising method, storage medium and terminal device

Abstract

The present application discloses an image denoising method, a storage medium and a terminal device. The image denoising method determines several feature maps of an image to be denoised through an image denoising model, determines an attention map corresponding to the image to be denoised based on a target feature map in the several feature maps, determines an enhanced feature map corresponding to the image to be denoised based on the attention map, and finally determines a denoised image corresponding to the image to be denoised based on the enhanced feature map and the several feature maps. In this way, after obtaining the attention map carrying image detail information, the image denoising model enhances the detail information in the attention map to obtain an enhanced feature map, so that the denoised image determined according to the enhanced feature map can both remove the image noise carried by the image to be denoised and retain the image detail information, thereby improving the image quality of the denoised image.

Description

Technical Field

[0001] The present application relates to the field of image processing technology, and in particular to an image denoising method, a storage medium and a terminal device. Background Art

[0002] With the continuous development of smart terminals, smart terminals are generally equipped with night scene shooting functions, which can improve the image quality of images captured in night scenes. However, when the image acquisition device is affected by the external environment when shooting images, the captured images will have image noise. Especially when shooting images under low illumination conditions at night, the captured images are often degraded due to insufficient light, so that the captured images carry image noise and reduce image quality. Summary of the invention

[0003] The technical problem to be solved by the present application is to provide an image denoising method, a storage medium and a terminal device in view of the deficiencies of the prior art.

[0004] In order to solve the above technical problems, the first aspect of the embodiments of the present application provides an image denoising method, which is applied to an image denoising model. The method includes:

[0005] The image denoising model determines a number of feature maps corresponding to the image to be denoised;

[0006] The image denoising model determines an attention map corresponding to the image to be denoised based on a target feature map among the plurality of feature maps;

[0007] The image denoising module determines an enhanced feature map corresponding to the image to be denoised based on the attention map;

[0008] The image denoising model determines a denoised image corresponding to the image to be denoised based on the enhanced feature map, the plurality of feature maps and the image to be denoised.

[0009] In the image denoising method, the image sizes of the feature maps in the plurality of feature maps are different.

[0010] The image denoising method, wherein the target feature map is a feature map with the smallest image size among a plurality of feature maps.

[0011] The image denoising method, wherein the image denoising model includes a downsampling module, the downsampling module includes a plurality of cascaded downsampling units, the plurality of downsampling units and a plurality of feature Figure 1 One-to-one correspondence is used to determine the corresponding feature maps through each downsampling unit.

[0012] The image denoising method, wherein the image denoising model includes an attention module, the attention module includes a first attention unit and a second attention unit; the image denoising model determines the attention map corresponding to the image to be denoised based on the target feature map in the plurality of feature maps, specifically comprising:

[0013] The first attention unit determines a first attention map corresponding to the image to be denoised based on the target feature map;

[0014] The second attention unit determines a second attention map corresponding to the image to be denoised based on the first attention map, and determines an attention map corresponding to the image to be denoised based on the second attention map and the target feature map.

[0015] The image denoising method, wherein the first attention unit includes a first convolution unit, a first global average pooling layer and a first fusion unit; the first attention module determines the first attention map corresponding to the image to be denoised based on the target feature map, specifically comprising:

[0016] The first convolution unit determines a first feature map corresponding to the image to be denoised based on the target feature map;

[0017] The first global average pooling layer determines a third attention map corresponding to the image to be denoised based on the first feature map;

[0018] The first fusion unit determines a first attention map corresponding to the image to be denoised based on the third attention map and the first feature map.

[0019] The image denoising method, wherein the second attention unit includes a second global average pooling layer, a global maximum pooling layer, a second convolution unit and a second fusion unit;

[0020] The second global average pooling layer determines a fourth attention map corresponding to the image to be denoised based on the first attention map;

[0021] The global maximum pooling layer determines a fifth attention map corresponding to the image to be denoised based on the first attention map;

[0022] The second convolution unit determines a second feature map corresponding to the image to be denoised based on the fourth attention map and the fifth attention map;

[0023] The second fusion unit determines a second attention map corresponding to the image to be denoised based on the second feature map and the first attention map, and determines an attention map corresponding to the image to be denoised based on the second attention map and the target feature map.

[0024] The image denoising method, wherein the attention module includes several attention modules, the several attention modules are cascaded in sequence, and the input item of the attention module located at the front in the cascade order among the several attention modules is the target feature map, the output item of the attention module located at the last is the attention map corresponding to the image to be denoised, and the attention map output by the previous attention module in two adjacent attention models is the input item of the next attention module.

[0025] The image denoising method, wherein the image denoising model includes a plurality of cascaded enhancement modules and fusion modules, and the image denoising module determines the enhanced feature map corresponding to the image to be denoised based on the attention map, specifically including:

[0026] The enhancement module located at the front in the cascade order among the plurality of enhancement modules determines a first enhancement feature map corresponding to the image to be denoised based on the attention map, and the enhancement module located at the end determines a second enhancement feature map corresponding to the image to be denoised based on the attention map, wherein the output item of the previous enhancement module among two adjacent enhancement modules is the input item of the next enhancement module;

[0027] The image denoising model determines an enhanced feature map corresponding to the image to be denoised based on the first enhanced feature map and the second enhanced feature map.

[0028] The image denoising method, wherein the image denoising model includes an upsampling module, and the image denoising model determines the denoised image corresponding to the image to be denoised based on the enhanced feature map, the plurality of feature maps and the image to be denoised, specifically comprising:

[0029] The upsampling module determines an upsampling feature map corresponding to the image to be denoised based on the enhanced image and a plurality of feature maps;

[0030] The fusion module determines a denoised image corresponding to the image to be denoised based on the up-sampled feature map and the image to be denoised.

[0031] The image denoising method, wherein the generation process of the image denoising model comprises:

[0032] Acquire a training image set, wherein the training image set includes a plurality of training image groups, each training image group includes a plurality of training images, and the denoised images corresponding to the training images in each training image group are the same;

[0033] The preset network model is trained based on the training image set to obtain the image denoising model.

[0034] The image denoising method, wherein the step of obtaining a training image set specifically comprises:

[0035] Obtain several shooting scenes;

[0036] For each shooting scene, a plurality of candidate images corresponding to the shooting scene are obtained, wherein there are at least two candidate images among the plurality of candidate images, and the exposure times corresponding to the two candidate images are different;

[0037] Fusing the plurality of candidate images to obtain a fused image;

[0038] The plurality of candidate images are used as a training image group, and the fused image is used as a denoised image corresponding to each fused image in the training image group, so as to obtain a training image set.

[0039] The image denoising method, wherein the step of training a preset network model based on the training image set to obtain the image denoising model specifically includes:

[0040] Inputting a training image in a training image set into a preset network model, and outputting a predicted denoised image corresponding to the training image through the preset network model;

[0041] Determine a mean square error between the predicted denoised image and the denoised image corresponding to the training image, and use the mean square error as a loss value corresponding to the training image;

[0042] The preset network model is trained based on the loss value to obtain the image denoising model.

[0043] A second aspect of an embodiment of the present application provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in any of the image denoising methods described above.

[0044] A third aspect of an embodiment of the present application provides a terminal device, comprising: a processor, a memory, and a communication bus; the memory stores a computer-readable program that can be executed by the processor;

[0045] The communication bus realizes the connection and communication between the processor and the memory;

[0046] When the processor executes the computer-readable program, the steps in any of the above-mentioned image denoising methods are implemented.

[0047] Beneficial effect: Compared with the prior art, the present application provides an image denoising method, a storage medium and a terminal device. The image denoising method determines several feature maps of the image to be denoised through an image denoising model, determines the attention map corresponding to the image to be denoised based on the target feature map in the several feature maps, determines the enhanced feature map corresponding to the image to be denoised based on the attention map, and finally determines the denoised image corresponding to the image to be denoised based on the enhanced feature map and the several feature maps. The present application determines the attention map corresponding to the image to be denoised through an image denoising model, determines the enhanced feature map based on the attention map, and finally determines the denoised image based on the enhanced feature map and the several feature maps. In this way, after obtaining the attention map carrying the image detail information, the image denoising model enhances the detail information in the attention map to obtain the enhanced feature map, so that the denoised image determined according to the enhanced feature map can both remove the image noise carried by the image to be denoised and retain the image detail information, thereby improving the image quality of the denoised image. BRIEF DESCRIPTION OF THE DRAWINGS

[0048] In order to more clearly illustrate the technical solutions in the embodiments of the present application, the drawings required for use in the description of the embodiments will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without inventive work.

[0049] Figure 1 This is a flowchart of the image denoising method provided in this application.

[0050] Figure 2 This is a schematic diagram of the model structure of the image denoising model in the image denoising method provided in this application.

[0051] Figure 3 This is a schematic diagram of the model structure of the attention module in the image denoising model in the image denoising method provided in this application.

[0052] Figure 4 This is a schematic diagram of the model structure of the enhancement module in the image denoising model in the image denoising method provided in this application.

[0053] Figure 5 This is a schematic diagram of the structure of the terminal device provided in this application. DETAILED DESCRIPTION

[0054] The present application provides an image denoising method, a storage medium and a terminal device. To make the purpose, technical solution and effect of the present application clearer and more specific, the present application is further described in detail with reference to the accompanying drawings and examples. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0055] It will be understood by those skilled in the art that, unless expressly stated, the singular forms "one", "said", and "the" used herein may also include plural forms. It should be further understood that the term "comprising" used in the specification of the present application refers to the presence of the features, integers, steps, operations, elements and / or components, but does not exclude the presence or addition of one or more other features, integers, steps, operations, elements, components and / or groups thereof. It should be understood that when we refer to an element as being "connected" or "coupled" to another element, it may be directly connected or coupled to the other element, or there may be an intermediate element. In addition, the "connection" or "coupling" used herein may include wireless connection or wireless coupling. The term "and / or" used herein includes all or any unit and all combinations of one or more associated listed items.

[0056] It will be understood by those skilled in the art that, unless otherwise defined, all terms (including technical and scientific terms) used herein have the same meaning as those generally understood by those skilled in the art to which this application belongs. It should also be understood that terms such as those defined in general dictionaries should be understood to have meanings consistent with those in the context of the prior art, and will not be interpreted with idealized or overly formal meanings unless specifically defined as here.

[0057] In a specific implementation, the terminal device described in the embodiments of the present application includes, but is not limited to, other portable devices such as mobile phones with touch-sensitive surfaces (e.g., touch display screens and / or touch pads), laptop computers, or tablet computers. It should also be understood that in some embodiments, the device is not a portable communication device, but a desktop computer with a touch-sensitive surface (e.g., touch display screens and / or touch pads).

[0058] In the following discussion, a terminal device including a display and a touch-sensitive surface is described. However, it should be understood that the terminal device may also include one or more other physical user interface devices such as a physical keyboard, a mouse and / or a joystick.

[0059] The terminal device supports various applications, such as one or more of the following: drawing applications, presentation applications, word processing applications, video conferencing applications, disk burning applications, spreadsheet applications, game applications, telephone applications, video conferencing applications, email applications, instant messaging applications, exercise support applications, photo management applications, data camera applications, digital video camera applications, web browsing applications, digital music player applications and / or digital video playback applications, etc.

[0060] Various applications that can be executed on the terminal device can use at least one common physical user interface device such as a touch-sensitive surface. The first or multiple functions of the touch-sensitive surface and the corresponding information displayed on the terminal can be adjusted and / or changed between applications and / or within the corresponding application. In this way, the common physical framework of the terminal (e.g., the touch-sensitive surface) can support various applications with user interfaces that are intuitive and transparent to the user.

[0061] It should be understood that the sequence numbers and sizes of the steps in this embodiment do not mean the order of execution. The execution order of each process is determined by its function and internal logic, and should not constitute any limitation on the implementation process of the embodiment of the present application.

[0062] After research, the inventor found that in order to improve the night scene shooting effect of the terminal device, the terminal device is equipped with a night scene shooting function. However, when the image acquisition device shoots images, it will be affected by the external environment, so that the captured images will have image noise. Especially when shooting images under low illumination conditions at night, the captured images are often degraded due to insufficient light, so that the captured images carry image noise and reduce the image quality. Therefore, image denoising is an important part of the night scene shooting function. Through image denoising, a clear image is restored in the noisy image, and the image detail information in the image is retained.

[0063] The commonly used image denoising methods include BM3D based on filtering ideas, and data-driven deep network methods such as DnCnn, Ridnet, and CBDnet. Among them, the traditional denoising methods represented by BM3D mainly use three-dimensional block matching filtering. Although this method has good results, it takes a long time (for example, 3 seconds) to process an image (for example, a 4K picture) due to its high algorithm complexity, which cannot meet the real-time requirements of terminal devices.

[0064] In order to solve the above problems, in an embodiment of the present application, several feature maps of the image to be denoised are determined by an image denoising model, and based on the target feature map in the several feature maps, an attention map corresponding to the image to be denoised is determined, and based on the attention map, an enhanced feature map corresponding to the image to be denoised is determined, and finally, based on the enhanced feature map and the several feature maps, a denoised image corresponding to the image to be denoised is determined. The present application determines the attention map corresponding to the image to be denoised by an image denoising model, and determines the enhanced feature map based on the attention map, and finally determines the denoised image based on the enhanced feature map and the several feature maps. In this way, after obtaining the attention map carrying the image detail information, the image denoising model enhances the detail information in the attention map to obtain the enhanced feature map, so that the denoised image determined according to the enhanced feature map can both remove the image noise carried by the image to be denoised and retain the image detail information, thereby improving the image quality of the denoised image.

[0065] The application content is further explained below through the description of embodiments in conjunction with the accompanying drawings.

[0066] This implementation provides an image denoising method, such as Figure 1 As shown, the method includes:

[0067] S10. The image denoising model determines a plurality of feature maps of the image to be denoised, wherein the image size of each feature map in the plurality of feature maps is different.

[0068] Specifically, the image to be denoised may be an image taken by an image processing device loaded with the image denoising model (e.g., a smart phone, etc.), or an image taken by other external devices (e.g., a digital camera, etc.) and stored in the image processing device, or an image sent to the image processing device through the cloud. In this embodiment, the image to be denoised may be an image taken by an imaging system (e.g., a camera), wherein the imaging system may be configured by the image processing device itself or by other devices. For example, the image to be denoised is a landscape image taken by a mobile phone equipped with a camera; for another example, the image to be denoised is a portrait of a person taken by a digital camera and sent to the image processing device.

[0069] Furthermore, the image to be denoised is the original image data collected by a camera device, that is, a RAW image, wherein the original image data is image data that has not been processed and compressed. The camera device may be a CMOS (Complementary Metal-Oxide-Semiconductor) or CCD (Charge Coupled Device) image sensor, and the original image data may be the original data obtained by converting the captured light source signal into a digital signal by the camera device. In this way, the original image data is used as the image to be denoised, which can avoid the loss of image detail information generated during the processing and compression of the original data image, so that more image information can be retained in the image to be denoised, so that the denoised image obtained after denoising the image to be denoised can carry and retain more image information, thereby improving the image quality of the denoised image.

[0070] In addition, in practical applications, since the image to be denoised is the original image data, the image denoising model performs color channel separation on the image to be denoised after acquiring the image to be denoised, and uses the image obtained by color channel separation as the image to be denoised. The color channel separation refers to separating the image to be denoised by color channels according to the color order of the image to be denoised. For example, the image to be denoised is the original image data of H*W*1, and the color order of the image to be denoised is RGBG, where H represents the height of the first image, W represents the width of the first image, and 1 represents the color channel data of the first image. Then, after the image to be denoised is separated by color channels, an image A of H / 2*W / 2*4 can be generated, and image A is used as the image to be denoised, where H / 2 represents the height of image A, W / 2 represents the width of image A, and 4 represents the color channel data of image A. Here, the four color channels are respectively recorded as the first color channel 1, the second color channel 2, the third color channel 3 and the fourth color channel 4, wherein the first color channel stores R data, the second color channel stores G data, the third color channel stores B data, and the fourth color channel stores G data.

[0071] Further, the image denoising model is a trained network model, wherein the training process of the image denoising model can be performed in an image processing device (for example, a mobile phone equipped with a camera) that processes the image to be denoised; or it can be performed in other external devices. When the training process of the image denoising model is performed in other external devices, after the other devices complete the training of the image denoising model, the file corresponding to the image denoising model is transplanted to the image processing device, so that the image processing device is configured with the image denoising model. In addition, in an implementation of this embodiment, the image processing device can use the image denoising model as a functional module (recorded as an image denoising function), and the image denoising function is used to remove the noise carried by the image to obtain a denoised image. It can be understood that when the image processing device obtains the image to be denoised, the image processing device starts the image denoising function and uses the image to be denoised as the input item of the image denoising function to determine the denoised image of the image to be denoised through the image denoising model corresponding to the image denoising function. The denoised image is an output item of the image denoising model, and is used as the image after denoising corresponding to the image to be denoised.

[0072] Furthermore, the signal-to-noise ratio of the denoised image is greater than the signal-to-noise ratio of the image to be denoised, wherein the signal-to-noise ratio refers to the ratio of normal image information to noise information in the image, generally expressed in dB, and the higher the signal-to-noise ratio of the image, the less noise the image has. For example, for the image A to be denoised, the denoised image corresponding to the image A to be denoised is the denoised image A, and when the signal-to-noise ratio of the image A to be denoised is 50 dB, the signal-to-noise ratio of the denoised image A is greater than 50 dB, such as 60 dB.

[0073] Further, the plurality of feature maps are determined by the image denoising model based on the image to be denoised, and the image size of each feature map in the plurality of images is different. For example, the plurality of feature maps include feature map A, feature map B and feature map C, then the image size of feature map A, the image size of feature map B and the image size of feature map C are different from each other, for example, the image size of feature map A is 224*224, the image size of feature map B is 112*112, and the image size of feature map C is 56*56.

[0074] In one implementation of this embodiment, the image denoising model includes a downsampling module, the downsampling module includes a plurality of cascaded downsampling units, the plurality of downsampling units and a plurality of feature Figure 1One-to-one correspondence is used to determine the corresponding feature maps through each downsampling unit. It can be understood that the number of the downsampling units is the same as the number of the feature maps, and for each feature map in the feature maps, there is a downsampling unit in the downsampling units, and the output item of the downsampling unit is the feature map. Therefore, the image denoising model determines several feature maps of the image to be denoised, specifically including: the target downsampling unit determines the first feature map corresponding to the image to be denoised based on the image to be denoised, wherein the target downsampling unit is the downsampling unit located at the front in the cascade order; the target downsampling unit inputs the feature map into the candidate downsampling unit; the candidate downsampling unit determines the second feature map corresponding to the image to be denoised based on the first feature map, wherein the candidate downsampling unit is a downsampling unit located after the target downsampling unit and adjacent to the target downsampling unit in the cascade order; the candidate downsampling unit is used as the target downsampling unit, the second feature map is used as the first feature map, and the step of continuing to determine the second feature map corresponding to the image to be denoised based on the first feature map by the candidate downsampling unit is performed until the candidate downsampling unit is the downsampling unit located at the end in the cascade order, so as to obtain several feature maps corresponding to the image to be denoised, wherein the several feature maps include the first feature map and several second feature maps.

[0075] Furthermore, the several downsampling units are cascaded in sequence, and the output item of the previous downsampling unit in the two adjacent downsampling units in the cascade order is the input item of the next downsampling unit, and the output item of each downsampling unit is used as a feature map corresponding to the image to be denoised. Therefore, in the two downsampling units adjacent in the cascade order, the image size of the feature map output by the previous downsampling unit is larger than the image size of the feature map output by the next downsampling unit. In addition, the model structure of each sampling unit in the several downsampling units is the same, and one downsampling unit is used as an example for explanation. The downsampling unit includes a convolution layer and an activation function, the step size of the convolution layer is 2, and the activation function can be a ReLU activation function. The generalization ability of the model can be better enhanced by the ReLU activation function.

[0076] For example: Figure 2As shown, the downsampling module includes three downsampling units, which are respectively recorded as the first downsampling unit, the second downsampling unit and the third downsampling unit. The first downsampling unit, the second downsampling unit and the third downsampling unit all include a convolution layer with a step size of 2 and a ReLU activation function. Among them, the first downsampling unit is connected to the second downsampling unit; the second downsampling unit is connected to the third downsampling unit, the output item of the first downsampling unit is feature map A, the output item of the second downsampling unit is feature map B, and the output item of the third downsampling unit is feature map C, then feature map A, feature map B and feature map C are three feature maps corresponding to the image to be denoised, among which feature map A is the input item of the second downsampling unit, feature map B is the input item of the third downsampling unit, and feature map C is used for subsequent determination of the attention map.

[0077] Of course, in practical applications, the input item of the downsampling unit located at the front in the cascade order can be the image to be denoised, or it can be a feature image determined based on the image to be denoised. Figure 2 As shown, the input item of the downsampling unit located at the front in the cascade order is a feature image determined based on the image to be denoised. It can be understood that the image denoising model includes a convolution module, the input item of the convolution module is the image to be denoised, and the output item of the convolution module is a convolution feature map, which is the input item of the downsampling unit located at the front in the cascade order. Among them, the step size of the convolution layer in the convolution module is 1, so that the convolution feature map determined by the convolution module can retain the original image information of the image to be denoised.

[0078] S20. The image denoising model determines an attention map corresponding to the image to be denoised based on a target feature map among the several feature maps, wherein the target feature map is a feature map with the smallest image size among the several feature maps.

[0079] Specifically, the image encoding module is embedded with an attention mechanism, and the attention map is determined by the attention mechanism based on the image feature map. By utilizing an attention mechanism similar to that of the human eye, the noise features in the feature map are enhanced, so that the texture information, color information and image details in the feature map after processing based on the attention mechanism will be enhanced, which can facilitate the subsequent removal of noise features, reduce the residual image noise in the denoised image, and improve the image quality of the denoised image.

[0080] Further, in an implementation of this embodiment, the image denoising model includes an attention module, and the attention module includes a first attention unit and a second attention unit; the image denoising model determines the attention map corresponding to the image to be denoised based on the target feature map in the plurality of feature maps, specifically including:

[0081] The first attention unit determines a first attention map corresponding to the image to be denoised based on the target feature map;

[0082] The second attention unit determines a second attention map corresponding to the image to be denoised based on the first attention map, and determines an attention map corresponding to the image to be denoised based on the second attention map and the target feature map.

[0083] Specifically, the first attention unit and the second attention unit are cascaded, and the output item of the first attention unit is the input item of the second attention unit, wherein the first attention unit is embedded based on the image attention mechanism, and can focus on the image to be denoised from the image features in the image, and retain the texture information in the image to be denoised; the second attention unit is embedded based on the spatial attention mechanism, and can focus on the image to be denoised from the spatial direction, and retain the texture information and color information in the image to be denoised. Therefore, it can be seen from the attention image obtained through the second attention map and the target feature map that the attention map can learn the image details of the image to be denoised through the target feature map while preserving the color information and texture features of the image to be denoised, so that the attention image can preserve the texture information, color information and image details of the image to be denoised.

[0084] In one implementation of this embodiment, Figure 3 As shown, the first attention unit includes a first convolution unit, a first global average pooling layer and a first fusion unit; the first attention module determines the first attention map corresponding to the image to be denoised based on the target feature map, which specifically includes:

[0085] The first convolution unit determines a first feature map corresponding to the image to be denoised based on the target feature map;

[0086] The first global average pooling layer determines a third attention map corresponding to the image to be denoised based on the first feature map;

[0087] The first fusion unit determines a first attention map corresponding to the image to be denoised based on the third attention map and the first feature map.

[0088] Specifically, the first convolution unit is cascaded with the first global average pooling layer, the first global average pooling layer is cascaded with the first fusion unit, and the first convolution unit is connected with the first fusion unit. Thus, the input item of the first convolution unit is the target feature map, the first feature map output by the first convolution unit is respectively input into the first global average pooling layer and the first fusion unit, the third attention map output by the first global average pooling layer is input into the first fusion unit, and the third attention map output by the first global average pooling layer and the first feature map output by the first convolution unit are fused through the first fusion unit to obtain the first attention map.

[0089] The first convolution unit includes a first convolution layer, a first activation function layer and a second convolution layer, and the first convolution layer, the first activation function layer and the second convolution layer are cascaded in sequence, wherein the input item of the first convolution layer is the target feature map, and the output item of the second convolution layer is the first feature map. The first convolution layer is connected to the first activation function layer, and its own output item is input into the first activation function layer, and the output item of the first convolution layer is sparsed by the first activation function layer. In a specific implementation of this embodiment, the first activation function layer can be embedded with a ReLU function, and the pixel values ​​of the pixel points with negative pixel values ​​in the output items of the first convolution layer are converted to 0 by the ReLU function, so that the output items of the first convolution layer can be sparsed, so that the texture features can be better learned later.

[0090] The first global average pooling layer is used to select the most prominent feature of the texture information in the first feature map to learn the texture information of the image to be denoised. The input item of the first global average pooling layer is the first feature map of the first convolution unit, and the output item of the first global average pooling layer is the third attention map.

[0091] The first fusion unit is used to perform dot multiplication of the pixels in the first feature map and the pixels in the third attention map to obtain the first attention map, wherein the image scale of the first feature map is the same as the image scale of the third attention map. For example, if the image scale of the first feature map is 56*56*64, then the image scale of the third attention map is 56*56*64. In addition, a convolution unit A and an activation function layer A are arranged between the first global average pooling layer and the first fusion unit, wherein the convolution unit A includes a convolution layer A, an activation function layer B and a convolution layer B, wherein the convolution layer A, the activation function layer B and the convolution layer B are cascaded in sequence, wherein the input item of the convolution layer A is the output item of the first global pooling layer, and the output item of the convolution layer B is the input item of the first fusion unit. The convolution unit B is connected to the activation function layer A, and its own output item is input to the activation function layer A, and the pixel value of each pixel in the output item of the convolution unit B is converted to between 0-1 (including 0 and 1) through the activation function layer A. In a specific implementation of this embodiment, the activation function layer A can be embedded with a Sigmoid function, and the pixel value of each pixel point in the output item of the convolution unit B is converted to between 0 and 1 through the Sigmoid function; the activation function layer B can be embedded with a ReLU function, and the pixel value of the pixel point with a negative pixel value in the output item of the convolution layer A is converted to 0 through the ReLU function, so that the output item of the convolution layer A can be sparse, so that the texture features can be better learned later.

[0092] Further, in one implementation of this embodiment, Figure 3 As shown, the second attention unit includes a second global average pooling layer, a global maximum pooling layer, a second convolution unit and a second fusion unit;

[0093] The second global average pooling layer determines a fourth attention map corresponding to the image to be denoised based on the first attention map;

[0094] The global maximum pooling layer determines a fifth attention map corresponding to the image to be denoised based on the first attention map;

[0095] The second convolution unit determines a second feature map corresponding to the image to be denoised based on the fourth attention map and the fifth attention map;

[0096] The second fusion unit determines a second attention map corresponding to the image to be denoised based on the second feature map and the first attention map, and determines an attention map corresponding to the image to be denoised based on the second attention map and the first attention map.

[0097] Specifically, the input items of the second global average pooling layer and the global maximum pooling layer are both the first attention map, and the second global average pooling layer and the global maximum pooling layer are both connected to the second convolution unit, and the second convolution unit is connected to the second fusion unit. Among them, the first global average pooling layer is used to select the most prominent features about the texture information in the first feature map to learn the texture information of the image to be denoised, and the global maximum pooling layer is used to pay attention to the overall information of the image to be denoised, and is sensitive to the background information of the image to be denoised, so as to select the color information of the image to be denoised. In this way, the texture information and color information of the image to be denoised can be learned through the second global average pooling layer and the global maximum pooling layer, which can improve the denoising effect of the image.

[0098] The second convolution unit includes a splicing layer, a convolution layer, and a second activation function layer, the splicing layer, the convolution layer, and the second activation function layer are cascaded in sequence, and the splicing layer is connected to the second global average pooling layer and the global maximum pooling layer respectively, the second activation function layer is connected to the second fusion unit, the input items of the splicing layer are the fourth attention map input by the second global average pooling layer and the fifth attention map input by the global maximum pooling layer, and the splicing layer splices the fourth attention map and the fifth attention map according to the channel, wherein the image size of the fourth attention map is the same as the image size of the fifth attention map. In addition, the second activation function layer can be embedded with a Sigmoid function, and the pixel value of each pixel in the output item of the convolution layer is converted to between 0 and 1 through the Sigmoid function.

[0099] The second fusion unit includes a first fusion layer and a second fusion layer, the input item of the first fusion layer is the first attention map and the second feature map output by the second convolution unit, and the input item of the second fusion layer is the second feature map and the target feature map. The first fusion layer is used to perform dot multiplication on the pixel points of the first attention map and the pixel points of the second feature map to fuse the first attention map and the second feature map, and the second fusion layer is used to add the second feature map and the target feature map to obtain the attention map.

[0100] Further, in an implementation of the present embodiment, in order to increase the texture information and color information in the attention map, the attention module includes several attention modules with the same model structure, and the several attention modules are cascaded in sequence, and the input item of the attention module located at the front in the cascade order among the several modules is the target feature map, and the output item of the attention module located at the end is the attention map corresponding to the image to be denoised, and the attention map output by the previous attention module in the two adjacent attention models is the input item of the next attention module. For example, the several attention modules include attention module A, attention module B and attention module C, and the attention modules A, attention module B and attention module C are cascaded in sequence, the input item of attention module A is the target feature map, the input item of attention module B is the feature map output by attention module A, the input item of attention module C is the output item of attention module B, and the output item of attention module C is the attention map corresponding to the image to be denoised. It can be understood that the attention map output by attention module A is the target feature map corresponding to attention module B, and the attention map output by attention module B is the target feature map corresponding to attention module C. In addition, in a specific implementation of the present embodiment, the several attention modules include 6 attention modules.

[0101] S30: The image denoising module determines an enhanced feature map corresponding to the image to be denoised based on the attention map.

[0102] Specifically, the enhanced feature map is determined based on the attention map, and the texture information and color information of the image to be denoised are stored in the enhanced feature map, while the image details of the image to be denoised in the enhanced feature map are enhanced. In this way, in the process of upsampling the candidate, the image details of the image to be denoised can be effectively learned to improve the image denoising effect. In one implementation of this embodiment, the enhanced feature map can be obtained through an enhancement module. Accordingly, the image denoising module includes a number of cascaded enhancement modules, and the input item of the enhancement module located at the front in the cascade order is the attention map, and the output item of the enhancement module is the enhanced feature map. The image details in the attention map are enhanced by the enhancement module to obtain an enhanced feature map.

[0103] Based on this, the image denoising module determines the enhanced feature map corresponding to the image to be denoised based on the attention map, specifically including:

[0104] The enhancement module located at the front in the cascade order among the plurality of enhancement modules determines a first enhancement feature map corresponding to the image to be denoised based on the attention map, and the enhancement module located at the end determines a second enhancement feature map corresponding to the image to be denoised based on the attention map, wherein the output item of the previous enhancement module among two adjacent enhancement modules is the input item of the next enhancement module;

[0105] The image denoising model determines an enhanced feature map corresponding to the image to be denoised based on the first enhanced feature map and the second enhanced feature map.

[0106] Specifically, the several enhancement modules are cascaded in sequence, and the output item of the previous enhancement module is the input item of the next enhancement module for two adjacent enhancement modules in the cascade order; the input item of the enhancement module located at the front is the attention map, and the second enhancement feature map output by the last enhancement module and the first enhancement feature map output by the front enhancement module are used to determine the enhancement feature map. Therefore, the image denoising model includes a fusion module, and the input items of the fusion module are the first enhancement feature map and the second enhancement feature map. The fusion module is used to splice the first enhancement feature map and the second enhancement feature map according to the channels, and use the spliced ​​map as the enhancement feature map, so that the enhancement feature map includes the first enhancement feature map output by the front enhancement module and the second enhancement feature map output by the last enhancement module, and more image detail information can be included while the image details in the enhancement feature map are enhanced.

[0107] For example, several enhancement modules include enhancement module A, enhancement module B and enhancement module C. Enhancement module A, enhancement module B and enhancement module C are cascaded in sequence, the input item of enhancement module A is the attention map, the input item of enhancement module B is the output item of enhancement module A, the input item of enhancement module C is the output item of enhancement module B, and the fusion image of the output item of enhancement module C and the output item of enhancement module A is the enhanced feature map corresponding to the image to be denoised.

[0108] In one implementation of this embodiment, Figure 4 As shown, the enhancement module includes several cascaded enhancement units and convolution units. For two adjacent enhancement units in the cascade order, the fusion graph of the input item of the previous enhancement unit and the output item of the previous enhancement unit is the input item of the next enhancement unit; the input item of the enhancement unit located at the front is the input item of the enhancement module, the output item and input item of the enhancement unit located at the end are the input items of the convolution unit, and the fusion graph of the output item of the convolution unit and the input item of the front enhancement unit is the output item of the enhancement module. In this way, by entering the input item and output item of the enhancement unit into a short connection, the image details in the output item can be increased, thereby improving the image details in the enhanced feature map output by the enhancement module. Based on this, after the enhanced feature map is output by the enhancement module, the enhanced feature map is fused with the attention map, so that the image detail information, texture information and color information are enhanced, and the loss of image details in the downsampling process can be reduced. In a specific implementation of this embodiment, the enhancement unit includes two cascaded convolution units, each of which includes a convolution layer and an activation function layer, and the convolution layer is connected to the activation function layer.

[0109] S40: The image denoising model determines a denoised image corresponding to the image to be denoised based on the enhanced feature map, the plurality of feature maps and the image to be denoised.

[0110] Specifically, the denoised image is an output image obtained by removing the noise of the image to be denoised by the image denoising model, wherein the denoised image is an output item of the image denoising model, and the signal-to-noise ratio of the denoised image is higher than the signal-to-noise ratio of the image to be denoised. For example, the signal-to-noise ratio of the image to be denoised is 50 dB, and the signal-to-noise ratio of the denoised image is 60 dB.

[0111] In one implementation of this embodiment, after removing the attention module and the enhancement module, the image denoising model can be a codec structure of UNET, and the image denoising model can include the downsampling module in the upsampling module image denoising model as the encoding structure, the upsampling module as the decoding structure, and the downsampling module corresponds to the upsampling module, wherein the downsampling module corresponds to the upsampling module means that the number of upsampling units included in the upsampling module is the same as the number of downsampling units included in the downsampling module, and the upsampling units correspond one-to-one to the downsampling units.

[0112] Based on this, the image denoising model determines the denoised image corresponding to the image to be denoised based on the enhanced feature map, the plurality of feature maps and the image to be denoised, specifically including:

[0113] The upsampling module determines an upsampling feature map corresponding to the image to be denoised based on the enhanced image and a plurality of feature maps;

[0114] The image denoising model determines a denoised image corresponding to the image to be denoised based on the upsampled feature map and the image to be denoised.

[0115] Specifically, the upsampling module includes a plurality of cascaded upsampling units, and the plurality of upsampling modules and the plurality of feature Figure 1One-to-one correspondence, for two upsampling units adjacent to each other in the cascade order, the output item of the previous upsampling unit and the feature map corresponding to the next upsampling unit are the input items of the next upsampling unit; the input item of the upsampling unit located at the front is the enhanced feature map and its corresponding feature map, and the output item of the upsampling unit located at the end and the image to be denoised determine the denoised image corresponding to the image to be denoised. In addition, the upsampling unit is a pixel reorganization (pixel shuffle), wherein the pixel shuffle is used to convert a H×W low-resolution input image (LowResolution) into a high-resolution image (High Resolution) of rH×rW through a Sub-pixel operation, wherein r is the upsampling step size, and the Sub-pixel operation is specifically for a low-resolution input image A with an image scale of H×W×1, and a convolution operation is performed on the low-resolution input image A to obtain an image with an image scale of H×W×r 2 The feature map of the feature map is 2 The channel maps of each channel are spliced ​​to obtain a high-resolution image with an image scale of rH×rW×1.

[0116] In order to further illustrate the image denoising model in this embodiment, the image denoising model is described below using a specific embodiment. Figure 2 The image denoising model includes a first convolution unit 100, a downsampling module 200, six cascaded attention modules 300, six cascaded enhancement modules 400, a fusion module 500, a second convolution unit 600, an upsampling module 700 and a third convolution unit 800, wherein the first convolution unit 100 is connected to the downsampling module 200, the downsampling module 200 is connected to the attention model 300 located at the front in the cascade order, and the attention module 300 located at the end in the cascade order is connected to the attention module 300 located at the back in the cascade order. The enhancement module 400 located at the front is connected, the enhancement module 400 located at the front in the cascade order and the enhancement module 400 located at the end in the cascade order are both connected to the fusion module 500, the fusion module 500 is connected to the second convolution unit 600, the second convolution unit 600 is connected to the upsampling module 700, the upsampling module 700 is connected to the third convolution unit 800, and the output item of the third convolution unit 800 is added to the input item of the first convolution unit 100 to obtain a denoised image.

[0117] Furthermore, the first convolution unit includes a convolution layer and an activation function, wherein the step size of the convolution layer is 1, and the activation function is a ReLU function. The downsampling module includes three downsampling units, and the model structures of the three downsampling units are the same, and all include a convolution layer and an activation function, wherein the step size of the convolution layer is 2, and the activation function is a ReLU function; correspondingly, the upsampling module includes three upsampling units, and the model structures of the first two upsampling units in the cascade order are the same, and all include a pixelshuffle unit and a convolution layer, wherein the sampling coefficient of the pixelshuffle unit is 2, and the last upsampling unit in the cascade order includes a pixelshuffle unit, wherein the sampling coefficient of the pixelshuffle unit is 2.

[0118] The second convolution unit includes a first convolution block and a second convolution block, the fusion module is connected to the first convolution block, the first convolution block is connected to the second convolution block, and the output item of the second convolution block is added to the output item of the second convolution block as the input item of the upsampling module. The third convolution unit includes a convolution layer and an activation function, wherein the step size of the convolution layer is 1, and the activation function is a ReLU function. In addition, the model structure of the attention module is the same as the model structure of the attention module in the above step S20, and the model structure of the enhancement module is the same as the model structure of the enhancement module in the above step S30, which will not be repeated here, and the specific description of the parameters of step S20 and step S30 can be referred to.

[0119] Further, in an implementation of this embodiment, after determining the model structure of the image denoising model, the training process of the image denoising model may include:

[0120] Get a training image set;

[0121] The preset network model is trained based on the training image set to obtain the image denoising model.

[0122] Specifically, the training image set includes several training image groups, each training image group includes several training images, and the denoised images corresponding to the training images in each training image group are the same. It can be understood that the training image set includes several training image groups, and any two training images in the several training images in each training image group are recorded as the first image and the second image, the image size of the first image is equal to the image size of the second image, and the first image and the second image correspond to the same image scene, wherein the first image and the second image correspond to the same image scene means that the similarity between the image content carried by the first image and the image content carried by the second image reaches a preset threshold, and the image size of the first image is the same as the image size of the second image, so that when the first image and the second image overlap, the coverage rate of the object carried by the first image to the object corresponding to it in the second image reaches a preset condition. Wherein, the preset threshold can be 99%, and the preset condition can be 99.5%, etc.

[0123] In an implementation of this embodiment, the acquiring of the training image set specifically includes:

[0124] Obtain several shooting scenes;

[0125] For each shooting scene, a plurality of candidate images corresponding to the shooting scene are obtained, wherein there are at least two candidate images among the plurality of candidate images, and the exposure times corresponding to the two candidate images are different;

[0126] Fusing the plurality of candidate images to obtain a fused image;

[0127] The plurality of candidate images are used as a training image group, and the fused image is used as a denoised image corresponding to each fused image in the training image group, so as to obtain a training image set.

[0128] Specifically, any two of the plurality of shooting scenes are different, and for each shooting scene, the image sizes of any two of the plurality of candidate images obtained are equal, and the two candidate images correspond to the same image scene, and there are at least two candidate images among the plurality of candidate images, and the exposure times corresponding to the two candidate images are different. The exposure time refers to the time interval from the opening to the closing of the shutter, and the aperture blades of the camera lens can leave the influence of the object on the film within the time interval, and when the exposure time of the camera is long, more light enters the aperture, so that the noise carried by the image can be reduced, so that among the two candidate images with different exposure times, the signal-to-noise ratio of the candidate image with the long exposure time is higher than that of the candidate image with the short exposure time. For example, each of the several training image groups includes 150 training images, and the exposure time corresponding to each training image in the 150 training images is different. For example, the 150 training images are sorted according to the exposure time to obtain a training image sequence. Among two adjacent training images in the training image sequence, the exposure time of the latter training image is 0.1 ms longer than the exposure time of the former training image, and the exposure time of the first training image is 0.1 ms, then the exposure time of the second training image is 0.2 ms, the exposure time of the third training image is 0.3 ms, the exposure time of the fourth training image is 0.4 ms, ..., the exposure time of the 150th training image is 15 ms.

[0129] In addition, since there are at least two candidate images among the candidate images, and the exposure times corresponding to the two candidate images are different, it can be known that there are at least two candidate images among the candidate images that carry different image noise. Therefore, after acquiring the candidate images, the candidate images are fused, and the fused image obtained by fusion is used as the denoised image corresponding to each candidate image among the candidate images, wherein the fusion of the candidate images can be based on the candidate image that is in the first position according to the shooting order, and the other candidate images are sequentially fused to the basic image according to the shooting order to obtain the fused image. In addition, in an implementation of the present embodiment, the candidate images are a plurality of continuous images, and the continuous images can be continuous images obtained by shooting when the image acquisition device is in a continuous shooting state, or continuous images obtained by shooting by continuously triggering the image acquisition device, etc.

[0130] Further, in an implementation of this embodiment, after obtaining the training sample image set, the preset network model can be trained by the training image set to obtain the image denoising model. Accordingly, the training of the preset network model based on the training image set to obtain the image denoising model specifically includes:

[0131] Inputting a training image in a training image set into a preset network model, and outputting a predicted denoised image corresponding to the training image through the preset network model;

[0132] Determine a mean square error between the predicted denoised image and the denoised image corresponding to the training image, and use the mean square error as a loss value corresponding to the training image;

[0133] The preset network model is trained based on the loss value to obtain the image denoising model.

[0134] Specifically, the model structure of the preset network model is the same as the model structure of the image denoising model, and the difference is that the model parameters of the preset network model are different from the model parameters of the image denoising model. The model parameters of the preset network model are initial model parameters, and the model parameters of the image denoising model are trained model parameters. For example, the preset network model includes a downsampling module, several attention modules, several enhancement modules and an upsampling module. The downsampling module is used to extract several feature maps corresponding to the training image, the several attention modules are used for the attention maps corresponding to the training image, and the several enhancement modules are used for the enhanced feature maps corresponding to the training image; the upsampling module is used to determine the denoised image corresponding to the image to be denoised. Among them, the downsampling module, several attention modules, several enhancement modules and upsampling module in the preset network model can refer to the description of the model structure of the downsampling module, several attention modules, several enhancement modules and upsampling module in the image denoising model, and will not be repeated here.

[0135] Furthermore, the loss value is the mean square error of the predicted denoised image and the denoised image corresponding to the training image. It is understandable that when the preset network model is trained, after the training image is input into the preset network model, the preset network model determines the predicted denoised image corresponding to the training image, and determines the loss value based on the predicted denoised image and the denoised image corresponding to the training image. The calculation formula of the loss value can be:

[0136]

[0137] Where N is the number of training images in each training image group, observed t is the denoised image, predicted t To predict the denoised image.

[0138] Further, in an implementation of this embodiment, the preset network model is trained based on the loss value by using the Adam method to optimize the model parameters of the preset network model, and the initial parameters of the prediction network model can be initialized using the Xavier method, with an initial learning rate of 10 -4,The learning rate adopts polynomial decay method, the number of iterations is 6000 epochs, and the batch size is 1.

[0139] Further, training the preset network model refers to training the model parameters of the preset network model, and obtaining the image denoising model refers to the training of the preset network model meeting the preset conditions. Among them, the preset conditions include the preset conditions including the loss value meeting the preset requirements or the number of training times reaching the preset number of times. The preset requirements can be determined according to the accuracy of the image denoising model, which is not described in detail here. The preset number of times can be the maximum number of training times of the preset network model, for example, 6000 times. Therefore, after the loss value is calculated, it is determined whether the loss value meets the preset requirements; if the loss value meets the preset requirements, the training is terminated; if the loss value does not meet the preset requirements, it is determined whether the number of training times of the preset network model reaches the prediction number, and if it does not reach the preset number, the network parameters of the preset network model are corrected according to the loss value; if it reaches the preset number of times, the training is terminated. In this way, judging whether the training of the preset network model is terminated by the loss value and the number of training times can avoid the training of the preset network model entering an infinite loop due to the failure of the loss value to meet the preset requirements.

[0140] Furthermore, since the network parameters of the preset network model are modified when the training situation of the preset network model does not meet the preset conditions (i.e., the loss value does not meet the preset requirements and the number of training times does not reach the preset number of times), after the network parameters of the preset network model are corrected according to the loss value, it is necessary to continue to train the preset network model, that is, based on the preset preset network model, each training image in the training image group is input into the preset network model, and the predicted image corresponding to each training image is output through the preset network model. Wherein, when continuing to input each training image in the training image group into the preset network model, each training image input into the preset network model is a training image that has not been input into the preset network model as an input item. For example, all training image groups in the training image set have a unique batch identifier (e.g., a batch number), and the batch identifier of the training image group input for the first training is different from the batch identifier of the training image group input for the second training, such as the image number of the training image group input for the first training is 1, the image number of the training image group input for the second training is 2, and the image number of the training image group input for the Nth training is N. Of course, in practical applications, since the number of training image groups in the training image set is limited, in order to improve the training effect of the evaluation network model, the training image groups in the training image set can be sequentially input into the preset network model to train the preset network model. After all the training image groups in the training image set have been input into the preset network model, the operation of sequentially inputting the training image groups in the training image set into the preset network model can be continued, so that the training image groups in the training image set are input into the preset network model in a cycle. In addition, while continuing to sequentially input the training image groups in the training image set into the preset network model, the input order of each training image group in the training image set can be adjusted to obtain an adjusted training image set, and the training image groups in the adjusted training image set are input into the preset network model.

[0141] In summary, the present embodiment provides an image denoising method, which determines several feature maps of the image to be denoised through an image denoising model, determines the attention map corresponding to the image to be denoised based on the target feature map in the several feature maps, determines the enhanced feature map corresponding to the image to be denoised based on the attention map, and finally determines the denoised image corresponding to the image to be denoised based on the enhanced feature map and the several feature maps. The present application determines the attention map corresponding to the image to be denoised through an image denoising model, determines the enhanced feature map based on the attention map, and finally determines the denoised image based on the enhanced feature map and the several feature maps. In this way, after obtaining the attention map carrying the image detail information, the image denoising model enhances the detail information in the attention map to obtain the enhanced feature map, so that the denoised image determined according to the enhanced feature map can both remove the image noise carried by the image to be denoised and retain the image detail information, thereby improving the image quality of the denoised image.

[0142] Based on the above-mentioned image denoising method, this embodiment provides a computer-readable storage medium, which stores one or more programs. The one or more programs can be executed by one or more processors to implement the steps in the image denoising method as described in the above-mentioned embodiment.

[0143] Based on the above image denoising method, the present application also provides a terminal device, such as Figure 5 As shown, it includes at least one processor (processor) 20; display screen 21; and memory (memory) 22, and may also include a communications interface (Communications Interface) 23 and a bus 24. Among them, the processor 20, the display screen 21, the memory 22 and the communication interface 23 can communicate with each other through the bus 24. The display screen 21 is configured to display a preset user guide interface in the initial setting mode. The communication interface 23 can transmit information. The processor 20 can call the logic instructions in the memory 22 to execute the method in the above embodiment.

[0144] In addition, the logic instructions in the memory 22 can be implemented in the form of software functional units and can be stored in a computer-readable storage medium when sold or used as an independent product.

[0145] The memory 22 is a computer-readable storage medium that can be configured to store software programs, computer executable programs, such as program instructions or modules corresponding to the methods in the embodiments of the present disclosure. The processor 20 executes functional applications and data processing by running the software programs, instructions or modules stored in the memory 22, that is, implementing the methods in the above embodiments.

[0146] The memory 22 may include a program storage area and a data storage area, wherein the program storage area may store an operating system and at least one application required for a function; the data storage area may store data created according to the use of the terminal device, etc. In addition, the memory 22 may include a high-speed random access memory and may also include a non-volatile memory. For example, a variety of media that can store program codes, such as a USB flash drive, a mobile hard disk, a read-only memory (ROM), a random access memory (RAM), a disk or an optical disk, may also be a transient storage medium.

[0147] In addition, the specific process of loading and executing the multiple instruction processors in the above-mentioned storage medium and the terminal device has been described in detail in the above-mentioned method, and will not be described one by one here.

[0148] Finally, it should be noted that the above embodiments are only used to illustrate the technical solutions of the present application, rather than to limit it. Although the present application has been described in detail with reference to the aforementioned embodiments, those skilled in the art should understand that they can still modify the technical solutions described in the aforementioned embodiments, or make equivalent replacements for some of the technical features therein. However, these modifications or replacements do not deviate the essence of the corresponding technical solutions from the spirit and scope of the technical solutions of the embodiments of the present application.

Claims

1. An image denoising method, characterized in that: Applied to an image denoising model, the method comprises: The image denoising model determines a number of feature maps corresponding to the image to be denoised; The image denoising model determines an attention map corresponding to the image to be denoised based on a target feature map among the plurality of feature maps; The image denoising module determines an enhanced feature map corresponding to the image to be denoised based on the attention map; The image denoising model determines a denoised image corresponding to the image to be denoised based on the enhanced feature map, the plurality of feature maps and the image to be denoised; The image denoising model includes an attention module, and the attention module includes a first attention unit and a second attention unit; the image denoising model determines the attention map corresponding to the image to be denoised based on the target feature map in the plurality of feature maps, specifically comprising: The first attention unit determines a first attention map corresponding to the image to be denoised based on the target feature map, wherein the first attention map includes texture information in the image to be denoised; The second attention unit determines a second attention map corresponding to the image to be denoised based on the first attention map, and determines an attention map corresponding to the image to be denoised based on the second attention map and the target feature map, wherein the second attention map includes texture information and color information in the image to be denoised; The attention module includes a plurality of attention modules, which are cascaded in sequence, and the input item of the attention module located at the front in the cascade order among the plurality of attention modules is the target feature map, the output item of the attention module located at the last is the attention map corresponding to the image to be denoised, and the attention map output by the previous attention module in two adjacent attention models is the input item of the next attention module; The first attention unit and the second attention unit are cascaded, and the output item of the first attention unit is the input item of the second attention unit.

2. The image denoising method according to claim 1, characterized in that: The image sizes of the feature maps are different.

3. The image denoising method according to claim 1, characterized in that: The target feature map is a feature map with the smallest image size among the plurality of feature maps.

4. The image denoising method according to claim 1, characterized in that: The image denoising model includes a downsampling module, which includes a plurality of cascaded downsampling units. The plurality of downsampling units correspond one-to-one to a plurality of feature maps, so that the corresponding feature maps are determined by each downsampling unit.

5. The image denoising method according to claim 1, characterized in that: The first attention unit includes a first convolution unit, a first global average pooling layer and a first fusion unit; The first attention unit determines, based on the target feature map, a first attention map corresponding to the image to be denoised, specifically comprising: The first convolution unit determines a first feature map corresponding to the image to be denoised based on the target feature map; The first global average pooling layer determines a third attention map corresponding to the image to be denoised based on the first feature map; The first fusion unit determines a first attention map corresponding to the image to be denoised based on the third attention map and the first feature map.

6. The image denoising method according to claim 1, characterized in that: The second attention unit includes a second global average pooling layer, a global maximum pooling layer, a second convolution unit and a second fusion unit; The second global average pooling layer determines a fourth attention map corresponding to the image to be denoised based on the first attention map; The global maximum pooling layer determines a fifth attention map corresponding to the image to be denoised based on the first attention map; The second convolution unit determines a second feature map corresponding to the image to be denoised based on the fourth attention map and the fifth attention map; The second fusion unit determines a second attention map corresponding to the image to be denoised based on the second feature map and the first attention map, and determines an attention map corresponding to the image to be denoised based on the second attention map and the target feature map.

7. The image denoising method according to claim 1, characterized in that: The image denoising model includes a plurality of cascaded enhancement modules, and the image denoising module determines, based on the attention map, an enhanced feature map corresponding to the image to be denoised, specifically including: The enhancement module located at the front in the cascade order among the plurality of cascaded enhancement modules determines a first enhancement feature map corresponding to the image to be denoised based on the attention map, and the enhancement module located at the last determines a second enhancement feature map corresponding to the image to be denoised based on the attention map, wherein the output item of the previous enhancement module among two adjacent enhancement modules is the input item of the next enhancement module; The image denoising model determines an enhanced feature map corresponding to the image to be denoised based on the first enhanced feature map and the second enhanced feature map.

8. The image denoising method according to claim 1, characterized in that: The image denoising model includes an upsampling module. The image denoising model determines the denoised image corresponding to the image to be denoised based on the enhanced feature map, the plurality of feature maps and the image to be denoised, and specifically includes: The upsampling module determines an upsampling feature map corresponding to the image to be denoised based on the enhanced image and a plurality of feature maps; The fusion module determines a denoised image corresponding to the image to be denoised based on the upsampled feature map and the image to be denoised.

9. The image denoising method according to any one of claims 1 to 8, characterized in that: The generation process of the image denoising model includes: Acquire a training image set, wherein the training image set includes a plurality of training image groups, each training image group includes a plurality of training images, and the denoised images corresponding to the training images in each training image group are the same; The preset network model is trained based on the training image set to obtain the image denoising model.

10. The image denoising method according to claim 9, characterized in that: The obtaining of the training image set specifically comprises: Obtain several shooting scenes; For each shooting scene, a number of candidate images corresponding to the shooting scene are obtained, wherein at least two candidate images exist among the candidate images. images, the exposure times corresponding to the two candidate images are different; Fusing the plurality of candidate images to obtain a fused image; The plurality of candidate images are used as a training image group, and the fused image is used as a denoised image corresponding to each fused image in the training image group, so as to obtain a training image set.

11. The image denoising method according to claim 9, characterized in that: The step of training a preset network model based on the training image set to obtain the image denoising model specifically includes: Inputting a training image in a training image set into a preset network model, and outputting a predicted denoised image corresponding to the training image through the preset network model; Determine a mean square error between the predicted denoised image and the denoised image corresponding to the training image, and use the mean square error as a loss value corresponding to the training image; The preset network model is trained based on the loss value to obtain the image denoising model.

12. A computer-readable storage medium, characterized in that: The computer-readable storage medium stores one or more programs, and the one or more programs can be executed by one or more processors to implement the steps in the image denoising method according to any one of claims 1 to 11.

13. A terminal device, characterized in that: include: A processor, a memory and a communication bus; the memory stores a computer-readable program that can be executed by the processor; The communication bus realizes the connection and communication between the processor and the memory; When the processor executes the computer-readable program, the steps in the image denoising method according to any one of claims 1 to 11 are implemented.

Images