Image processing method, device, electronic device and computer readable storage medium

By performing convolution processing and fusion noise reduction processing on the image, the problems of unclear image and high computing resource consumption caused by traditional tone mapping algorithms are solved, efficient image processing is achieved, and high-definition target images are obtained.

CN113674169BActive Publication Date: 2024-11-15GUANGDONG OPPO MOBILE TELECOMMUNICATIONS CORP LTD
View PDF 0 Cites 0 Cited by

Patent Information

Application Number
CN202110924588.X
Authority / Receiving Office
CN · China
Patent Type
Patents(China)
Current Assignee / Owner
Filing Date
2021-08-12
Publication Date
2024-11-15
Estimated Expiration
2041-08-12

AI Technical Summary

Technical Problem

Traditional tone mapping algorithms can easily lead to unclear images and consume a lot of computing resources.

Method used

By convolution processing of the image to be processed, the tone template image and the noise template image are separated, and the fusion process and noise reduction process are performed to obtain a high-definition target image and reduce computing resource consumption.

Benefits of technology

Improves the clarity of the image, while saving computing resources, and retains the detailed characteristics of the image.

✦ Generated by Eureka AI based on patent content.

Smart Images

  • Figure CN113674169B_ABST
    Figure CN113674169B_ABST
Patent Text Reader

Abstract

The present application relates to an image processing method, device, electronic device and computer-readable storage medium, comprising: performing convolution processing on an image to be processed to obtain a tone template image and a noise template image respectively; performing fusion processing on the tone template image and the image to be processed to obtain a tone mapping image; performing denoising processing on the tone mapping image based on the noise template image to obtain a first target image. The method can improve the clarity of the image and reduce the amount of calculation.
Need to check novelty before this filing date? Find Prior Art

Description

Technical Field

[0001] The present application relates to the field of imaging technology, and in particular to an image processing method, device, electronic device and computer-readable storage medium. Background Art

[0002] With the development of imaging technology, tone mapping algorithms have emerged. The tone mapping algorithm can adjust the grayscale of the image by mapping the image color, making the processed image look more comfortable. However, traditional tone mapping can easily lead to unclear images. Summary of the invention

[0003] The embodiments of the present application provide an image processing method, device, electronic device, and computer-readable storage medium, which can improve the clarity of images and save computing resources for image processing.

[0004] An image processing method, comprising:

[0005] Perform convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively;

[0006] Fusing the tone template image and the image to be processed to obtain a tone mapping image;

[0007] The tone mapping image is subjected to denoising processing based on the noise template image to obtain a first target image.

[0008] An image processing device, comprising:

[0009] A convolution module is used to perform convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively;

[0010] A fusion module, used for fusing the tone template image and the image to be processed to obtain a tone mapping image;

[0011] The denoising module is used to perform denoising processing on the tone mapping image based on the noise template image to obtain a first target image.

[0012] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0013] Perform convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively;

[0014] Fusing the tone template image and the image to be processed to obtain a tone mapping image;

[0015] The tone mapping image is subjected to denoising processing based on the noise template image to obtain a first target image.

[0016] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0017] Perform convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively;

[0018] Fusing the tone template image and the image to be processed to obtain a tone mapping image;

[0019] The tone mapping image is subjected to denoising processing based on the noise template image to obtain a first target image.

[0020] A computer program product, comprising a computer program / instruction, wherein the computer program / instruction is executed by a processor to:

[0021] Perform convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively;

[0022] Fusing the tone template image and the image to be processed to obtain a tone mapping image;

[0023] The tone mapping image is subjected to denoising processing based on the noise template image to obtain a first target image.

[0024] The above-mentioned image processing method, device, electronic device, computer-readable storage medium and computer program product can separate a tone template image containing tone information and a noise template image containing noise information from the convolution processing of the same image by performing convolution processing on the image to be processed, thereby reducing the amount of calculation and saving computing resources. By fusing the tone template image and the image to be processed, the detail information and features of the original image can be better expressed while tone mapping the image to be processed. By performing denoising processing on the tone mapped image based on the noise template image, the clarity of the image can be improved, thereby obtaining a first target image that retains the detail features of the image to be processed and ensures clarity.

[0025] A training method for an image processing model, comprising:

[0026] Acquire training sample images, and hue label images and noise label images corresponding to the training sample images respectively;

[0027] Performing convolution processing on the training sample image through an image processing model to obtain a tone sample image and a noise sample image respectively;

[0028] Fusing the tone sample image and the training sample image to obtain a tone mapping prediction image;

[0029] Performing denoising on the tone mapping prediction image according to the noise sample image to obtain a noise prediction image;

[0030] Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and training is continued until the training stop condition is met, thereby obtaining a trained image processing model.

[0031] A training device for an image processing model, comprising:

[0032] An acquisition module, used to acquire a training sample image, and a tone label image and a noise label image respectively corresponding to the training sample image;

[0033] A sample convolution module, used for performing convolution processing on the training sample image through an image processing model to obtain a tone sample image and a noise sample image respectively;

[0034] A sample fusion module, used for fusing the tone sample image and the training sample image to obtain a tone mapping prediction image;

[0035] A sample denoising module, configured to perform denoising on the tone mapping prediction image according to the noise sample image to obtain a noise prediction image;

[0036] An adjustment module is used to adjust the parameters of the image processing model based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, and continue training until the training stop condition is met, thereby obtaining a trained image processing model.

[0037] A computer device comprises a memory and a processor, wherein the memory stores a computer program, and when the processor executes the computer program, the following steps are implemented:

[0038] Acquire training sample images, and hue label images and noise label images corresponding to the training sample images respectively;

[0039] Performing convolution processing on the training sample image through an image processing model to obtain a tone sample image and a noise sample image respectively;

[0040] Fusing the tone sample image and the training sample image to obtain a tone mapping prediction image;

[0041] Performing denoising on the tone mapping prediction image according to the noise sample image to obtain a noise prediction image;

[0042] Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and training is continued until the training stop condition is met, thereby obtaining a trained image processing model.

[0043] A computer-readable storage medium stores a computer program, which, when executed by a processor, implements the following steps:

[0044] Acquire training sample images, and hue label images and noise label images corresponding to the training sample images respectively;

[0045] Performing convolution processing on the training sample image through an image processing model to obtain a tone sample image and a noise sample image respectively;

[0046] Fusing the tone sample image and the training sample image to obtain a tone mapping prediction image;

[0047] Performing denoising on the tone mapping prediction image according to the noise sample image to obtain a noise prediction image;

[0048] Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and training is continued until the training stop condition is met, thereby obtaining a trained image processing model.

[0049] A computer program product, comprising a computer program / instruction, wherein the computer program / instruction is executed by a processor to:

[0050] Acquire training sample images, and hue label images and noise label images corresponding to the training sample images respectively;

[0051] Performing convolution processing on the training sample image through an image processing model to obtain a tone sample image and a noise sample image respectively;

[0052] Fusing the tone sample image and the training sample image to obtain a tone mapping prediction image;

[0053] Performing denoising on the tone mapping prediction image according to the noise sample image to obtain a noise prediction image;

[0054] Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and training is continued until the training stop condition is met, thereby obtaining a trained image processing model.

[0055] The training method, device, electronic device, computer-readable storage medium and computer program product of the above-mentioned image processing model can reduce the amount of calculation by obtaining training sample images, and tone label images and noise label images corresponding to the training sample images, respectively, and performing convolution processing on the training sample images through the image processing model to obtain tone sample images containing tone information and noise sample images containing noise information, respectively. The tone sample images and training sample images are fused to obtain a tone mapping prediction image, and the tone mapping prediction image is denoised according to the noise sample image to obtain a noise prediction image. Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and the training is continued until the training stop condition is met, and a trained image processing model is obtained, so that tone mapping and image denoising can be realized in the same model, which can effectively save computing resources and improve the efficiency of image processing, and ensure the clarity of the image while retaining the image detail information. BRIEF DESCRIPTION OF THE DRAWINGS

[0056] In order to more clearly illustrate the embodiments of the present application or the technical solutions in the prior art, the drawings required for use in the embodiments or the description of the prior art will be briefly introduced below. Obviously, the drawings described below are only some embodiments of the present application. For ordinary technicians in this field, other drawings can be obtained based on these drawings without paying creative work.

[0057] Figure 1 A diagram showing an application environment of an image processing method in an embodiment;

[0058] Figure 2 is a flowchart of an image processing method in one embodiment;

[0059] Figure 3 Schematic diagram of the structure of a convolutional neural network in one embodiment;

[0060] Figure 4 A flowchart of obtaining a tone template image and a noise template image in one embodiment;

[0061] Figure 5 A flowchart of obtaining a first target image after obtaining a tone template image and a noise template image in one embodiment;

[0062] Figure 6 is a flowchart of an image processing method in another embodiment;

[0063] Figure 7 is a flowchart of a method for training an image processing model in one embodiment;

[0064] Figure 8 A flowchart of obtaining a hue label image and a noise label image respectively corresponding to a training sample image in one embodiment;

[0065] Fig. 9 is a structural block diagram of an image processing device in one embodiment;

[0066] Fig.10 is a structural block diagram of a training device for an image processing model in one embodiment;

[0067] Fig.11 FIG. 4 is a diagram showing the internal structure of an electronic device in one embodiment. DETAILED DESCRIPTION

[0068] In order to make the purpose, technical solution and advantages of the present application more clearly understood, the present application is further described in detail below in conjunction with the accompanying drawings and embodiments. It should be understood that the specific embodiments described herein are only used to explain the present application and are not used to limit the present application.

[0069] It is understood that the terms "first", "second", etc. used in this application may be used herein to describe various elements, but these elements are not limited by these terms. These terms are only used to distinguish a first element from another element. For example, without departing from the scope of this application, a first target image may be referred to as a second target image, and similarly, a second target image may be referred to as a first target image. Both the first target image and the second target image are target images, but they are not the same target image.

[0070] Figure 1 FIG. 1 is a schematic diagram of an application environment of an image processing method in an embodiment. Figure 1As shown, the application environment includes an electronic device 110 and a server 120. In one embodiment, the electronic device 110 and the server 120 can each execute the image processing method separately, or the electronic device 110 and the server 120 can execute the image processing method in collaboration. When the electronic device 110 and the server 120 execute the image processing method in collaboration, the electronic device 110 obtains the image to be processed and sends the image to be processed to the server 120. The server 120 performs convolution processing on the image to be processed to obtain a tone template image and a noise template image, respectively. The server performs fusion processing on the tone template image and the image to be processed to obtain a tone mapping image. The server performs denoising processing on the tone mapping image based on the noise template image to obtain a first target image, and returns the first target image to the electronic device 110.

[0071] The electronic device 110 communicates with the server 120 via a network. The electronic device 110 may be, but is not limited to, various personal computers, laptops, smart phones, tablet computers, and portable wearable devices, and the server 120 may be implemented as an independent server or a server cluster consisting of multiple servers.

[0072] Figure 2 FIG. 1 is a flowchart of an image processing method in an embodiment. The image processing method in this embodiment is executed on Figure 1 The electronic device in the example is used to describe. Figure 2 As shown, the image processing method includes steps 202 to 206.

[0073] Step 202: Perform convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively.

[0074] The image to be processed may be a high dynamic range image (HDR), specifically, any one of RGB (Red, Green, Blue) images, RAW images, grayscale images, depth images, images corresponding to the Y component in YUV images, etc. RAW images are raw data that the image sensor converts the captured light source signal into a digital signal. The "Y" in the YUV image represents brightness (Luminance or Luma), which is the grayscale value, and "U" and "V" represent chrominance (Chrominance or Chroma), which is used to describe the color and saturation of the image and is used to specify the color of the pixel.

[0075] The tone template image is an image related to tone separated from the image to be processed, and the noise template image is a noise image separated from the image to be processed.

[0076] Specifically, the electronic device may obtain the image to be processed from a local device or other device or network, or the electronic device may obtain the image to be processed by photographing a scene with a camera. The image to be processed may also be a video frame in a video obtained from a local device or other device or network, or a video frame in a video photographed with a camera.

[0077] The electronic device performs convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively. Further, the electronic device performs different convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively.

[0078] In one embodiment, the Figure 3 The convolutional neural network shown performs convolution, pooling and deconvolution on the image to be processed, and obtains a tone template image and a noise template image respectively.

[0079] In one implementation, the electronic device inputs the image to be processed into a tone mapping network, which is composed of at least two convolution layers, and performs convolution processing on the image to be processed through the tone mapping network to obtain a tone template image. The electronic device inputs the image to be processed into a noise network, which is composed of at least two convolution layers, and performs convolution processing on the image to be processed through the noise network to obtain a noise template image.

[0080] Step 204 , the tone template image and the image to be processed are fused to obtain a tone mapped image.

[0081] Specifically, the electronic device may fuse the tone template image and the image to be processed to obtain a tone mapping image after the fusion process.

[0082] In one embodiment, the electronic device may perform weighted averaging processing on the tone template image and the image to be processed to obtain a tone mapping image.

[0083] In one embodiment, the tone template image and the image to be processed are fused to obtain a tone mapping image, including: performing pixel multiplication on the tone template image and the image to be processed to obtain a tone mapping image.

[0084] Step 206: Perform noise reduction processing on the tone mapping image based on the noise template image to obtain a first target image.

[0085] Specifically, the electronic device performs noise reduction processing on the tone mapping image based on the noise template image to obtain a first target image after noise reduction. The first target image may be a low dynamic range image (Low-Dynamic Range, LDR for short).

[0086] In one embodiment, the electronic device may perform weighted averaging processing on the noise template image and the tone mapping image to obtain a first target image after noise reduction.

[0087] In one embodiment, performing noise reduction processing on the tone mapping image based on the noise template image to obtain the first target image includes: performing pixel addition processing on the noise template image and the tone mapping image to obtain the first target image.

[0088] The image processing method in this embodiment can separate the tone template image containing tone information and the noise template image containing noise information from the convolution processing of the same image by performing convolution processing on the image to be processed, thereby reducing the amount of calculation and saving computing resources. By fusing the tone template image and the image to be processed, the detail information and features of the original image can be better expressed while tone mapping the image to be processed. Denoising the tone mapped image based on the noise template image can improve the clarity of the image, thereby obtaining a first target image that retains the detail features of the image to be processed and ensures clarity.

[0089] In one embodiment, performing convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively includes: performing normalization processing on the image to be processed, and performing convolution processing on the normalized image to obtain a tone template image and a noise template image respectively;

[0090] Fusing the tone template image and the image to be processed to obtain a tone mapping image, including: fusing the tone template image and the normalized image to obtain a tone mapping image;

[0091] The method performs denoising on the tone mapping image based on the noise template image to obtain a first target image, including: performing denoising on the tone mapping image based on the noise template image to obtain a denoised image; and performing image restoration on the denoised image to obtain the first target image.

[0092] Specifically, the electronic device may perform normalization processing on the image to be processed to obtain a normalized image.

[0093] In one embodiment, the electronic device obtains the maximum value max and the minimum value min of the pixels in the image to be processed, and then subtracts the minimum value min from each pixel value pixel in the image to be processed, and divides the result by the difference between the maximum value max and the minimum value min to obtain a normalized pixel value pixel_out, thereby obtaining a normalized image. The specific calculation formula is as follows:

[0094]

[0095] Next, the electronic device may perform different convolution processes on the normalized image to obtain a tone template image and a noise template image respectively. The electronic device may perform fusion processing on the tone template image and the normalized image to obtain a tone mapping image.

[0096] The tone mapping image is subjected to denoising based on the noise template image to obtain a denoised image, and then the denoised image is subjected to image restoration to obtain a first target image. Further, after obtaining the denoised image, the denoised image is cropped, that is, values ​​greater than 1 are set to 1, and values ​​less than 0 are set to 0, and then the image is multiplied pixel by pixel by 255 to obtain the first target image. The restoration formula is as follows:

[0097] pixel_out=max(0,min(1.0,pixel))×255.0

[0098] Among them, pixel is the input pixel and pixel_out is the output pixel.

[0099] In this embodiment, the image to be processed is normalized to reduce the amount of calculation for subsequent processing and improve the calculation efficiency. The normalized image is convolved to separate a tone template image containing tone information and a noise template image containing noise information. The tone template image containing tone information and the normalized image are fused to obtain a tone mapping image, which can effectively compress the tone of the image to be processed from a high dynamic range to a low dynamic range and retain the detailed features of the image. The tone mapping image is subjected to denoising and image restoration based on the noise template image, which can reduce the noise of the image and obtain a clearer first target image.

[0100] In one embodiment, the normalized image is subjected to convolution processing to obtain a tone template image and a noise template image, respectively, including:

[0101] The normalized image is convolved based on the tone mapping parameters to obtain a corresponding tone template image; the normalized image is convolved based on the noise extraction parameters to obtain a corresponding noise template image.

[0102] Among them, the tone mapping parameters and the noise extraction parameters are parameters obtained from the trained image processing model.

[0103] Specifically, the electronic device may obtain tone mapping parameters, and perform convolution processing on the normalized image based on the tone mapping parameters to obtain a tone mapping template. The electronic device may obtain tone mapping parameters, and perform convolution processing on the normalized image based on the noise extraction parameters to obtain a noise template image.

[0104] In one embodiment, Figure 4 As shown, the image to be processed is a high dynamic range image, and the trained image processing model includes a tone mapping network and a noise network. The tone mapping network includes tone mapping parameters, and the noise network includes noise extraction parameters.

[0105] Step 402: The electronic device acquires a high dynamic range image and inputs the high dynamic range image into a trained image processing model.

[0106] Step 404 , normalizing the high dynamic range image using an image processing model to obtain a normalized image.

[0107] Step 406, sending the normalized image to the encoder network of the image processing model, wherein the encoder network is composed of a number of convolutional layers and pooling layers.

[0108] Step 408: Send the image features obtained by the encoder network to the decoder network, wherein the decoder network is composed of a number of convolutional layers and deconvolutional layers. The image features after passing through the decoder network are sent to the tone mapping network and the noise network respectively.

[0109] Step 410 , convolution processing is performed on the decoded image features using the tone mapping parameters of the tone mapping network to obtain a tone template image 414 .

[0110] Step 412, convolution processing is performed on the decoded image features through a noise network to obtain a noise template image 416.

[0111] In this embodiment, the normalized image is convolved based on the tone mapping parameters, which can reduce the amount of calculation, thereby quickly separating the tone template image containing tone information from the image to be processed. The normalized image is convolved based on the noise extraction parameters, which can also reduce the amount of calculation, thereby quickly separating the noise template image containing image noise from the image to be processed.

[0112] In one embodiment, the tone template image and the image to be processed are fused to obtain a tone mapping image, which includes: performing pixel multiplication on the tone template image and the image to be processed to obtain a tone mapping image.

[0113] Specifically, the electronic device may perform pixel matching between the noise template image and the tone mapping image to determine the matching pixels between the noise template image and the tone mapping image. The electronic device performs dot multiplication processing on the pixel values ​​of the matching pixels in the noise template image and the pixel values ​​in the tone mapping image to obtain the pixel values ​​of the corresponding pixels in the tone mapping image. In the same processing manner, the pixel values ​​of each matching pixel are dot multiplied to obtain the pixel values ​​corresponding to each pixel in the tone mapping image, thereby obtaining the tone mapping image.

[0114] In this embodiment, pixel multiplication is performed on the tone template image and the image to be processed, so that the brightness range of the image to be processed can be adjusted and the obtained tone mapping image can effectively retain the detail information and color information of the image to be processed.

[0115] In one embodiment, performing noise reduction processing on the tone mapping image based on the noise template image to obtain the first target image includes: performing pixel addition processing on the noise template image and the tone mapping image to obtain the first target image.

[0116] Specifically, the electronic device may match the noise template image and the tone mapping image for pixels, and determine the matching pixels between the noise template image and the tone mapping image. The electronic device adds the pixel values ​​of the matching pixels in the noise template image and the pixel values ​​in the tone mapping image to fuse the matching pixels, and obtain the pixel values ​​of the corresponding pixels in the first target image. According to the same processing method, each pixel in the first target image and the pixel value corresponding to each pixel can be obtained, thereby obtaining the first target image.

[0117] In this embodiment, pixel addition processing is performed on the noise template image and the tone mapping image to obtain the first target image, which can simply and effectively reduce the noise of the image and improve the clarity of the image.

[0118] like Figure 5 , which is a flowchart of obtaining a first target image after obtaining a tone template image and a noise template image in one embodiment. The normalized image 502 and the tone template image 504 are subjected to pixel multiplication processing in step 506 to obtain a tone mapping image 508. The tone mapping image 508 and the noise template image 510 are subjected to pixel addition denoising processing in step 512 to obtain a denoised image 514. The denoised image 514 is subjected to image restoration processing in step 516 to obtain a first target image 518.

[0119] like Figure 6, which is a flow chart of an image processing method in one embodiment. Obtain an image to be processed 602, perform normalization processing in step 604 on the image to be processed 602, and obtain a normalized image. Input the normalized image into a convolutional neural network 606. The convolutional neural network 606 includes an encoder network, a decoder network, a tone mapping network, and a noise network. The convolutional neural network 606 performs convolution processing on the normalized image to obtain a tone template image 608 and a noise template image 610, respectively.

[0120] The normalized image and the tone template image 608 are subjected to tone mapping processing in step 612, that is, pixel multiplication processing is performed to obtain a tone mapping image. The tone mapping image and the noise template image 610 are subjected to denoising processing, that is, pixel addition denoising processing is performed to obtain a denoised image. The denoised image is subjected to image restoration processing in step 616 to obtain a first target image 618.

[0121] In one embodiment, an image processing method is provided. The image processing method in this embodiment is executed on Figure 1 The electronic device in the example is described as follows, including:

[0122] The image to be processed is subjected to convolution processing to obtain a tone template image and a noise template image respectively; the image to be processed is subjected to denoising processing based on the noise template image, and the tone template image and the image subjected to denoising processing are fused to obtain a second target image.

[0123] Specifically, the electronic device performs different convolution processes on the images to be processed to obtain a tone template image and a noise template image respectively.

[0124] In one embodiment, performing convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively includes: performing normalization processing on the image to be processed, and performing convolution processing on the normalized image to obtain a tone template image and a noise template image respectively.

[0125] In one embodiment, convolution processing is performed on the normalized image to obtain a tone template image and a noise template image, respectively, including: convolution processing is performed on the normalized image based on tone mapping parameters to obtain a corresponding tone template image; convolution processing is performed on the normalized image based on noise extraction parameters to obtain a corresponding noise template image.

[0126] After obtaining the tone template image and the noise template image, the electronic device can perform noise reduction processing on the image to be processed based on the noise template image. Further, the noise template image and the image to be processed can be pixel-added to reduce the noise of the image to be processed. The tone template image and the image after noise reduction processing are fused to obtain a second target image. Further, the tone template image and the image after noise reduction processing are pixel-multiplied to obtain a second target image.

[0127] In one embodiment, performing noise reduction processing on the image to be processed based on the noise template image, and fusing the tone template image with the image after noise reduction processing to obtain a second target image, includes:

[0128] The normalized image is subjected to denoising based on the noise template image, and the tone template image and the denoising image are fused to obtain a second target image.

[0129] In one embodiment, performing noise reduction processing on the normalized image based on the noise template image includes: performing pixel addition processing on the noise template image and the normalized image to obtain the noise reduced image;

[0130] The tone template image and the image after noise reduction processing are fused to obtain a second target image, including: performing pixel multiplication processing on the tone template image and the image after noise reduction processing to obtain the second target image.

[0131] In this embodiment, by performing convolution processing on the image to be processed, a tone template image containing tone information and a noise template image containing noise information can be separated from the convolution processing of the same image, which can reduce the amount of calculation and save computing resources. By performing noise reduction processing on the image to be processed based on the noise template image, the noise of the image can be reduced and the clarity of the image can be improved. By fusing the tone template image and the image after noise reduction processing, the detailed information and features of the original image can be better expressed.

[0132] It can be understood that the specific processing processes such as normalization, fusion, denoising, pixel addition and pixel multiplication in this embodiment can refer to the corresponding processing in the above embodiments and will not be repeated here.

[0133] Figure 7 FIG. 1 is a flowchart of a method for training an image processing model in an embodiment. The method for training an image processing model in this embodiment is to run on Figure 1 The electronic device in the example is used to describe. Figure 7 As shown, the training method of the image processing model includes:

[0134] Step 702: Obtain training sample images, and the hue label images and noise label images corresponding to the training sample images.

[0135] The training sample image may be a high dynamic range image, specifically any one of an RGB image, a RAW image, a grayscale image, a depth image, an image corresponding to the Y component in a YUV image, etc. The hue label image is a real image used to characterize the hue in the training sample image, and the noise label image is a real image used to characterize the noise in the training sample image.

[0136] Specifically, the electronic device may obtain each training sample image from a local device or other device or network, or the electronic device may obtain each training sample image by photographing a scene with a camera. The training sample image may also be a video frame in a video obtained from a local device or other device or network, or a video frame in a video photographed with a camera.

[0137] When there are multiple training sample images, the electronic device obtains a tone label image and a noise label image corresponding to each training sample image respectively.

[0138] Step 704: Perform convolution processing on the training sample image through the image processing model to obtain a tone sample image and a noise sample image respectively.

[0139] Specifically, the electronic device inputs the training sample image into the image processing model, and performs convolution processing on the image to be processed through the image processing model to obtain a tone sample image and a noise sample image respectively. Further, the image processing model performs different convolution processing on the training sample image to obtain a tone sample image and a noise sample image respectively.

[0140] In one embodiment, the image processing model includes an encoder network, a decoder network, a tone mapping network and a noise network. The image processing model can be used to normalize the image to be processed to obtain a normalized image. The normalized image is sent to the encoder network of the image processing model for encoding, and the image features obtained by encoding are sent to the decoder network for decoding. The image features after passing through the decoder network are respectively sent to the tone mapping network and the noise network, and the decoded image features are convolved through the tone mapping network to obtain a tone sample image. The decoded image features are convolved through the noise network to obtain a noise sample image.

[0141] Step 706: fuse the tone sample image and the training sample image to obtain a tone mapping prediction image.

[0142] Specifically, the electronic device may fuse the tone sample image and the training sample image to obtain a tone mapping prediction image after the fusion process.

[0143] In one embodiment, the tone sample image and the training sample image are fused to obtain a tone mapping prediction image, including: performing pixel multiplication on the tone sample image and the training sample image to obtain a tone mapping prediction image.

[0144] Step 708: Perform denoising on the tone mapping prediction image according to the noise sample image to obtain a noise prediction image.

[0145] Specifically, the electronic device performs noise reduction processing on the tone mapping prediction image based on the noise sample image to obtain a noise prediction image after noise reduction. The noise prediction image may be a low dynamic range image.

[0146] In one embodiment, performing denoising on the tone mapping prediction image based on the noise sample image to obtain the noise prediction image includes: performing pixel addition processing on the noise sample image and the tone mapping prediction image to obtain the noise prediction image.

[0147] Step 710, based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, adjust the parameters of the image processing model and continue training until the training stop condition is met, thereby obtaining a trained image processing model.

[0148] Specifically, the electronic device calculates the difference between the tone mapping prediction image and the corresponding tone label image, and calculates the difference between the noise prediction image and the corresponding noise label image, adjusts the parameters of the image processing model according to the two differences, and continues to train the image processing model with adjusted parameters until the training stop condition is met, thereby obtaining a trained image processing model.

[0149] In one embodiment, the training stop condition may be that the loss error is less than an error threshold, or the number of training iterations reaches a preset number of iterations, etc.

[0150] For example, the electronic device calculates a first loss error between a tone mapping prediction image and a corresponding tone label image, and calculates a second loss error between a noise prediction image and a corresponding noise label image, and adjusts the parameters of the image processing model according to the first loss error and the second loss error. Furthermore, when at least one of the first loss error is greater than a first error threshold and the second loss error is greater than a second error threshold exists, the parameters of the image processing model are adjusted, and the image processing model with the adjusted parameters is continuously trained until the first loss error is not greater than the first error threshold and the second loss error is not greater than the second error threshold, thereby obtaining a trained image processing model.

[0151] In this embodiment, a training sample image and a tone label image and a noise label image corresponding to the training sample image are obtained, and a convolution process is performed on the training sample image through an image processing model to obtain a tone sample image containing tone information and a noise sample image containing noise information, respectively, which can reduce the amount of calculation. The tone sample image and the training sample image are fused to obtain a tone mapping prediction image, and the tone mapping prediction image is denoised according to the noise sample image to obtain a noise prediction image. Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and the training is continued until the training stop condition is met, and a trained image processing model is obtained, so that tone mapping and image denoising can be realized in the same model, which can effectively save computing resources and improve the efficiency of image processing, and ensure the clarity of the image while retaining the image detail information.

[0152] In one embodiment, convolution processing is performed on the training sample image through an image processing model to obtain a tone sample image and a noise sample image, respectively, including:

[0153] Performing convolution processing on the training sample image based on the initial tone mapping parameters of the image processing model to obtain a corresponding tone sample image; performing convolution processing on the training sample image based on the initial noise extraction parameters of the image processing model to obtain a corresponding noise sample image;

[0154] Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and training continues, including: based on the difference between the tone mapping prediction image and the corresponding tone label image, the initial tone mapping parameters of the image processing model are adjusted, and based on the difference between the noise prediction image and the corresponding noise label image, the initial noise extraction parameters of the image processing model are adjusted and training continues.

[0155] Specifically, the untrained image processing model includes initial tone mapping parameters and initial noise extraction parameters, and the initial tone mapping parameters may refer to the original tone mapping parameters included in the untrained image processing model, or may be the adjusted tone mapping parameters. The initial noise extraction parameters may refer to the original noise extraction parameters included in the untrained image processing model, or may be the adjusted noise extraction parameters.

[0156] The training sample image is convolved with the initial tone mapping parameters of the tone mapping network in the image processing model to obtain the corresponding tone sample image. The training sample image is convolved with the initial noise extraction parameters of the noise network in the image processing model to obtain the corresponding noise sample image.

[0157] In one embodiment, the normalized image is convolved with the initial tone mapping parameters of the tone mapping network to obtain a corresponding tone sample image. The normalized image is convolved with the initial noise extraction parameters of the noise network to obtain a corresponding noise sample image.

[0158] Based on the difference between the tone mapping prediction image and the corresponding tone label image, the initial tone mapping parameters of the tone mapping network are adjusted. Based on the difference between the noise prediction image and the corresponding noise label image, the initial noise extraction parameters of the noise network are adjusted. The image processing model after adjusting the initial tone mapping parameters and the initial noise extraction parameters continues to be trained until the training stop condition is met, and a trained image processing model is obtained. The trained image processing model includes tone mapping parameters and noise extraction parameters.

[0159] In this embodiment, convolution processing is performed on the training sample image based on the initial tone mapping parameters, which can reduce the amount of calculation, thereby quickly separating the tone sample image containing tone information from the original image. Convolution processing is performed on the training sample image based on the noise extraction parameters, which can also reduce the amount of calculation, thereby quickly separating the noise sample image containing image noise from the original image. Based on the difference between the tone mapping prediction image and the corresponding tone label image, the initial tone mapping parameters of the image processing model are adjusted, and based on the difference between the noise prediction image and the corresponding noise label image, the initial noise extraction parameters of the image processing model are adjusted, so that the tone mapping parameters and noise extraction parameters of the model can be accurately adjusted, which can effectively improve the processing accuracy of the model.

[0160] In one embodiment, obtaining a training sample image, and a hue label image and a noise label image respectively corresponding to the training sample image, includes:

[0161] Acquire each frame of training sample image, perform registration processing on each frame of training sample image to obtain each registered image; perform tone mapping processing on each registered image, and select a tone label image from the tone mapped image; perform multi-frame denoising processing on the tone mapped image to obtain a noise label image.

[0162] Specifically, the electronic device may acquire multiple frames of training sample images, and select a reference image from the multiple frames of training sample images. Further, the first frame of training sample images may be selected as the reference image.

[0163] The reference image is used to register each frame of the training sample image to obtain each frame of the registered image. The electronic device can perform tone mapping on each frame of the registered image to obtain the corresponding tone mapped image.

[0164] The electronic device may select a tone label image from each tone mapped image, and perform multi-frame noise reduction on each tone mapped image to obtain a noise label image. Further, the electronic device may add and average matching pixels in each tone mapped image to generate a new image, and use the new image as the noise label image.

[0165] In one embodiment, tone mapping processing may be performed on the reference image to obtain a tone label image.

[0166] In one embodiment, tone mapping processing may be performed on each frame of the registered image and the reference image by using a local tone mapping algorithm or a global tone mapping algorithm.

[0167] In this embodiment, each frame of training sample images is obtained, and registration processing is performed on each frame of training sample images to obtain each registered image. Tone mapping processing is performed on each registered image, and a tone label image is selected from the tone mapped image. Multi-frame denoising processing is performed on the tone mapped image to obtain a noise label image, which can ensure the reliability of the tone label image and the noise label image used as the comparison label.

[0168] like Figure 8 FIG. 1 is a flowchart of obtaining a hue label image and a noise label image corresponding to a training sample image in one embodiment.

[0169] Step 802, obtain 10 training sample images of the same scene, and use the first training sample image as a reference image. For each scene, 10 still images can be collected, or more than 10 images can be collected, and the collection method is continuous collection on the tripod. The displacement of continuous images on the tripod is small, and continuous collection on the tripod is conducive to subsequent registration and multi-frame processing. It is also possible to collect multiple groups of training sample images for different scenes, for example, 50 groups, each with 10 images.

[0170] Step 804: perform image registration on the remaining 9 training sample images respectively using the reference image to obtain 9 registered images.

[0171] Step 806 , tone mapping is performed on the reference image and the nine registered images respectively to obtain ten tone mapped images, and the tone mapped images of the reference image are used as tone label images 808 .

[0172] Step 810 , add pixels of the 10 tone-mapped images and take the average to obtain a noisy label image 812 .

[0173] In one embodiment, an image processing method is provided, which is performed by an electronic device and includes:

[0174] Each frame of training sample image is obtained, and registration processing is performed on each frame of training sample image to obtain each registered image.

[0175] Next, tone mapping is performed on each registered image, and a tone label image is selected from the tone mapped image; and multi-frame noise reduction is performed on the tone mapped image to obtain a noise label image.

[0176] Furthermore, the training sample image is convolved based on the initial tone mapping parameters of the tone mapping network in the image processing model to obtain the corresponding tone sample image, and the training sample image is convolved based on the initial noise extraction parameters of the noise network in the image processing model to obtain the corresponding noise sample image.

[0177] Next, the tone sample image and the training sample image are fused to obtain a tone mapping prediction image.

[0178] Furthermore, the tone mapping prediction image is subjected to denoising according to the noise sample image to obtain a noise prediction image.

[0179] Furthermore, based on the difference between the tone mapping prediction image and the corresponding tone label image, the initial tone mapping parameters of the image processing model are adjusted, and based on the difference between the noise prediction image and the corresponding noise label image, the initial noise extraction parameters of the image processing model are adjusted and training is continued until the training stop condition is met to obtain a trained image processing model.

[0180] Obtain the image to be processed and input the image to be processed into the trained image processing model. The image processing model includes a tone mapping network and a noise network.

[0181] The image to be processed is normalized by the image processing model, and the normalized image is convolved based on the tone mapping parameters of the tone mapping network to obtain a corresponding tone template image.

[0182] Next, the normalized image is convolved based on the noise extraction parameters of the noise network to obtain the corresponding noise template image.

[0183] Next, pixel multiplication is performed on the tone template image and the normalized image to obtain a tone mapping image.

[0184] Furthermore, pixel addition processing is performed on the tone mapping image based on the noise template image to obtain a denoised image.

[0185] Furthermore, the denoised image is subjected to image restoration processing to obtain a first target image.

[0186] In this embodiment, each frame of training sample image is registered, and each registered image is tone mapped, and a tone label image is selected from the tone mapped image, and the tone mapped image is subjected to multi-frame denoising to obtain a noise label image, which can ensure the reliability of the tone label image and the noise label image used as the comparison label.

[0187] The training sample image is convolved by the image processing model to obtain a tone sample image containing tone information and a noise sample image containing noise information, respectively, which can reduce the amount of calculation. The tone sample image and the training sample image are fused to obtain a tone mapping prediction image, and the tone mapping prediction image is denoised according to the noise sample image to obtain a noise prediction image. Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and the training is continued until the training stop condition is met, and a trained image processing model is obtained, so that tone mapping and image denoising can be realized in the same model, which can effectively save computing resources and improve the efficiency of image processing.

[0188] By performing convolution processing on the image to be processed, the tone template image containing tone information and the noise template image containing noise information can be separated from the convolution processing of the same image, which can reduce the amount of calculation and save computing resources. By fusing the tone template image and the image to be processed, the detail information and features of the original image can be better expressed while tone mapping the image to be processed. Denoising the tone mapped image based on the noise template image can improve the clarity of the image, thereby obtaining a first target image that retains the detail features of the image to be processed and ensures clarity.

[0189] It should be understood that although Figure 2-Figure 8 The steps in the flowchart are shown in sequence as indicated by the arrows, but these steps are not necessarily executed in the order indicated by the arrows. Unless otherwise specified in this document, there is no strict order restriction for the execution of these steps, and these steps can be executed in other orders. Moreover, Figure 2-Figure 8At least part of the steps may include multiple sub-steps or multiple stages. These sub-steps or stages are not necessarily executed at the same time, but can be executed at different times. The execution order of these sub-steps or stages is not necessarily sequential, but can be executed in turn or alternately with other steps or at least part of the sub-steps or stages of other steps.

[0190] Fig. 9 FIG. 1 is a structural block diagram of an image processing device according to an embodiment. Fig. 9 As shown, the image processing device comprises:

[0191] The convolution module 902 is used to perform convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively.

[0192] The fusion module 904 is used to fuse the tone template image and the image to be processed to obtain a tone mapping image.

[0193] The denoising module 906 is configured to perform denoising processing on the tone mapping image based on the noise template image to obtain a first target image.

[0194] In this embodiment, by performing convolution processing on the image to be processed, a tone template image containing tone information and a noise template image containing noise information can be separated from the convolution processing of the same image, which can reduce the amount of calculation and save computing resources. By fusing the tone template image and the image to be processed, the detail information and features of the original image can be better expressed while tone mapping the image to be processed. Denoising the tone mapped image based on the noise template image can improve the clarity of the image, thereby obtaining a first target image that retains the detail features of the image to be processed and ensures clarity.

[0195] In one embodiment, the convolution module 902 is further used to perform normalization processing on the image to be processed, and perform convolution processing on the normalized image to obtain a tone template image and a noise template image respectively;

[0196] The fusion module 904 is further used to fuse the tone template image and the normalized image to obtain a tone mapping image;

[0197] The noise reduction module 906 is further configured to perform noise reduction processing on the tone mapping image based on the noise template image to obtain a noise-reduced image; and perform image restoration processing on the noise-reduced image to obtain a first target image.

[0198] In this embodiment, the image to be processed is normalized to reduce the amount of calculation for subsequent processing and improve the calculation efficiency. The normalized image is convolved to separate a tone template image containing tone information and a noise template image containing noise information. The tone template image containing tone information and the normalized image are fused to obtain a tone mapping image, which can effectively compress the tone of the image to be processed from a high dynamic range to a low dynamic range and retain the detailed features of the image. The tone mapping image is subjected to denoising and image restoration based on the noise template image, which can reduce the noise of the image and obtain a clearer first target image.

[0199] In one embodiment, the convolution module 902 is further used to perform convolution processing on the normalized image based on the tone mapping parameters to obtain a corresponding tone template image; and to perform convolution processing on the normalized image based on the noise extraction parameters to obtain a corresponding noise template image.

[0200] In this embodiment, the normalized image is convolved based on the tone mapping parameters, which can reduce the amount of calculation, thereby quickly separating the tone template image containing tone information from the image to be processed. The normalized image is convolved based on the noise extraction parameters, which can also reduce the amount of calculation, thereby quickly separating the noise template image containing image noise from the image to be processed.

[0201] In one embodiment, the fusion module 904 is further used to perform pixel multiplication processing on the tone template image and the image to be processed to obtain a tone mapping image;

[0202] In this embodiment, pixel multiplication is performed on the tone template image and the image to be processed, so that the brightness range of the image to be processed can be adjusted and the obtained tone mapping image can effectively retain the detail information and color information of the image to be processed.

[0203] In one embodiment, the noise reduction module 906 is further configured to perform pixel addition processing on the noise template image and the tone mapping image to obtain a first target image.

[0204] In this embodiment, pixel addition processing is performed on the noise template image and the tone mapping image to obtain the first target image, which can simply and effectively reduce the noise of the image and improve the clarity of the image.

[0205] In one embodiment, an image processing device is provided, including a convolution module 902 and a processing module, wherein:

[0206] The convolution module 902 is used to perform convolution processing on the image to be processed to obtain a tone template image and a noise template image respectively.

[0207] The processing module is used to perform noise reduction processing on the image to be processed based on the noise template image, and fuse the tone template image with the image after noise reduction processing to obtain a second target image.

[0208] In this embodiment, by performing convolution processing on the image to be processed, a tone template image containing tone information and a noise template image containing noise information can be separated from the convolution processing of the same image, which can reduce the amount of calculation and save computing resources. By performing noise reduction processing on the image to be processed based on the noise template image, the noise of the image can be reduced and the clarity of the image can be improved. By fusing the tone template image and the image after noise reduction processing, the detailed information and features of the original image can be better expressed.

[0209] The division of the modules in the above-mentioned image processing device is only for illustration. In other embodiments, the image processing device may be divided into different modules as needed to complete all or part of the functions of the above-mentioned image processing device.

[0210] Fig.10 FIG. 4 is a structural block diagram of a training device for an image processing model according to an embodiment of the present invention. Fig.10 As shown, the device comprises:

[0211] An acquisition module 1002 is used to acquire a training sample image, and a tone label image and a noise label image corresponding to the training sample image;

[0212] The sample convolution module 1004 is used to perform convolution processing on the training sample image through the image processing model to obtain a tone sample image and a noise sample image respectively;

[0213] The sample fusion module 1006 is used to fuse the tone sample image and the training sample image to obtain a tone mapping prediction image;

[0214] A sample denoising module 1008 is used to perform denoising on the tone mapping prediction image according to the noise sample image to obtain a noise prediction image;

[0215] The adjustment module 1010 is used to adjust the parameters of the image processing model based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, and continue training until the training stop condition is met, thereby obtaining a trained image processing model.

[0216] In this embodiment, a training sample image and a tone label image and a noise label image corresponding to the training sample image are obtained, and the training sample image is convolved by an image processing model to obtain a tone sample image containing tone information and a noise sample image containing noise information, which can reduce the amount of calculation. The tone sample image and the training sample image are fused to obtain a tone mapping prediction image, and the tone mapping prediction image is denoised according to the noise sample image to obtain a noise prediction image. Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and the training is continued until the training stop condition is met, and a trained image processing model is obtained, so that tone mapping and image denoising can be realized in the same model, which can effectively save computing resources and improve the efficiency of image processing, and ensure the clarity of the image while retaining the image detail information.

[0217] In one embodiment, the sample convolution module 1004 is further used to perform convolution processing on the training sample image based on the initial tone mapping parameters of the image processing model to obtain a corresponding tone sample image; perform convolution processing on the training sample image based on the initial noise extraction parameters of the image processing model to obtain a corresponding noise sample image;

[0218] The adjustment module 1010 is also used to adjust the initial tone mapping parameters of the image processing model based on the difference between the tone mapping prediction image and the corresponding tone label image, and to adjust the initial noise extraction parameters of the image processing model based on the difference between the noise prediction image and the corresponding noise label image and continue training.

[0219] In this embodiment, convolution processing is performed on the training sample image based on the initial tone mapping parameters, which can reduce the amount of calculation, thereby quickly separating the tone sample image containing tone information from the original image. Convolution processing is performed on the training sample image based on the noise extraction parameters, which can also reduce the amount of calculation, thereby quickly separating the noise sample image containing image noise from the original image. Based on the difference between the tone mapping prediction image and the corresponding tone label image, the initial tone mapping parameters of the image processing model are adjusted, and based on the difference between the noise prediction image and the corresponding noise label image, the initial noise extraction parameters of the image processing model are adjusted, so that the tone mapping parameters and noise extraction parameters of the model can be accurately adjusted, which can effectively improve the processing accuracy of the model.

[0220] In one embodiment, the acquisition module 1002 is also used to acquire each frame of training sample images, perform registration processing on each frame of training sample images to obtain each registered image; perform tone mapping processing on each registered image, and select a tone label image from the tone mapped image; perform multi-frame denoising processing on the tone mapped image to obtain a noise label image.

[0221] In this embodiment, each frame of training sample images is obtained, and registration processing is performed on each frame of training sample images to obtain each registered image. Tone mapping processing is performed on each registered image, and a tone label image is selected from the tone mapped image. Multi-frame denoising processing is performed on the tone mapped image to obtain a noise label image, which can ensure the reliability of the tone label image and the noise label image used as the comparison label.

[0222] The division of the various modules in the above-mentioned training device of the image processing model is only for illustration. In other embodiments, the training device of the image processing model can be divided into different modules as needed to complete all or part of the functions of the above-mentioned training device of the image processing model.

[0223] The specific limitations on the image processing device and the image processing model training device can correspond to the limitations on the image processing method and the image processing model training method mentioned above, which will not be repeated here. The various modules in the above-mentioned image processing device and the image processing model training device can be implemented in whole or in part by software, hardware and a combination thereof. The above-mentioned modules can be embedded in or independent of the processor in the computer device in the form of hardware, or can be stored in the memory of the computer device in the form of software, so that the processor can call and execute the operations corresponding to the above modules.

[0224] Fig.11Schematic diagram of the internal structure of an electronic device in an embodiment. The electronic device can be any terminal device such as a mobile phone, a tablet computer, a laptop computer, a desktop computer, a PDA (Personal Digital Assistant), a POS (Point of Sales), a car computer, a wearable device, etc. The electronic device includes a processor and a memory connected via a system bus. Among them, the processor may include one or more processing units. The processor may be a CPU (Central Processing Unit) or a DSP (Digital Signal Processing), etc. The memory may include a non-volatile storage medium and an internal memory. The non-volatile storage medium stores an operating system and a computer program. The computer program can be executed by the processor to implement an image processing method and a training method for an image processing model provided in the following embodiments. The internal memory provides a cache operating environment for the operating system computer program in the non-volatile storage medium.

[0225] The image processing device provided in the embodiments of the present application, and the implementation of each module in the training device of the image processing model can be in the form of a computer program. The computer program can be run on a terminal or a server. The program module constituted by the computer program can be stored in the memory of an electronic device. When the computer program is executed by a processor, the steps of the method described in the embodiments of the present application are implemented.

[0226] The embodiment of the present application also provides a computer-readable storage medium. One or more non-volatile computer-readable storage media containing computer-executable instructions, when the computer-executable instructions are executed by one or more processors, the processors execute the image processing method and the steps of the image processing model training method.

[0227] An embodiment of the present application also provides a computer program product comprising instructions, which, when executed on a computer, enables the computer to execute an image processing method and a method for training an image processing model.

[0228] Any reference to memory, storage, database or other medium used in this application may include non-volatile and / or volatile memory. Non-volatile memory may include ROM (Read-Only Memory), PROM (Programmable Read-only Memory), EPROM (Erasable Programmable Read-Only Memory), EEPROM (Electrically Erasable Programmable Read-only Memory) or flash memory. Volatile memory may include RAM (Random Access Memory), which is used as an external cache memory. By way of illustration and not limitation, RAM is available in many forms, such as SRAM (Static Random Access Memory), DRAM (Dynamic Random Access Memory), SDRAM (Synchronous Dynamic Random Access Memory), Double Data Rate DDRSDRAM (Double Data Rate Synchronous Dynamic Random Access memory), ESDRAM (Enhanced Synchronous Dynamic Random Access memory), SLDRAM (Sync Link Dynamic Random Access Memory), RDRAM (Rambus Dynamic Random Access Memory), and DRDRAM (Direct Rambus Dynamic Random Access Memory).

[0229] The above-mentioned embodiments only express several implementation methods of the present application, and the descriptions thereof are relatively specific and detailed, but they cannot be understood as limiting the scope of the present application. It should be pointed out that, for a person of ordinary skill in the art, several variations and improvements can be made without departing from the concept of the present application, and these all belong to the protection scope of the present application. Therefore, the protection scope of the present application shall be subject to the attached claims.

Claims

1. An image processing method, It is characterized in that include: Inputting the image to be processed into a trained image processing model, wherein the image processing model has tone mapping parameters and noise extraction parameters; Performing convolution processing on the image to be processed based on the tone mapping parameters to obtain a tone template image; Performing convolution processing on the image to be processed based on the noise extraction parameters to obtain a noise template image; By using the image processing model, the tone template image and the image to be processed are fused to obtain a tone mapping image; The tone mapping image is subjected to denoising processing based on the noise template image through the image processing model to obtain a first target image.

2. The method according to claim 1, It is characterized in that The method further comprises: Performing normalization processing on the image to be processed to obtain a normalized image; The step of fusing the tone template image and the image to be processed to obtain a tone mapping image includes: fusing the tone template image and the normalized image to obtain a tone mapping image; The step of performing denoising on the tone mapping image based on the noise template image to obtain a first target image includes: Performing denoising on the tone mapping image based on the noise template image to obtain a denoised image; Perform image restoration processing on the denoised image to obtain a first target image.

3. The method according to claim 2, It is characterized in that The step of performing convolution processing on the image to be processed based on the tone mapping parameters to obtain a tone template image includes: Performing convolution processing on the normalized image based on the tone mapping parameters to obtain a corresponding tone template image; The step of performing convolution processing on the image to be processed based on the noise extraction parameter to obtain a noise template image includes: The normalized image is convolved based on the noise extraction parameters to obtain a corresponding noise template image.

4. The method according to claim 1, It is characterized in that The step of fusing the tone template image and the image to be processed to obtain a tone mapping image includes: Performing pixel multiplication on the tone template image and the image to be processed to obtain a tone mapping image; The step of performing denoising on the tone mapping image based on the noise template image to obtain a first target image includes: The noise template image and the tone mapping image are subjected to pixel addition processing to obtain a first target image.

5. An image processing method, It is characterized in that include: Inputting the image to be processed into a trained image processing model, wherein the image processing model has tone mapping parameters and noise extraction parameters; Performing convolution processing on the image to be processed based on the tone mapping parameters to obtain a tone template image; Performing convolution processing on the image to be processed based on the noise extraction parameters to obtain a noise template image; The image to be processed is subjected to denoising processing based on the noise template image through the image processing model, and the tone template image and the image subjected to denoising processing are fused to obtain a second target image.

6. A training method for an image processing model, It is characterized in that include: Acquire training sample images, and hue label images and noise label images corresponding to the training sample images respectively; Performing convolution processing on the training sample image based on the initial tone mapping parameters of the image processing model to obtain a corresponding tone sample image; Performing convolution processing on the training sample image based on the initial noise extraction parameters of the image processing model to obtain a corresponding noise sample image; Fusing the tone sample image and the training sample image to obtain a tone mapping prediction image; Performing denoising on the tone mapping prediction image according to the noise sample image to obtain a noise prediction image; Based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, the parameters of the image processing model are adjusted and training is continued until the training stop condition is met, thereby obtaining a trained image processing model.

7. The method according to claim 6, It is characterized in that The adjusting the parameters of the image processing model and continuing the training based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, comprises: Based on the difference between the tone mapping prediction image and the corresponding tone label image, the initial tone mapping parameters of the image processing model are adjusted, and based on the difference between the noise prediction image and the corresponding noise label image, the initial noise extraction parameters of the image processing model are adjusted and training continues.

8. The method according to claim 6 or 7, It is characterized in that The acquiring of the training sample images, and the hue label images and noise label images respectively corresponding to the training sample images, comprises: Acquire each frame of training sample image, perform registration processing on each frame of the training sample image, and obtain each registered image; Performing tone mapping processing on each of the registered images, and selecting a tone label image from the tone mapped images; The image processed by tone mapping is subjected to multi-frame denoising to obtain a noisy label image.

9. An image processing device, It is characterized in that include: A module for inputting an image to be processed into a trained image processing model, wherein the image processing model has tone mapping parameters and noise extraction parameters; A convolution module, configured to perform convolution processing on the image to be processed based on the tone mapping parameters to obtain a tone template image; and perform convolution processing on the image to be processed based on the noise extraction parameters to obtain a noise template image; A fusion module, used for fusing the tone template image and the image to be processed through the image processing model to obtain a tone mapping image; The denoising module is used to perform denoising on the tone mapping image based on the noise template image through the image processing model to obtain a first target image.

10. An image processing device, It is characterized in that include: A module for inputting an image to be processed into a trained image processing model, wherein the image processing model has tone mapping parameters and noise extraction parameters; A convolution module, configured to perform convolution processing on the image to be processed based on the tone mapping parameters to obtain a tone template image; and perform convolution processing on the image to be processed based on the noise extraction parameters to obtain a noise template image; The processing module is used to perform noise reduction processing on the image to be processed based on the noise template image through the image processing model, and fuse the tone template image with the image after noise reduction processing to obtain a second target image.

11. A training device for an image processing model, It is characterized in that include: An acquisition module, used to acquire a training sample image, and a tone label image and a noise label image respectively corresponding to the training sample image; A sample convolution module, used for performing convolution processing on the training sample image based on the initial tone mapping parameters of the image processing model to obtain a corresponding tone sample image; Performing convolution processing on the training sample image based on the initial noise extraction parameters of the image processing model to obtain a corresponding noise sample image; A sample fusion module, used for fusing the tone sample image and the training sample image to obtain a tone mapping prediction image; A sample denoising module, configured to perform denoising on the tone mapping prediction image according to the noise sample image to obtain a noise prediction image; An adjustment module is used to adjust the parameters of the image processing model based on the difference between the tone mapping prediction image and the corresponding tone label image, and the difference between the noise prediction image and the corresponding noise label image, and continue training until the training stop condition is met, thereby obtaining a trained image processing model.

12. An electronic device comprising a memory and a processor, wherein the memory stores a computer program. It is characterized in that When the computer program is executed by the processor, the processor is caused to perform the steps of the method according to any one of claims 1 to 8.

13. A computer-readable storage medium having a computer program stored thereon, It is characterized in that When the computer program is executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.

14. A computer program product comprising a computer program / instructions, It is characterized in that When the computer program / instructions are executed by a processor, the steps of the method according to any one of claims 1 to 8 are implemented.